Before this change the transport could still be reading the request
body when Call returned, in two ways:
- the goroutine writing a multipart upload form was never stopped or
waited for, so after a failed request it kept reading the file body
while the caller retried
- the transport itself is documented to close the request body,
possibly in a different goroutine after the request has finished,
and hiding the body's Close method removed any way to wait for that
Both of these raced with callers which seek to the start of a pooled
buffer and retry the request, corrupting the retried body, and the
abandoned multipart goroutine could read pooled pages after they had
been returned to the pool.
Call now waits for the transport to finish with the request body
before returning, and the multipart form writer is stopped and waited
for once the request has finished.
NoCloserNotify hides the Close method of the reader passed in like
NoCloser, but calls a notify function (once only) when the returned
body is closed. This lets callers of http.NewRequest find out when the
transport has finished with a request body, as it is documented to
possibly close it in a different goroutine after the request has
finished.
NoCloser hides the Close method of an io.Reader but in doing so it
also hid io.WriterTo if the underlying reader implemented it, forcing
io.Copy to fall back to a buffered Read loop.
Following io.NopCloser, return a variant which forwards WriteTo when
the wrapped reader supports it.
Factor the connection-backed write handle pool out of the smb backend
into a generic lib/filepool.Pool[T] with its own tests, so it can be
reused by other backends that implement fs.OpenWriterAter over a
connection pool.
The smb backend keeps its behaviour, opening and releasing handles
through small closures passed to the pool.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Splainte <r.wycke@hotmail.fr>
*pool.RW implements io.Closer, so when one is used directly as an
http.Request body the transport closes it after each attempt. A retry
which then seeks and re-reads the RW would index the freed pages and
panic with "index out of range" in readPage.
Read, Write, ReadFrom, WriteTo and Seek now return ErrClosed (which
wraps io/fs.ErrClosed) after Close, and Close is safe to call more
than once.
Add response validation for calls made with Range open options. Verify
Content-Range, Content-Length, response status, and the complete
representation size before a backend accepts the response body.
Return a shared sentinel when a server ignores a partial range so callers
can avoid retrying the same unsupported request.
If the source supplied fewer bytes than its declared size, the
multipart upload was completed anyway, storing a truncated object and
reporting a successful upload.
Check the number of bytes read from the source against the declared
size before finalising and abort the upload with an error if they do
not match.
This affects all backends which use lib/multipart for chunked uploads:
azureblob, b2, drime, internxt, oracleobjectstorage, s3 and shade.
This was found by the new FsPutShortEOF and TestRcatSizeShortEOF
integration tests.
The auth proxy was only given the user and their password or public
key, so a proxy program had no way to restrict logins to particular
networks, or to record where an authentication attempt came from.
The JSON sent to the program now has a client_ip key holding the bare
IP the client connected from, with the port stripped so IPv6 arrives
as 2001:db8::1 rather than [2001:db8::1]:52344. An IPv4-mapped IPv6
address is reported as plain IPv4 so that a client arriving over a
dual-stack listener still matches IPv4 networks. The key is omitted
when the client has no IP address.
The IP is also mixed into the backend cache key. That is needed as the
program is only run on a cache miss, so a client from a
non-allowlisted address presenting valid credentials within the 5
minute cache lifetime would get a cache hit and be let in without the
program being consulted at all.
When --cert and --key were supplied TLS was only applied to the listener if
exactly one --addr was given. With two or more --addr flags every listener
without an explicit tls:// prefix silently served cleartext HTTP, so adding a
second --addr to an HTTPS server quietly disabled TLS on both.
Now when TLS is configured every listener serves TLS. An individual listener
can be prefixed with http:// to serve unencrypted HTTP on that address, and
tls:// still marks a listener as TLS explicitly. Using a tls:// address
without --cert and --key is now an error instead of silently serving
cleartext with an https:// URL.
Addresses GHSA-mfvx-7rcj-9m5g finding 3.
Before this change rclone read a proxy response with http.ReadResponse
over an unrestricted buffered reader. A malicious or compromised
configured proxy, or an active on-path actor controlling a plaintext
HTTP-proxy hop, can grow memory until the process fails.
This fixes the problem by restrincting the read to 1MB maximum.
A server that redirects an HTTPS request to a plaintext HTTP URL on the
same host would cause Go's http.Client to replay the configured
credentials (Basic Authorization, cookies, secret headers) over the
network in cleartext.
Refuse to follow such downgrade redirects by default in lib/rest and wire
the webdav backend's client to use it. The `auth_redirect` option remains
the opt-in escape hatch for servers that legitimately need auth preserved
across redirects.
Fixes GHSA-h4mf-4v27-hggj
The rclone web GUI uses the Prefer header for sync/copy operations,
but the CORS middleware's preflight response did not include it in
Access-Control-Allow-Headers. This caused the browser to block
remote GUI copy requests with a CORS policy error:
Request header field prefer is not allowed by
Access-Control-Allow-Headers in preflight response.
Add 'Prefer' to the allowed headers list so remote GUI
access works correctly for file copy operations.
Fixes#9614
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Previously running rclone config (or driving it via the rc API or web
GUI) with -vv would write secrets to the debug log.
This was dangerous as users debugging a failing config flow often
paste their -vv logs into the forum or GitHub issues.
These values are now redacted from the log as "XXX". Values whose
option is known are only redacted if the option is marked IsPassword
or Sensitive, so normal answers remain visible.
Use --dump auth to see the unredacted values when debugging a config
flow - rclone prints a warning that secrets will appear in the log
when this is in effect.
This was discovered by CodeQL: https://github.com/rclone/rclone/security/code-scanning/182
rclone's shared Google Drive and Google Photos client_id is being
retired and will stop working during 2026. When creating a new remote
that would use it, the config wizard now warns the user and asks the
user to enter their own client_id and secret instead. Service account
and environment auth are unaffected as they don't use the shared
client_id.
See: https://forum.rclone.org/t/google-drive-and-google-photos-users-action-required/54005
The shared Google Drive and Google Photos client_id is being retired and
will stop working during 2026. Warn users who rely on it (ie who have not
configured their own client_id) so they can create their own in advance.
The warning is only shown for auth flows that actually use the shared
client_id, not for service account, environment or anonymous auth.
See: https://forum.rclone.org/t/google-drive-and-google-photos-users-action-required/54005
A Range header requesting a suffix longer than the object (e.g.
"bytes=-90407" against a 5 byte object) caused RangeOption.Decode to
compute a negative offset (size - End), which serve.Object then used
directly as a slice/seek offset and panicked with "slice bounds out of
range". FixRangeOption (used by backends like OneDrive/Box that lack
native suffix-range support) had the same root cause: it produced a
RangeOption with a negative Start, which Header() silently dropped,
turning the request into the wrong byte range instead of erroring or
serving the whole object.
Per RFC 7233 section 2.1, when the suffix-length exceeds the
representation size, the entire representation should be served.
Clamp the computed offset/start to 0 in both places.
Fixes#6310
rclone starts a http listener if the user is adding a remote that requires
oauth. The problem is that sometimes this breaks or the user stops midway with
no way of us knowing that. Once that happens, clients of the api break. In the
best scenario, new remotes cannot be configured without a restart.
This change adds 2 oauth related endpoints, one to get the status of the listener
and one to stop it. This allows clients to go back to normal without having to
restart rclone.
- config/oauthstop - cancels an in-progress OAuth flow and releases the port.
- config/oauthstatus - reports "running" / "stopped".
The golang.org/x/net/http2/h2c package was deprecated in v0.54.0 in
favour of setting the http.Server Protocols field to enable unencrypted
HTTP/2.
This replaces the h2c.NewHandler wrapping added in e863f751f with
http.Server.Protocols, which is supported by the standard library
since Go 1.24.
Note that the stdlib only supports HTTP/2 prior-knowledge on cleartext
connections, not HTTP/1.1 Upgrade: h2c negotiation. In practice clients
use prior-knowledge or require TLS, so this should not affect users.
The previous wording "Already have a token - refresh?" was misleading
because answering yes triggers a full re-authorization flow, not an
OAuth2 refresh token grant. Updated to "Token already configured -
replace it?" to accurately describe what happens.
Also updated the SugarSync backend which has its own copy of the prompt,
and the docs for box, drive, and onedrive that reference it.
URLPathEscapeAll was only passing [A-Za-z0-9/] through unencoded, causing
it to percent-encode RFC 3986 unreserved characters (-, ., _, ~). Per RFC
3986 §2.3, unreserved characters MUST NOT be percent-encoded, and a URI
that unnecessarily encodes them is not equivalent to one that does not.
Servers that perform strict path matching without normalising
percent-encoded characters will reject the over-encoded form with a 404.
Before: /files/my-report.pdf → /files/my%2Dreport%2Epdf
After: /files/my-report.pdf → /files/my-report.pdf
Reserved characters (spaces, semicolons, colons, etc.) continue to be
encoded as before.
Before this change server side copies would show at 0% until they were
done then show at 100%.
With support from the backend, server side copies can now be accounted
in real time. This will only work for backends which have been
modified and themselves get feedback about how copies are going.
If the transfer fails, the bytes accounted will be reversed.
Add gzip compression for directory listings and text assets served over HTTP.
This reduces the rclone repository file listing from 40 kB to 8 kB and reduces
the rclone MANUAL.txt from 2.7 MB to 700 kB.
This makes listings and assets served across the network load faster.
The compression level of 5 should be a good balance between size and speed.
The WebDAV implementation already permits redirects on PROPFIND for
listing paths in the `listAll` method but does not permit this for
metadata in `readMetaDataForPath`. This results in a strange experience
for endpoints that heavily use redirects -
```
rclone lsl endpoint:
```
functions and lists `hello_world.txt` in its output but
```
rclone lsl endpoint:hello_world.txt
```
Fails with a HTTP 307.
The git history for this setting indicates this was done to avoid
an issue where redirects cause a verb change to GET in the Go HTTP
client; it does not appear to be problematic with HTTP 307.
To fix, a new `CheckRedirect` function is added in the `rest` library
to force the client to use the same verb across redirects, forcing this
for the PROPFIND case.
Before this change we read sleepTime before acquiring the pacer token
and uses that possibly stale value to schedule the token return. When
many goroutines enter while sleepTime is high (e.g., 10s), each
goroutine caches this 10s value. Even if successful calls rapidly
decay the pacer state to 0, the queued goroutines still schedule 10s
token returns, so the queue drains at 1 req/10s for the entire herd.
This can create multi‑minute delays even after the pacer has dropped
to 0.
After this change we refresh the sleep time after getting the token.
This problem was introduced by the desire to skip reading the pacer
token entirely when sleepTime is 0 in high performance backends (eg
s3, azure blob).
It was possible in the presence of --max-connections and recursive
calls to the pacer to deadlock it leaving all connections waiting on
either a max connection token or a pacer token.
This fixes the problem by making sure we return the pacer token on
schedule if we take it.
This also short circuits the pacer token if sleepTime is 0.
* Adds "aix/ppc64" to the cross-compile target list.
* Including AIX in the build tag of "metadata_other.go".
* Excluding AIX from the main ncdu build tags.
* Marking AIX as an unsupported platform for ncdu.
* Excluding AIX from the fallback redirect implementation.
* Excluding AIX from unix build tags to avoid undefined unix.WNOHANG.
Nick Craig-Wood
Sune MølgaardandNick Craig-Wood
VXNCXNXandNick Craig-Wood
MoraxandNick Craig-Wood
Loi NguyenandNick Craig-Wood
am-at-enrollvbandGitHub
Amit MishraandNick Craig-Wood
Hakan İSMAİLandGitHub
FTCHDandGitHub
KTibowandNick Craig-Wood
albertony
TheBabuandNick Craig-Wood
Andrew FurmanandGitHub
Leon BrocardandGitHub
Brian BockelmanandNick Craig-Wood
Romāns PotašovsandNick Craig-Wood
yyandGitHub
Nicolas DessartandNick Craig-Wood
dougalandNick Craig-Wood
Lakshmi-SurekhaandGitHub