The dataflow analysis behind SA4023, new in the staticcheck 0.8.0
bundled with golangci-lint v2.13.0, makes linting large packages more
than 10x slower (89s vs 7s for backend/s3 alone) which took the CI
lint job past its 30 minute limit. golangci-lint no longer enforces
its run timeout during analysis so the job ran until cancelled, and
the cancellation meant the lint cache was never saved, making every
subsequent run cold and guaranteeing the timeout repeated.
The check also produces false positives (eg claiming operations.Delete
never returns nil).
This commit removes the workings of the old web ui which hasn't been
maintained for 6 years. If users supply --rc-web-gui then rclone will
exit with an error pointing users at the maintained `rclone gui`
command.
Five of the advisories released with v1.75.0 now have CVEs assigned:
- GHSA-45pq-889g-fcgh serve restic path traversal: CVE-2026-71309
- GHSA-xhf4-832v-7xcr lib/proxy CONNECT header OOM: CVE-2026-71310
- GHSA-8c48-q9wj-3w37 ftp command injection: CVE-2026-71311
- GHSA-2m8m-jhrm-w6j2 sftp PowerShell command injection: CVE-2026-71312
- GHSA-7p4m-qxvv-g567 local file name escape: CVE-2026-71313
GHSA-6jcg-q3wp-x2f4 (squashfs) loses its CVE-PENDING marker as GitHub
declined to issue a CVE from the rclone repository - the vulnerable code
is in go-diskfs so any CVE must come from an advisory there.
GHSA-mfvx-7rcj-9m5g (pprof) keeps its CVE-PENDING marker as the CVE
request is still awaiting allocation.
Before this change, when no_data_encryption was set, uploads from
local disk advertised the hash of the encrypted data even though the
data was uploaded unencrypted.
On backends which check upload hashes (eg b2) this made uploads of
small files fail with errors like "Checksum did not match data
received", and made chunked uploads store an incorrect hash so the
files failed their checksum on download with "corrupted on transfer:
SHA1 hashes differ".
See: https://forum.rclone.org/t/sha1-mismatches-on-b2-with-no-data-encryption-true/54121
A client which started a multipart upload and vanished without either
completing or aborting it used to hold on to its resources forever.
Incomplete multipart uploads which have had no activity for
--multipart-expiry (default 24h) are now aborted and cleaned up
exactly as if the client had called AbortMultipartUpload, with a
NOTICE logged.
An upload with a part still being received is never expired, and each
completed part restarts the clock. Late operations on an expired
upload fail with NoSuchUpload, as they do on real S3 when a lifecycle
rule has aborted the upload.
Set --multipart-expiry 0 to keep incomplete uploads forever.
Multipart uploads used to be streamed directly to the remote with their
own PutStream machinery, bypassing the VFS, a design left over from
before the VFS could abandon a streaming write.
They are now written through the VFS exactly like plain object PUTs in
every cache mode. The parts are written, in part-number order, to a
temporary object which is renamed into place server-side on
completion.
With the default --vfs-cache-mode off the parts stream through the VFS
to the remote as they arrive. With --vfs-cache-mode writes or above
they are buffered in the VFS cache and uploaded by its write-back.
User visible changes:
- Multipart uploads now show in rclone's transfer stats and obey
--bwlimit (previously they bypassed both).
- Remotes without streaming upload support now spool the upload to a
temporary file on local disk instead of buffering it in memory.
- Multipart uploads are never buffered in memory because of missing
remote capabilities - only --disable-multipart-streaming does that.
- Remotes that upload atomically now also write to a temporary object
renamed into place, so an in-progress multipart upload is no longer
briefly visible under its final key.
- On the few remotes with no server-side move or copy the parts are
written straight to the final object in all cache modes.
- With --vfs-cache-mode writes, plain PUTs and multipart uploads to the
same key go through the same cache entry, so an earlier PUT still in
the write-back window can no longer be written back over a newer
multipart upload.
- Failed write-backs are retried by the VFS without the client having
to restart the upload, and completed objects are served from the
cache for read-after-write.
The temporary objects that uploads are written to before being renamed
into place are now named .rclone_temp_put_* and .rclone_temp_multipart_*,
and the whole .rclone_temp_ prefix is reserved: any object whose name
starts with it is hidden from S3 listings. This gives a single pattern
for cleaning up leftovers from killed servers:
rclone delete --min-age 24h --include ".rclone_temp_*" remote:path
The .rclone_multipart_upload_* objects rclone v1.75 used are still
hidden from listings so leftovers from an older server stay invisible
to S3 clients.
The mtime metadata fallback was nested inside the X-Amz-Meta-Mtime
branch, so it only ran when X-Amz-Meta-Mtime was present but invalid -
and then set the modtime from the invalid value's failed parse rather
than parsing mtime. An object PUT with only mtime metadata kept the
upload time as its modtime.
Now the two keys are checked independently, as TouchObject already
does.
streamPart did not check whether the upload had been torn down, but
AbortMultipartUpload sets the reorder buffer map to nil, so an abort
arriving while a part body was still being received panicked with an
assignment to a nil map once the part was buffered.
Now a part whose upload has been aborted or completed under it is
rejected with NoSuchUpload.
A PUT which failed part way through removed the object at the
destination key. As well as differing from real S3 (where a failed PUT
never affects the stored object), this raced with the client's
automatic retry of the same PUT: the retry stored the object and
returned 200 OK, then the failed first attempt's cleanup deleted it,
silently losing an acknowledged upload. The interrupted upload could
also be committed as a truncated object, since closing the write handle
gave the streaming upload a clean end of stream.
Now a failed or interrupted PUT never disturbs the object at the key:
- The object at the key is never removed on error.
- On backends where a partial upload is visible at its final name
(PartialUploads), and when the VFS cache mode is writes or above, the
upload is written to a temporary object which is renamed into place
on success and removed on failure, as streamed multipart uploads
already do. Backends which upload atomically are still streamed
straight to the destination.
- An interrupted or short body fails the upload via
WriteFileHandle.CloseWithError instead of committing truncated data,
and a body which ends cleanly short of its declared size is rejected
with IncompleteBody.
Closing a streaming write handle sends a clean EOF to the backend
upload, so a writer which knows its data source failed part way through
had no way to stop the truncated file being stored as if it were
complete.
CloseWithError closes the handle failing the upload with the given
reason instead. EOF-like reasons are rewritten so the layers reading
the pipe can't mistake them for a clean end of stream and commit the
partial file.
Needed by serve s3 to abort interrupted PUTs - see #9718
If the source supplied fewer bytes than its declared size, the upload
request failed but a retry could report success even though the stored
file was truncated, because the retry re-sent an already exhausted
reader.
Count the bytes actually read from the source and if they do not match
the declared size return an error.
This was found by the new FsPutShortEOF integration test.
If the source supplied fewer bytes than its declared size, the upload
request failed but a retry could report success even though the stored
file was truncated, because the retry re-sent an already exhausted
reader.
Count the bytes actually read from the source and if they do not match
the declared size, remove the partially uploaded file and return an
error.
This was found by the new FsPutShortEOF integration test.
If the source supplied fewer bytes than its declared size, the
multipart upload was completed anyway, storing a truncated file and
reporting a successful upload.
Check the number of bytes read from the source against the declared
size before finalising and abort the upload with an error if they do
not match.
This was found by the new FsPutShortEOF integration test.
If the source supplied fewer bytes than its declared size, the
multipart upload was completed anyway, storing a truncated object and
reporting a successful upload.
Check the number of bytes read from the source against the declared
size before finalising and abort the upload with an error if they do
not match.
This affects all backends which use lib/multipart for chunked uploads:
azureblob, b2, drime, internxt, oracleobjectstorage, s3 and shade.
This was found by the new FsPutShortEOF and TestRcatSizeShortEOF
integration tests.
If the source supplied fewer bytes than its declared size, the
truncated file was stored and the upload reported success with the
object claiming the declared size.
Count the bytes actually read from the source and if they do not match
the declared size, remove the truncated file and return an error.
This was found by the new TestRcatSizeShortEOF integration test.
The file is created at the declared size and the data then written
with ranged writes, so if the source supplied fewer bytes than
declared, the remainder of the file was left as zeroes and the upload
reported success.
Count the bytes actually read from the source and if they do not match
the declared size, delete the partially uploaded file (if newly
created) and return an error.
This was found by the new FsPutShortEOF and TestRcatSizeShortEOF
integration tests.
The precision field in the backend overview YAML files can hold
fs.ModTimeNotSupported (100 years in nanoseconds) which overflows int
on 32 bit platforms, making the YAML for those backends fail to parse
and causing rclone to log 18 internal errors on every invocation.
Use int64 for the precision field and add a test that parses every
embedded backend YAML file so this is caught on 32 bit test runs.
When an upload failed part way through with a retryable error (eg a
502 from the block storage servers) the pacer retried the whole upload
call with the same input stream. The stream had already been partially
consumed, so the retry re-created the upload draft and committed just
the remainder of the stream as a complete file, silently truncating
it. With restic over serve restic this corrupted the repository as the
truncated pack was reported as successfully uploaded.
This fixes it by using CallNoRetry for the upload, as the other
backends do, so retryable errors are returned wrapped in a RetryError
for the caller to retry the upload with a fresh stream.
Uploads through RcatSize with a known size - used by rcat --size, the
rc operations/uploadfile and the serve backends, eg serve restic - did
not check the size of the uploaded object. If the source stream ended
before the declared size worth of data had been read, the truncated
object was reported as a successful upload. This could corrupt data
for callers which trust the result, eg a restic repository accessed
via serve restic (see #9722).
This adds the same size check operations.Copy performs after a copy,
respecting --ignore-size and backends which do not report sizes.
When the source supplied fewer bytes than its declared size, the
compressed data file was stored under a name containing the declared
size while the metadata recorded the actual number of bytes read.
NewObject looks the data file up by the size in the metadata, so the
resulting object could never be read again, and the upload reported
success.
Check that the number of bytes read matches the declared size after
uploading the data and before writing the metadata, and remove the
data file and return an error if it does not.
This was found by the new FsPutShortEOF integration test.
The append loop retries everything once the upload session has
started, so a cancelled context error was retried through all the low
level retries with exponential backoff before the upload gave up.
A source which returned EOF before supplying as many bytes as it
declared would either commit a truncated file (if the shortfall was
within the final chunk) or loop forever appending empty chunks to the
upload session. Return an error wrapping io.ErrUnexpectedEOF instead.
Note that all dropbox uploads use the chunked upload path with the
default batch_mode of sync, so this affected uploads of every size.
A source reader which returns EOF before supplying as many bytes as it
declared should not cause a backend Put to hang, panic or create an
object claiming the declared size.
The layers above detect a truncated transfer by comparing sizes after
the upload, so a backend may either return an error or create a
shorter object.
See #9704
Write the git configuration for each test's fake home directory as a
file instead of running three "git config --global" commands, and drop
the "git annex version" invocation from repository setup. This removes
four subprocess launches from each of the fifteen test cases.
The subtests within each end to end test function already run in
parallel, but the three test functions themselves ran one after
another.
Marking the functions parallel lets all their subtests overlap.
The end to end tests exercise rclone via a separate subprocess spawned
by git-annex, and that subprocess is not built with race
instrumentation.
Running them in the race test therefore adds several minutes to CI
without providing any race coverage. The unit tests in this package
still run under the race detector.
Run the full "git annex testremote" suite for a single layout mode and
use "testremote --fast" for the rest.
The full suite repeats the same protocol operations across a matrix of
key sizes and chunk configurations, which exercises client side
git-annex behaviour rather than rclone.
The migration test ran the full "git annex testremote" suite for every
layout mode after verifying the migration with "git annex fsck".
The fsck calls already prove the migrated data is accessible via the
builtin special remote, and TestEndToEnd covers the special remote
protocol with testremote, so the extra five full testremote runs
duplicated coverage at a cost of tens of seconds each on CI.
The cmd/gitannex end to end tests routinely take 230-530s and on slow
CI runners the package exceeds the go test default timeout of 10m.
Raise the per-package timeout to 20m in quicktest and racequicktest to
give slow runners headroom.
Previously cross-compile.go ran NumCPU builds in parallel, each of
which ran an unrestricted go build using -p NumCPU internally, giving
up to NumCPU^2 concurrent compile processes and enormous load averages.
Pass -p to each go build, sized so the total parallelism is about
NumCPU, sharing the CPUs between however many builds are actually
selected. This can be overridden with the new -build-p flag.
The local backend built every OS path by joining the root with the source
name converted through the configured encoding, so the encoding was the only
thing keeping a name from turning into path syntax.
With an encoding which omits Dot (Slash, None, Raw) rclone's standard "．．"
decodes back to a real "..", and with an encoding which omits BackSlash a name
like "..\file" becomes a native path on Windows. filepath.Join then resolved
those out of the destination the user chose, so a source object called
"../marker.txt" - an s3 key of "tenant/../marker.txt" listed with the remote
rooted at "tenant", say - created or overwrote a file outside it.
localPath now joins the name to the root and checks with filepath.Rel that the
result is still inside it. localPath is the only place the root is joined to a
name, so threading the error through newObject and newDirectory covers every
operation.
Default configurations were not affected, as encoder.OS includes Dot on all
platforms and BackSlash on Windows.
Fixes GHSA-7p4m-qxvv-g567
A request path beginning with "../" escaped the path the server was
started on, letting a client list, read, create, overwrite and delete
objects outside it.
The check added for CVE-2026-59733 rejected non-canonical paths by
comparing them with path.Clean, but path.Clean cannot resolve leading
".." elements in a relative path so it leaves them in place and the
comparison comes out equal. Only interior traversal such as "a/../../x"
was rejected. Whether a path then escaped depended on the backend:
those which join the root with the remote before encoding it - webdav,
ftp, sftp, http and memory - resolved the ".." away, while local and s3
encode the dot elements first and were unaffected.
A bare "." was accepted for the same reason, which on bucket backends
addresses the served directory's own key.
Validate with io/fs.ValidPath instead, which rejects ".", ".." and empty
elements wherever they appear. The empty path stays valid as the root of
the API, and "." is excluded explicitly because ValidPath accepts it as
the root of an FS.
The VFS calls backends from goroutines of its own. A panic on any of these
cannot be recovered. So a backend panicking on a single file killed the whole
process, taking down a mount or every user of a serve command, even for servers
such as serve http whose library recovers panics raised on its own request
goroutines.
Recover panics at those goroutines and log them with a stack trace. Where the
surrounding code already handles a failure, recover around the backend call
itself rather than the whole goroutine, so a panicking upload is retried like
any other failed upload and a panicking download is reported to the waiters,
instead of abandoning the work part way through and leaving the bookkeeping
inconsistent.
Addresses GHSA-6jcg-q3wp-x2f4
Three ways a single client could deny service to everyone else connected to
the same serve sftp process:
A session "subsystem" request payload is a length-prefixed string, but it was
sliced at a fixed offset without checking its length, so a client sending a
truncated payload panicked the out-of-band request goroutine and killed the
process. Decode the payload instead, the way the neighbouring "exec" request
already does.
Rejecting a request then left the goroutine handling that channel waiting
forever to learn what kind of channel it was, because nothing was ever sent
on the channel it waits on. A client could open channels in a loop making
unsupported requests and grow the server's goroutines and memory without
bound. Signal the waiter when the requests run out so the channel is torn
down.
Separately, the request handlers - and reads, writes and closes on the file
handles they return - run on pkg/sftp packet worker goroutines which have no
panic recovery. A panic raised by a backend while serving one request took the
process down with it. Recover panics at that boundary, log them with a stack
trace, and return them to the requesting client as an error instead.
Addresses GHSA-6jcg-q3wp-x2f4.
The archive backend passes remote .sqfs bytes straight to the go-diskfs squashfs
parser, which does not validate several attacker-controlled superblock and
metadata fields. A crafted image can make the parser panic.
Recover panics at the go-diskfs boundary and return an "invalid or corrupt
squashfs image" error instead. As well as the parse entry points (Read, ReadDir,
OpenFile) this wraps the reader returned by Open, since the parser reads file
data lazily and can panic long after the image opened successfully.
Addresses GHSA-6jcg-q3wp-x2f4.
The builtin authentication compared the configured username and password
with ==, whose run time depends on how much of the value matches, giving a
timing side-channel that could in principle help guess the password.
serve sftp, serve s3 and the auth proxy already use subtle.ConstantTimeCompare
so bring serve ftp in line with them. An empty configured password
still accepts any password.
Addresses GHSA-mfvx-7rcj-9m5g finding 4.
When --cert and --key were supplied TLS was only applied to the listener if
exactly one --addr was given. With two or more --addr flags every listener
without an explicit tls:// prefix silently served cleartext HTTP, so adding a
second --addr to an HTTPS server quietly disabled TLS on both.
Now when TLS is configured every listener serves TLS. An individual listener
can be prefixed with http:// to serve unencrypted HTTP on that address, and
tls:// still marks a listener as TLS explicitly. Using a tls:// address
without --cert and --key is now an error instead of silently serving
cleartext with an https:// URL.
Addresses GHSA-mfvx-7rcj-9m5g finding 3.
With --rc-serve set the root listing enumerated the names of all configured
remotes without any authentication.
Make the root listing obey the same fail-closed rule as the rest of
the rc endpoints: it now requires authentication to be configured or
an explicit opt out with --rc-no-auth.
Addresses GHSA-mfvx-7rcj-9m5g finding 2.
The pprof debug handlers were accessible without authentication disclosing the
process command line (which can carry backend credentials passed on the command
line) and runtime profiles.
Mount the pprof handlers only when when auth is configured or --rc-no-auth was
passed - so they obey the same rule as the rc endpoints.
Addresses GHSA-mfvx-7rcj-9m5g finding 1.
Before this change rclone read a proxy response with http.ReadResponse
over an unrestricted buffered reader. A malicious or compromised
configured proxy, or an active on-path actor controlling a plaintext
HTTP-proxy hop, can grow memory until the process fails.
This fixes the problem by restrincting the read to 1MB maximum.
Before this change, rclone sent stack traces to the client on panic
capture in the rc. Stack traces can leak information which could be
useful to an attacker.
The FTP control channel is line oriented and the ftp library writes
command arguments (paths) straight onto it without escaping, so a
filename containing CR/LF can inject an independent FTP command.
This fix makes sure CR/LF are therefore always encoded to safe symbols
regardless of the configured encoding, which is what the default
encoding already does.
An audit of the shell escaping alongside the PowerShell fix found the
Windows Command Prompt path only rejected the double quote delimiter. A
remote filename containing % or ! can trigger environment variable expansion
even inside double quotes, and a newline or carriage return ends the
command, so a crafted filename could alter the command run during
server-side hashing. Reject paths containing these characters, matching
the existing treatment of the double quote.
PowerShell treats several Unicode smart-quote characters (U+2018, U+2019,
U+201A, U+201B) as single-quote delimiters in addition to the ASCII
apostrophe. The quoting helper only doubled the ASCII apostrophe, so a
remote filename containing one of these could close the quoted path and
inject statements that ran as the SSH account during server-side hashing.
Double all five delimiters when wrapping a PowerShell path so a filename
is always treated as data.
Fixes GHSA-2m8m-jhrm-w6j2
The AWS SDK signs S3 Express (directory bucket) requests with a session
token in the x-amz-s3session-token header. Go's HTTP client treats it as
an ordinary custom header and copies it when following a redirect to
another host, and it was missing from the list of secret headers the
redirect policy strips. Add it to the list.
The redirect tests derived their inputs from the production header list,
so a header accidentally dropped from that list would silently lose test
coverage rather than fail. The test list is now a deliberately literal
copy, kept in sync with the production list by a new test, so removing a
header from either list is a test failure. There is also a new
regression test verifying the Referer header that net/http generates
automatically - which for a presigned request carries the signed query
string - is not forwarded across hosts.
See GHSA-8mxv-9xhp-86h4
The CheckRedirect policy stripped only the X-Amz-Security-Token header when a
redirect crossed a host, leaving other origin-bound secrets exposed:
- Go removes Authorization on a hostname change but not on a same-host scheme
downgrade, so an IBM IAM bearer token was re-sent over plaintext HTTP.
- The SSE-C and copy-source SSE-C customer-key headers (which Go has no reason
to treat as sensitive) were forwarded across a redirect to another host.
- On a cross-host redirect net/http copies the previous request URL into the
Referer header; for a presigned request that URL carries the signature and
session token in its query string, disclosing them to the new origin.
This now refuses outright to follow any HTTPS->HTTP redirect - an S3 endpoint
has no legitimate reason to downgrade the transport, and this closes the
plaintext-exposure class regardless of which header or query string carries the
secret. On a cross-host redirect also strip the known credential headers and the
Referer.
The IDrive e2 region-lookup call posts the access key ID to a fixed HTTPS
endpoint but used a bare http.Client that would follow a redirect downgrading
to plaintext HTTP. Apply the same CheckRedirect policy as the main S3 client so
the request can't be replayed over an unencrypted connection.
Fixes GHSA-8mxv-9xhp-86h4
A server that redirects an HTTPS request to a plaintext HTTP URL on the
same host would cause Go's http.Client to replay the configured
credentials (Basic Authorization, cookies, secret headers) over the
network in cleartext.
Refuse to follow such downgrade redirects by default in lib/rest and wire
the webdav backend's client to use it. The `auth_redirect` option remains
the opt-in escape hatch for servers that legitimately need auth preserved
across redirects.
Fixes GHSA-h4mf-4v27-hggj
When making a release from master, the range since the last minor
release includes all the commits which were cherry-picked to the
previous stable branch and already published in its point releases,
so their changelog lines had to be deleted by hand.
make_changelog.py now finds the previous stable branch (eg
v1.74-stable) from the version being released and skips any commits
released there, detected via cherry-pick -x trailers and git cherry
patch equivalence. Skipped commits are listed on stderr for review.
Releases made from a stable branch are unaffected.
Nick Craig-Wood