purgeCheck listed a directory's children with includeAll set, so the Drive API
returned trashed children as well as live ones. The only thing that came of
that was the trashedFiles flag, which is used solely to decide whether to trash
the directory or delete it permanently, and when use_trash is on the directory
gets trashed either way.
That made the common case pay for nothing: after trashing N files in a
directory, removing the now empty directory paged through all N trashed entries,
about one API call and pacer delay per 1000, instead of running a single query
that returns nothing.
Ask the server to filter out trashed children when the directory is going to be
trashed anyway. Hard deletes still enumerate them, since there the distinction
decides whether the directory can be removed permanently (#1040), and
--drive-trashed-only is left alone because it deliberately lists trashed items.
Write the git configuration for each test's fake home directory as a
file instead of running three "git config --global" commands, and drop
the "git annex version" invocation from repository setup. This removes
four subprocess launches from each of the fifteen test cases.
The subtests within each end to end test function already run in
parallel, but the three test functions themselves ran one after
another.
Marking the functions parallel lets all their subtests overlap.
The end to end tests exercise rclone via a separate subprocess spawned
by git-annex, and that subprocess is not built with race
instrumentation.
Running them in the race test therefore adds several minutes to CI
without providing any race coverage. The unit tests in this package
still run under the race detector.
Run the full "git annex testremote" suite for a single layout mode and
use "testremote --fast" for the rest.
The full suite repeats the same protocol operations across a matrix of
key sizes and chunk configurations, which exercises client side
git-annex behaviour rather than rclone.
The migration test ran the full "git annex testremote" suite for every
layout mode after verifying the migration with "git annex fsck".
The fsck calls already prove the migrated data is accessible via the
builtin special remote, and TestEndToEnd covers the special remote
protocol with testremote, so the extra five full testremote runs
duplicated coverage at a cost of tens of seconds each on CI.
The cmd/gitannex end to end tests routinely take 230-530s and on slow
CI runners the package exceeds the go test default timeout of 10m.
Raise the per-package timeout to 20m in quicktest and racequicktest to
give slow runners headroom.
Direct lookups of exported Dropbox Paper files retained the
caller-visible extension before export metadata processing appended it
again. Track when metadata was resolved through an export path so the
object keeps the requested remote name while listing behavior remains
unchanged.
The auth proxy was only given the user and their password or public
key, so a proxy program had no way to restrict logins to particular
networks, or to record where an authentication attempt came from.
The JSON sent to the program now has a client_ip key holding the bare
IP the client connected from, with the port stripped so IPv6 arrives
as 2001:db8::1 rather than [2001:db8::1]:52344. An IPv4-mapped IPv6
address is reported as plain IPv4 so that a client arriving over a
dual-stack listener still matches IPv4 networks. The key is omitted
when the client has no IP address.
The IP is also mixed into the backend cache key. That is needed as the
program is only run on a cache miss, so a client from a
non-allowlisted address presenting valid credentials within the 5
minute cache lifetime would get a cache hit and be let in without the
program being consulted at all.
The macOS NFS client sends SETATTR after SYMLINK, which arrives as
Chmod/Chown on the link path. These opened the target with vfs.Open,
which follows symlinks - a freshly created symlink usually dangles, so
the open failed with ENOENT, surfaced to the client as NFS3ERR_IO even
though the link was created.
Add path-based VFS.Chmod and VFS.Chown mirroring VFS.Chtimes. They do
not follow symlinks (lstat semantics, matching VFS.Stat) and return
ENOSYS when the node exists, since the VFS stores neither permissions
nor ownership; serve nfs calls them and masks ENOSYS as before.
Fixes#9627
Previously cross-compile.go ran NumCPU builds in parallel, each of
which ran an unrestricted go build using -p NumCPU internally, giving
up to NumCPU^2 concurrent compile processes and enormous load averages.
Pass -p to each go build, sized so the total parallelism is about
NumCPU, sharing the CPUs between however many builds are actually
selected. This can be overridden with the new -build-p flag.
The local backend built every OS path by joining the root with the source
name converted through the configured encoding, so the encoding was the only
thing keeping a name from turning into path syntax.
With an encoding which omits Dot (Slash, None, Raw) rclone's standard "．．"
decodes back to a real "..", and with an encoding which omits BackSlash a name
like "..\file" becomes a native path on Windows. filepath.Join then resolved
those out of the destination the user chose, so a source object called
"../marker.txt" - an s3 key of "tenant/../marker.txt" listed with the remote
rooted at "tenant", say - created or overwrote a file outside it.
localPath now joins the name to the root and checks with filepath.Rel that the
result is still inside it. localPath is the only place the root is joined to a
name, so threading the error through newObject and newDirectory covers every
operation.
Default configurations were not affected, as encoder.OS includes Dot on all
platforms and BackSlash on Windows.
Fixes GHSA-7p4m-qxvv-g567
A request path beginning with "../" escaped the path the server was
started on, letting a client list, read, create, overwrite and delete
objects outside it.
The check added for CVE-2026-59733 rejected non-canonical paths by
comparing them with path.Clean, but path.Clean cannot resolve leading
".." elements in a relative path so it leaves them in place and the
comparison comes out equal. Only interior traversal such as "a/../../x"
was rejected. Whether a path then escaped depended on the backend:
those which join the root with the remote before encoding it - webdav,
ftp, sftp, http and memory - resolved the ".." away, while local and s3
encode the dot elements first and were unaffected.
A bare "." was accepted for the same reason, which on bucket backends
addresses the served directory's own key.
Validate with io/fs.ValidPath instead, which rejects ".", ".." and empty
elements wherever they appear. The empty path stays valid as the root of
the API, and "." is excluded explicitly because ValidPath accepts it as
the root of an FS.
The VFS calls backends from goroutines of its own. A panic on any of these
cannot be recovered. So a backend panicking on a single file killed the whole
process, taking down a mount or every user of a serve command, even for servers
such as serve http whose library recovers panics raised on its own request
goroutines.
Recover panics at those goroutines and log them with a stack trace. Where the
surrounding code already handles a failure, recover around the backend call
itself rather than the whole goroutine, so a panicking upload is retried like
any other failed upload and a panicking download is reported to the waiters,
instead of abandoning the work part way through and leaving the bookkeeping
inconsistent.
Addresses GHSA-6jcg-q3wp-x2f4
Three ways a single client could deny service to everyone else connected to
the same serve sftp process:
A session "subsystem" request payload is a length-prefixed string, but it was
sliced at a fixed offset without checking its length, so a client sending a
truncated payload panicked the out-of-band request goroutine and killed the
process. Decode the payload instead, the way the neighbouring "exec" request
already does.
Rejecting a request then left the goroutine handling that channel waiting
forever to learn what kind of channel it was, because nothing was ever sent
on the channel it waits on. A client could open channels in a loop making
unsupported requests and grow the server's goroutines and memory without
bound. Signal the waiter when the requests run out so the channel is torn
down.
Separately, the request handlers - and reads, writes and closes on the file
handles they return - run on pkg/sftp packet worker goroutines which have no
panic recovery. A panic raised by a backend while serving one request took the
process down with it. Recover panics at that boundary, log them with a stack
trace, and return them to the requesting client as an error instead.
Addresses GHSA-6jcg-q3wp-x2f4.
The archive backend passes remote .sqfs bytes straight to the go-diskfs squashfs
parser, which does not validate several attacker-controlled superblock and
metadata fields. A crafted image can make the parser panic.
Recover panics at the go-diskfs boundary and return an "invalid or corrupt
squashfs image" error instead. As well as the parse entry points (Read, ReadDir,
OpenFile) this wraps the reader returned by Open, since the parser reads file
data lazily and can panic long after the image opened successfully.
Addresses GHSA-6jcg-q3wp-x2f4.
The builtin authentication compared the configured username and password
with ==, whose run time depends on how much of the value matches, giving a
timing side-channel that could in principle help guess the password.
serve sftp, serve s3 and the auth proxy already use subtle.ConstantTimeCompare
so bring serve ftp in line with them. An empty configured password
still accepts any password.
Addresses GHSA-mfvx-7rcj-9m5g finding 4.
When --cert and --key were supplied TLS was only applied to the listener if
exactly one --addr was given. With two or more --addr flags every listener
without an explicit tls:// prefix silently served cleartext HTTP, so adding a
second --addr to an HTTPS server quietly disabled TLS on both.
Now when TLS is configured every listener serves TLS. An individual listener
can be prefixed with http:// to serve unencrypted HTTP on that address, and
tls:// still marks a listener as TLS explicitly. Using a tls:// address
without --cert and --key is now an error instead of silently serving
cleartext with an https:// URL.
Addresses GHSA-mfvx-7rcj-9m5g finding 3.
With --rc-serve set the root listing enumerated the names of all configured
remotes without any authentication.
Make the root listing obey the same fail-closed rule as the rest of
the rc endpoints: it now requires authentication to be configured or
an explicit opt out with --rc-no-auth.
Addresses GHSA-mfvx-7rcj-9m5g finding 2.
The pprof debug handlers were accessible without authentication disclosing the
process command line (which can carry backend credentials passed on the command
line) and runtime profiles.
Mount the pprof handlers only when when auth is configured or --rc-no-auth was
passed - so they obey the same rule as the rc endpoints.
Addresses GHSA-mfvx-7rcj-9m5g finding 1.
Before this change rclone read a proxy response with http.ReadResponse
over an unrestricted buffered reader. A malicious or compromised
configured proxy, or an active on-path actor controlling a plaintext
HTTP-proxy hop, can grow memory until the process fails.
This fixes the problem by restrincting the read to 1MB maximum.
Before this change, rclone sent stack traces to the client on panic
capture in the rc. Stack traces can leak information which could be
useful to an attacker.
The FTP control channel is line oriented and the ftp library writes
command arguments (paths) straight onto it without escaping, so a
filename containing CR/LF can inject an independent FTP command.
This fix makes sure CR/LF are therefore always encoded to safe symbols
regardless of the configured encoding, which is what the default
encoding already does.
An audit of the shell escaping alongside the PowerShell fix found the
Windows Command Prompt path only rejected the double quote delimiter. A
remote filename containing % or ! can trigger environment variable expansion
even inside double quotes, and a newline or carriage return ends the
command, so a crafted filename could alter the command run during
server-side hashing. Reject paths containing these characters, matching
the existing treatment of the double quote.
PowerShell treats several Unicode smart-quote characters (U+2018, U+2019,
U+201A, U+201B) as single-quote delimiters in addition to the ASCII
apostrophe. The quoting helper only doubled the ASCII apostrophe, so a
remote filename containing one of these could close the quoted path and
inject statements that ran as the SSH account during server-side hashing.
Double all five delimiters when wrapping a PowerShell path so a filename
is always treated as data.
Fixes GHSA-2m8m-jhrm-w6j2
The AWS SDK signs S3 Express (directory bucket) requests with a session
token in the x-amz-s3session-token header. Go's HTTP client treats it as
an ordinary custom header and copies it when following a redirect to
another host, and it was missing from the list of secret headers the
redirect policy strips. Add it to the list.
The redirect tests derived their inputs from the production header list,
so a header accidentally dropped from that list would silently lose test
coverage rather than fail. The test list is now a deliberately literal
copy, kept in sync with the production list by a new test, so removing a
header from either list is a test failure. There is also a new
regression test verifying the Referer header that net/http generates
automatically - which for a presigned request carries the signed query
string - is not forwarded across hosts.
See GHSA-8mxv-9xhp-86h4
The CheckRedirect policy stripped only the X-Amz-Security-Token header when a
redirect crossed a host, leaving other origin-bound secrets exposed:
- Go removes Authorization on a hostname change but not on a same-host scheme
downgrade, so an IBM IAM bearer token was re-sent over plaintext HTTP.
- The SSE-C and copy-source SSE-C customer-key headers (which Go has no reason
to treat as sensitive) were forwarded across a redirect to another host.
- On a cross-host redirect net/http copies the previous request URL into the
Referer header; for a presigned request that URL carries the signature and
session token in its query string, disclosing them to the new origin.
This now refuses outright to follow any HTTPS->HTTP redirect - an S3 endpoint
has no legitimate reason to downgrade the transport, and this closes the
plaintext-exposure class regardless of which header or query string carries the
secret. On a cross-host redirect also strip the known credential headers and the
Referer.
The IDrive e2 region-lookup call posts the access key ID to a fixed HTTPS
endpoint but used a bare http.Client that would follow a redirect downgrading
to plaintext HTTP. Apply the same CheckRedirect policy as the main S3 client so
the request can't be replayed over an unencrypted connection.
Fixes GHSA-8mxv-9xhp-86h4
A server that redirects an HTTPS request to a plaintext HTTP URL on the
same host would cause Go's http.Client to replay the configured
credentials (Basic Authorization, cookies, secret headers) over the
network in cleartext.
Refuse to follow such downgrade redirects by default in lib/rest and wire
the webdav backend's client to use it. The `auth_redirect` option remains
the opt-in escape hatch for servers that legitimately need auth preserved
across redirects.
Fixes GHSA-h4mf-4v27-hggj
When making a release from master, the range since the last minor
release includes all the commits which were cherry-picked to the
previous stable branch and already published in its point releases,
so their changelog lines had to be deleted by hand.
make_changelog.py now finds the previous stable branch (eg
v1.74-stable) from the version being released and skips any commits
released there, detected via cherry-pick -x trailers and git cherry
patch equivalence. Skipped commits are listed on stderr for review.
Releases made from a stable branch are unaffected.
In this commit we attempted to wait for the success report of an
upload to fix the 500 error:
fe78b559d1 yandex: fix 500 errors by waiting for uploads to complete before setting modtime
However Yandex Disk finalizes an upload asynchronously on its servers.
Waiting for the upload operation to report success is not enough -
under load the server reports the operation as successful slightly
before the file is fully finalized, so setting the modification time
straight after an upload can still fail with 500 Internal Server
Error.
Yandex support recommend waiting 1.5s - 3s after the upload before
modifying the file's metadata, so add an --yandex-upload-wait option
(default off) to insert a delay between the upload completing and the
modification time being set.
Cover the portmap registration OpenBSD's kernel NFS client needs (recipe
from hjicks in #8578), the mount_nfs -T requirement, and why the server
advertises AUTH_UNIX, so an OpenBSD user doesn't need to read through the
issue thread to get a working mount.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
alliasgherandNick Craig-Wood
Loi NguyenandGitHub
debadityaandGitHub
am-at-enrollvbandGitHub
SillyZirandNick Craig-Wood
acoeurandGitHub
