CloudKit is inconsistent about how it encodes the isDeleted field on
album records, returning a JSON boolean (true/false) for some accounts
and a number (0/1) for others. The numeric form caused listing of a
photo library to fail with:
json: cannot unmarshal number into Go struct field
ckBoolField.records.fields.isDeleted.value of type bool
The encoding also varies over time, not just per account: a full HTTP
dump from the reporting user showed the server sending
"isDeleted" : { "value" : 0, "type" : "INT64" }
but the same account later reverted to the boolean encoding with no
client change. Asset records already deliver isDeleted as a number, so
both encodings are in active use server side and either may appear.
Accept both encodings when parsing CloudKit boolean fields.
See: https://forum.rclone.org/t/error-when-trying-to-list-contents-of-primarysync-directory-in-icloud-photos/54028
The session cache was only populated on a fresh username and password
login, so once a session ID was stored in the config every Fs
instance created its own Mega session with its own copy of the
account's node tree. The server side move code relies on all Fs
instances of a user sharing one session, and with separate sessions a
move between two rclone remotes grafted a node from one tree into
another, where the asynchronous event replays of the two sessions
raced and could detach the moved file from the destination directory
so it disappeared from listings.
Cache the session however the login was done. This also stops every
extra Fs instance re-downloading the whole account node tree.
The pacer callbacks only retried 429 responses, so transient server
errors like 502 Bad Gateway failed transfers immediately. Retry the
standard retryable status codes and transport errors everywhere, make
chunk upload retries resend the whole chunk, and fix a potential nil
pointer dereference when logging a bad token response.
The go-diskfs library now requires io/fs.ValidPath style paths (no
leading slash, "." for the root) so convert paths at the library
boundary. This fixes listing failures and the resulting test hangs
after the update to go-diskfs v1.9.3.
This fixes the code to compile with the updated dependencies:
- squashfs: implement the new Path method required by the go-diskfs
backend.Storage interface, returning an empty string as there is no
underlying path.
- dropbox: convert to and from the SDK's new DBXTime type (an alias
for time.Time) for ClientModified, TimeInvited and Expires.
- dropbox: remove a stray debug fmt.Printf from the shared folder
listing.
- pin go-systemd to v22.6.0 to fix netbsd builds
go-systemd v22.7.0 uses unix.ClockGettime and unix.CLOCK_MONOTONIC
which golang.org/x/sys does not define for netbsd, so the build fails.
See: https://github.com/coreos/go-systemd/issues/512
- pin google.golang.org/grpc to v1.80.0 to fix plan9 builds
grpc v1.81.0 and later use syscall.Errno, syscall.ECONNRESET and
syscall.ECONNABORTED which do not exist on plan9, so the build fails
See: https://github.com/grpc/grpc-go/issues/9253
The /file-entries/duplicate endpoint returns a 500 Server Error for
every request (reported to Drime 2026-06) which made all server side
copies fail after 10 retries. Remove the Copy method so rclone falls
back to downloading and re-uploading instead. It can be restored if
Drime fix the endpoint.
Microsoft has been progressively disabling sharing invitations on
both Business and Personal accounts - the driveItem invite API
returns 400 sharingFailed for any recipient on affected accounts -
which makes the permission writing tests impossible. Probe the API
once and skip the tests which need it when it is refused.
Open decoded range options against an unknown size, producing a
negative offset for suffix ranges (eg the last N bytes) and sending
syntactically invalid Range headers which the server ignored. A
suffix range then returned the whole file instead of the requested
tail.
Decode the options against the known object size so the offset is
always absolute, only send a Range header when one was requested, and
honour the requested count when the server ignores the Range header.
The Linkbox API is fronted by bot protection which under load
intermittently returns an HTML challenge page with a 200 status
instead of JSON. This surfaced as a fatal JSON decode error, and the
web API path misread the empty decoded result as an expired token,
hammering the login endpoint with requests which also failed to
decode and keeping the bot protection triggered.
Treat responses which fail to decode as retryable errors so the pacer
backs off until the block lifts, and only refresh the token when the
response actually parsed as JSON and reported an error.
Yandex Disk stores the modtime in a custom property set just after
the upload completes. The server sometimes silently drops this
property (the request returns success but the property is gone when
read back), leaving the object showing the upload time instead.
Check the modtime read back after upload and set it again if it
didn't stick.
put.io serializes trash operations per account so concurrent deletes
can fail with a 400 TRASH_LOCK_TIMEOUT error saying "There is an
ongoing blocking trash operation". This transient error was treated
as fatal - retry it instead.
Objects looked up by name used the /folder/itembyname.json endpoint
which does not return the file's MD5 hash, so NewObject returned
objects with no hash. Read the metadata with /file/info.json (which
does return the hash) instead, using itembyname only to find the file
id when it isn't known.
Copy and Move returned objects with only the id and size filled in,
leaving the modtime and hash empty, and SetModTime kept nanosecond
precision in memory while the server stores seconds. These made the
returned objects differ from a fresh listing of the same objects.
The Move method compared the source and destination paths relative to
their own Fs roots, so a server side move between two differently
rooted Fs instances on the same remote (as used by --backup-dir and
sync moves between remotes) was skipped as "already there" whenever
the relative paths coincided. With --backup-dir this destroyed the
file which was supposed to be backed up.
Compare the full paths including the Fs roots instead.
The Zoho config system ended every interactive create, update and
reconnect by calling m.Set("root_folder_id", workspaceID) in the
workspace_end state, with fs.ConfigChoose defaulting to the first
workspace. Any existing root_folder_id was therefore overwritten and
the remote silently repointed to the first workspace root.
This matters because reconnect is the documented fix for the 401
INVALID_OAUTHSCOPE download error - tokens issued before the
ZohoFiles.files.ALL scope was added lack download access - so users are
told to reconnect and then find all subsequent list/copy/sync/delete
operations pointed at a different, often shared, workspace.
Gate the workspace selection the way the drive backend does for team
drives (#5454): if a root_folder_id is already set, ask "Change current
root folder id ...?" defaulting to No and keep it; only run workspace
selection when it is empty or the user opts in. The token type rewrite
still runs on every reconnect so the scope refresh is unaffected. Also
expose root_folder_id as a standard advanced option (Sensitive, so
config redacted masks it) so it can be set and discovered like on
drive/box/onedrive.
Fixes#9575
The server stores the SMB last write time with 100 ns (FILETIME)
precision and returns it in full in listings, so advertise that as
the precision instead of one second. This makes syncs preserve
sub-second modtimes when comparing and copying files.
Note that this means rclone will consider files with modtimes
differing by less than a second as needing their modtime set where
before it did not.
The server computes the MD5 checksum asynchronously after upload, so
the object returned from Put and Update often had no MD5 while a
fresh listing shortly afterwards would report one. This broke
wrappers which compare exact hashes, such as the hasher backend's
fingerprint and the VFS cache. Retry reading the metadata for a short
time after upload until the MD5 appears.
The object returned from Put and Update had an empty MD5 checksum
while a fresh listing would report the MD5 the server computed for
the upload. This broke wrappers which compare exact hashes, such as
the hasher backend's fingerprint and the VFS cache. Stat the file
after upload to pick up the server computed MD5.
Update (which Put also uses) cleared the object's MD5 checksum after
upload, so until the object was re-read it had no hash while a fresh
listing would report the MD5 the server computed. This broke wrappers
which compare exact hashes, such as the hasher backend's fingerprint
and the VFS cache. Re-read the object's metadata after upload to pick
up the server computed MD5 (and authoritative size).
Update (which Put also uses) kept the caller's full precision modtime
in memory while the upload finalize request sends it rounded to
microseconds, so until the object was re-read the modtime did not
match what a fresh listing would report. This broke wrappers which
compare exact modtimes, such as the hasher backend's fingerprint and
the VFS cache. Round the modtime to microseconds to match what the
server stores.
Update (which Put also uses) kept the caller's full precision modtime
in memory while the server stores second precision, so until the
object was re-read the modtime did not match what a fresh listing
would report. This broke wrappers which compare exact modtimes, such
as the hasher backend's fingerprint and the VFS cache. Truncate the
modtime to second precision to match what the server stores.
The objects returned from Put, Move and Update kept the caller's full
nanosecond precision modtime (which write responses echo back) while
the server stores millisecond precision, so until the object was
re-read the modtime did not match what a fresh listing would report.
Update also discarded the upload response so the object had no SHA256
checksum until re-read. This broke wrappers which compare exact
modtimes and hashes, such as the hasher backend's fingerprint and the
VFS cache. Truncate the modtime to millisecond precision to match
what the server stores and populate the object from the upload
response.
Update and SetModTime kept the caller's full nanosecond precision
modtime in memory while the server stores SMB LastWriteTime with
100 ns (FILETIME) precision, so until the object was re-read the
modtime did not match what a fresh listing would report. This broke
wrappers which compare exact modtimes, such as the hasher backend's
fingerprint and the VFS cache. Truncate the modtime to 100 ns
precision to match what the server stores.
SetModTime kept the caller's full precision modtime in memory while
the PROPPATCH sets it on the server with second precision, so until
the object was re-read the modtime did not match what a fresh listing
would report. This broke wrappers which compare exact modtimes, such
as the hasher backend's fingerprint and the VFS cache. Truncate the
modtime to second precision to match what the server stores.
SetModTime kept the caller's full precision modtime in memory while
the server stores second precision, so until the object was re-read
the modtime did not match what a fresh listing would report. This
broke wrappers which compare exact modtimes, such as the hasher
backend's fingerprint and the VFS cache. Truncate the modtime to
second precision to match what the server stores.
SetModTime kept the caller's full precision modtime in memory while
the server stores second precision, so until the object was re-read
the modtime did not match what a fresh listing would report. This
broke wrappers which compare exact modtimes, such as the hasher
backend's fingerprint and the VFS cache. Truncate the modtime to
second precision to match what the server stores.
SetModTime kept the caller's full precision modtime in memory while
the server stores second precision, so until the object was re-read
the modtime did not match what a fresh listing would report. This
broke wrappers which compare exact modtimes, such as the hasher
backend's fingerprint and the VFS cache. Truncate the modtime to
second precision to match what the server stores.
SetModTime (and the no_check_upload Update path) kept the caller's
full precision modtime in memory while the server stores second
precision, so until the object was re-read the modtime did not match
what a fresh listing would report. This broke wrappers which compare
exact modtimes, such as the hasher backend's fingerprint and the VFS
cache. Truncate the modtime to second precision to match what the
server stores.
shouldRetry treated every Code=200501 API error as a transient storage
block error and retried it. Proton also returns Code=200501 with an
HTTP 422 for permanent validation failures (e.g. a content key packet
that cannot be verified, or an upload format the account is not
enabled for), so these were retried until the operation timed out
causing a multi-minute hang.
This fixes it by only retrying Code=200501 when it is not a permanent
client (4xx) error.
SetModTime (which Put and Update also use) kept the caller's full
precision modtime in memory while Chtimes sets it with second
precision, so until the object was re-read the modtime did not match
what a fresh listing would report. This broke wrappers which compare
exact modtimes, such as the hasher backend's fingerprint and the VFS
cache. Truncate the modtime to second precision to match what the
server stores.
Update and SetModTime kept the caller's full precision modtime in
memory while the server stores second precision, so until the object
was re-read the modtime did not match what a fresh listing would
report. This broke wrappers which compare exact modtimes, such as the
hasher backend's fingerprint and the VFS cache. Truncate the modtime
to second precision to match what the server stores.
Object.Update and SetModTime left the in-memory modtime at the
source's nanosecond precision, while the server stores milliseconds.
Afterwards the in-memory state did not match what a fresh List would
report, which broke wrappers that compare modtimes (e.g. the hasher
backend's fingerprint, which made it lose track of hashes whenever a
file was replaced).
The SDK serializes modtimes as UnixMilli and NewIncompleteFile
already rounds to milliseconds, so Put and the chunked upload path
are unaffected. Round the modtime to milliseconds in Update and
SetModTime too, so every path holds the same value the server
returns, without needing to re-read the metadata from the server
after upload.
Fixes#9308
When rclone sync replaced an existing file, it called Update which
pruned the old hash but never computed or stored the new one. This
left the file with no hash entry in the bolt DB.
This applies the same hashing logic in Put to Update: compute hashes
during the transfer via a hashingReader and store them afterwards. The
common hash-wrapping and hash-storing logic was extracted into a
function.
Fixes#9308
After PUTting a file to the upload URL, Yandex keeps the file locked
for writing until the upload operation finishes committing on the
server. The PUT returned before this happened, so the following
SetModTime raced the still-in-progress write and got spurious 500
Internal Server Error responses.
Capture the operation_id returned with the upload URL and poll the
operation status until it reports success before returning, so the file
is fully committed before we access it.
The background chunk cleaner slept for the whole ChunkCleanInterval
(default 1 minute) before checking its stop channel, and only ran
CleanUpCache via the select default branch. This meant a cache that
had been stopped by StopBackgroundRunners could keep running
CleanUpCache for up to an interval afterwards.
The cache backend tests all share a single on-disk chunk store (the
TestInternalCache remote), so a lingering cleaner from a finished test
could call CleanChunksBySize and os.RemoveAll chunks that a later,
unrelated test had just written. The later test would then read a
chunk back and get an unexpected EOF - eg
TestInternalMaxChunkSizeRespected failing intermittently on CI.
Wait on a timer and the stop channel together so a stop is honoured
immediately and the cleaner can never run again once stopped.
A folder shortcut pointing at one of its own ancestor folders (for
example a shortcut to FolderX placed inside FolderX) made rclone recurse
forever when dereferencing shortcuts, duplicating the folder contents
until the disk was full.
Rclone now detects when a folder shortcut targets an ancestor directory
using the directory cache, leaves that shortcut out of the listing and
logs an ERROR, so the rest of the drive can still be copied.
Fixes#7118Fixes#9565Closes#9051
cleanRootPath used filepath.Abs which prepends the current directory,
but the resulting absolute path does not always refer to the same
directory as the original relative path - for example when the current
directory is shadowed by a mount or has been removed. This made
"rclone copy --links . ../dst" fail where "cp -ra . ../dst" succeeds.
rclone now cleans the path lexically on non-Windows platforms instead,
leaving relative roots relative so the OS resolves them against the live
working directory. Windows still makes the path absolute as required for
UNC long-path conversion.
When the stored token is expired and refreshOrReLogin fails, the
returned error wrapped the original 401 and discarded authErr, hiding
the actionable reason (e.g. "account requires 2FA - please run: rclone
config reconnect remote:"). Wrap both errors so the user sees why
re-auth failed and what to do about it.
Fixes#9583
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Zoho's WorkDrive listing API returns "401 R008 Unauthorized access"
(not a 404) when a folder id no longer resolves to a listable folder,
because it was deleted or never existed. A freshly refreshed token still
gets it, so it is not a token problem and retrying it is futile - and can
escalate to a 429 F7008 rate-limit penalty.
Handle it as a missing directory instead: shouldRetry no longer retries a
bare R008 401, listAll maps it to fs.ErrorDirNotFound, and
readMetaDataForPath flushes the stale parent from the dircache and reports
the object as not found so a later create re-resolves the parent. This lets
the VFS self-heal a stale cached directory id instead of hard-failing the
operation, and stops the VFS integration tests failing on a stale directory id.
Fixes#9578
The /stream/upload endpoint's overwrite flag is the x-prefixed header
x-override-name-exist, matching its siblings x-filename, x-parent_id and
x-streammode. uploadLargeFile sent the un-prefixed override-name-exist, which
the endpoint ignores, so overwriting a file of 10 MiB or larger created a
renamed duplicate instead of updating the existing file. The small upload API
passes the flag as a query parameter and is unaffected.
The folder list limiter granted the caller-supplied time immediately in
the burst phase, so concurrent callers observing time.Now() out of order
could record grant times that moved backwards. The sliding safety log
indexes grants as an ordered history, so out-of-order grants could also
breach the rolling-window cap. Clamp each grant to be at or after the
previous one so grant times are always monotonic.
Seafile returns a content-addressed id (a git-like hash of the file
contents) rather than a unique per-object identifier. Two different
files with identical contents are therefore given the same id, even on
different servers.
operations.SameObject assumes IDs uniquely identify an object, so when
syncing between Seafile remotes it treated two distinct but
identical-content files as the same object. The guard in NeedTransfer
if Equal(ctx, src, dst) && !SameObject(src, dst)
then evaluated to false for unchanged files (Equal was true because the
sizes matched, and SameObject was true because the ids matched), so
rclone re-uploaded them on every sync.
Seafile has no modtime and no usable content hash, so dropping the IDer
implementation makes SameObject fall back to path comparison and lets
unchanged files be skipped correctly.
Fixes#8300
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
On a range request Azure returns the whole-blob MD5 in the
x-ms-blob-content-md5 header (BlobContentMD5) and leaves Content-MD5
empty. The download metadata decoder only read Content-MD5, so every
ranged read overwrote the object's MD5 with an empty string.
With --vfs-cache-mode full this changed the object's fingerprint between
opens, so the VFS cache judged every reopened file as stale and
re-downloaded it, defeating the cache and inflating egress. Prefer
BlobContentMD5 and never overwrite a known hash with an empty one.
With -l/--links rclone recreates a .rclonelink object as a symlink. A
malicious or compromised source could serve a symlink whose target points
outside the destination, plus a sibling object whose path traverses it, so
that rclone followed the planted symlink and wrote outside the destination
causing arbitrary file write.
When translating symlinks, rclone now performs all destination writes
(directory creation, file writes and symlink creation) through an os.Root
anchored at the destination. os.Root resolves every path component relative
to the destination's file descriptor and refuses any that escapes the root,
even under concurrent modification, so a planted symlink can never be
traversed out of the destination.
Symlinks are still reproduced verbatim - including ones whose target points
outside the destination - so backups remain faithful. Only writing
*through* such a link is refused. In-tree symlinks are unaffected.
Fixes CVE-2026-54572
Fixes GHSA-cf44-9pgv-m4xc
Nick Craig-Wood
Erol OzcanandNick Craig-Wood
dougalandNick Craig-Wood
jzunigax2andNick Craig-Wood