Nine tests fail on an ordinary Windows machine, eight in backend/local and
TestEnvironmentVariables in cmdtest, all with
symlink file.txt \?\C:\Users\...\symlink.txt: A required privilege is not
held by the client.
Windows grants SeCreateSymbolicLinkPrivilege only to an elevated process or one
running with Developer Mode enabled, and a default install gives an ordinary
user neither. CI does not see this because the windows-latest runner is
elevated, so the failures only show up on a contributor's own machine, where
AGENTS.md asks for make quicktest to pass before opening a pull request.
cmdtest already recognised the situation and attached a note to the failure
saying the test could safely be ignored. If it is safe to ignore then the test
knows it cannot run, so skip it and say why instead.
backend/local gains a helper that tries a symlink in t.TempDir() and skips if it
cannot make one, called from the six tests that need the privilege. Where a
platform can create symlinks the probe succeeds and nothing is skipped, so other
platforms are unchanged.
TestMetadata is skipped whole because it creates its symlink before anything
else and the object built from it is used throughout.
TestSymlinkEscapeConcurrent is left alone: it goes through putLink and ignores
the error, so it never needed the privilege.
Directory names which look like they have a --b2-versions version
string are now encrypted in full, so directories created by older
rclone (which left the version string in plain text) no longer
decrypt and vanished silently from listings.
DecryptDirName now falls back to the old form for such names so the
directory is listed, and logs the name it needs to be renamed to on
the underlying remote to make it accessible again. Document this in
the crypt docs.
The refresh endpoint returns a rotated token with a fresh expiry on
every successful call, but getUserInfo discarded it, so routine use
never extended the stored token's life. Once the stored token aged
out, accounts with 2FA enabled could not recover non-interactively
and required a manual reconnect.
Carry the rotated token out of getUserInfo and persist it in NewFs
via the same jwtToOAuth2Token + oauthutil.PutToken path that
refreshJWTToken uses, keeping f.cfg.Token in sync (same pattern as
refreshOrReLogin).
Fixes#9584
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The --b2-versions support added in 3fe2aaf96 strips a version string
from the last segment of a path before encrypting it, so that the
plain text version suffixes which the underlying backend appends to
encrypted file leaf names can be handled. EncryptDirName and
DecryptDirName share that code, so the last segment of a *directory*
name was version stripped too. Only file leaf names are ever given a
version string by the backend - a directory gets a
version-string-like name from the user, and such a name is encrypted
verbatim when it appears as the parent of a file name, so the same
directory ended up with two different encryptions.
Before this change, with a directory whose name matches rclone's
version format, eg dir-v2001-02-03-040506-123:
rclone copy file.txt crypt:dir-v2001-02-03-040506-123/
rclone ls crypt:dir-v2001-02-03-040506-123
# => "directory not found" - the file is invisible to listings
rclone mkdir crypt:dir-v2001-02-03-040506-123
# => creates a second directory with the same decrypted name
After this change EncryptDirName and DecryptDirName encrypt directory
names verbatim, so a directory encrypts the same way whether it is
named on its own or as the parent of a file. Version strings are only
added to file names by the underlying backend, so --b2-versions is
unaffected and the existing version tests are untouched.
A directory which was created by the old EncryptDirName will no longer
decrypt and will be reported as undecryptable in listings. Such
directories were already unusable - anything copied into one was
written to a different encrypted directory - so nothing which worked
before is broken by this.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
With no_head_object set, NewObject does not read any metadata, so the
destination object returned from a server side copy had a size of 0.
The size check in operations.Copy then failed with "corrupted on
transfer: sizes differ N vs 0" and deleted the newly copied object.
This also broke Move and hence renames through rclone mount.
Populate the destination object's size and MD5 from the source object
when no_head_object is set, as a server side copy produces an object
with identical content.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Dropbox is case insensitive and the path_display it returns in
change notifications may not match the case of the configured root.
Before this change the root was trimmed with a case sensitive prefix
match, so when the cases differed the full path was passed to the
ChangeNotify callback and the notification was ignored.
This trims the root case insensitively while preserving the display
case of the remaining path.
Write() overwrote a successful write's nil error with the stale
lastErr returned by kickWaiters() once the downloader had recorded
too many errors. download() then wrapped that stale error again and
stored it back as the new lastErr, so every subsequent write added
another "vfs reader: failed to write to cache file:" prefix - fixes#4998
Several documentation links pointed at anchors or paths that no longer
resolve, and the S3 directory buckets section named the config option
and flag in the plural, which does not match the backend.
Co-authored-by: shaurya <19599684+no-hup@users.noreply.github.com>
Co-authored-by: no-hup <shauryaj.finance@gmail.com>
A successful UploadPart whose response carries no ETag header made
WriteChunk panic dereferencing uout.ETag in a debug log line. The part
ETag is required by CompleteMultipartUpload, so an ETag-less 200 is
unusable: return a retryable error from inside the pacer callback so
the chunk is retried instead of crashing the transfer or completing
the upload with a broken part list.
Fixes#9822
Co-authored-by: Shurong Cao <170531907+CAOShurong@users.noreply.github.com>
Pikpak never returns MD5 for uploads which causes this test to fail.
Perhaps Pikpak should not declare MD5 but that is a bigger decision
being discussed in #9826
Nextcloud only stores a checksum which is supplied in the OC-Checksum
header of an upload, and discards it again when the modification time
is set with PROPPATCH. Re-sending the checksum in the PROPPATCH (as is
done for ownCloud) is rejected by Nextcloud with 403 Forbidden which
made the whole PROPPATCH fail, so SetModTime returned an error on any
object which had a hash. Uploads from sources without hashes, eg
streamed uploads with `rclone rcat`, were stored with no hash at all.
Use the Nextcloud PATCH extension with the X-Recalculate-Hash header
to have the server calculate and store the SHA1 of an object after a
streamed upload and after setting the modification time. This gives
a server side hash of the stored data which also lets rclone verify
streamed uploads.
If the source supplied fewer bytes than its declared size, the single
part upload path accepted the short body and stored a truncated file
recorded with the declared size, reporting a successful upload. The
multipart path already checks for this.
Count the bytes actually read and fail the upload if they do not match
the declared size, which cancels the partially created file.
This was found by the FsPutShortEOF integration test.
listSharedFolders already decoded shared-folder names with
f.opt.Enc.ToStandardName, but listReceivedFiles stored the raw name
returned by the Dropbox API unchanged. Names that require encoding
(e.g. a trailing space, which Dropbox itself rejects, so rclone
stores it as "name␠" via EncodeRightSpace) were therefore shown under
their raw, encoded form for received files instead of being decoded
back to the standard name, and findSharedFile could not resolve such
a file by its standard name.
Apply the same ToStandardName conversion listSharedFolders uses.
On a ranged download the metadata decoder stored the response's
Content-Length (the length of the range, not the blob) in the object's
size and only corrected it from the Content-Range total afterwards.
Object.Size() is read concurrently by the VFS cache and chunked reader
while a download is in progress, so with --vfs-read-chunk-size a reader
could observe the chunk length (e.g. 67108864 for 64M chunks) as the
object size. The VFS cache then logged
vfs cache: cached file (N) is unexpectedly larger than the remote
object (67108864). The cached file is likely corrupted after an
unclean shutdown; recovering ...
and truncated the read request against the bogus size, breaking
sequential reads of large blobs with --vfs-cache-mode full.
This applies the Content-Range correction before the size is stored so
the range length is never published as the object size.
The multipart upload copied the source into the request buffer without
checking how many bytes it had read, so a source that supplied fewer
bytes than its declared size was accepted by the server and reported as
a success with a truncated file stored.
Count the bytes actually read and fail the upload if they do not match
the declared size.
Signed-off-by: Rohit Behera <126186063+r0h1tb@users.noreply.github.com>
rclone v1.75.0 started creating files in Proton Drive's new
crypto-refresh encryption format, following guidance from Proton that
new file node keys should use the v6/AEAD profile. It turns out the
official Proton web app cannot decrypt files whose node key is a v6
key (but the Android app can), so every file uploaded with v1.75.0 (or
a beta after 2026-07-13) shows 'Item cannot be decrypted' in the web
app, even though rclone itself reads the files fine. Inspecting a file
created by the web app shows Proton itself still creates v4 node keys,
using the new format only for the file content.
New files are now created with the same fully pre-crypto-refresh
format as v1.74.4 (v4 node key, v3 PKESK content key, v1 SEIPD
blocks), which every Proton client can read. Reading files in the new
format still works, new revisions of files which already use the new
content format keep it, and the auxiliary fields (name, node
passphrase, extended attributes, block signatures) are pinned to the
old format regardless of the recipient key's preferences, as Proton
requires.
Files already uploaded with v1.75.0 cannot be repaired in place -
uploading a new revision does not change the file's node key. To make
such a file readable by the Proton apps again, delete it from the
remote and upload it again with a fixed version of rclone.
This updates Proton-API-Bridge to v1.0.5 and go-proton-api to v1.0.4.
See: https://forum.rclone.org/t/proton-drive-unable-to-decrypt/54087
The single-shot upload path sent the source straight to Box as a multipart
body with no Content-Length, so a source that supplied fewer bytes than its
declared size produced a short request that Box accepted and stored, and the
upload was reported as a success.
Count the bytes actually read and fail the upload if they do not match the
declared size. The multipart path already reads each chunk with io.ReadFull
and so already fails in this case.
The concurrent walker created by walk() only stopped when the callback
returned an error or the whole tree had been listed. Cancelling the
context (for example via the rc job/stop endpoint for an async
operations/size or recursive operations/list call) was therefore
ignored: the checkers kept pulling list jobs from the channel and kept
listing the entire tree, burning CPU and making job cancellation
useless for every backend without a native ListR implementation.
Make every checker select on ctx.Done() so a cancelled walk shuts down
promptly through the existing quit/drain path and reports the context
error. Also check the context between directory read chunks in the
local backend so a single huge directory does not block cancellation.
Update already wrapped the source in a counting reader but never looked at the
count, so a source that supplied fewer bytes than its declared size was
uploaded as a chunked request, accepted by the server and reported as a
success with a truncated file stored.
Compare the bytes actually read against the declared size.
When copying to the local backend with checksums enabled (the default),
rclone hashed the incoming data by wrapping the source reader in an
io.TeeReader. TeeReader has no WriteTo method, so io.Copy could not use
the source's fast path and fell back to its generic 32 KiB buffer loop.
The same wrapping also stopped the destination *os.File using
copy_file_range, since the source was no longer a raw fd.
This meant checksummed copies were written in 32 KiB chunks whereas
--ignore-checksum copies were written in much larger blocks (typically
1 MiB). On filesystems where small writes are expensive, such as FUSE
mounts like LucidLink, this made a big difference: copying a 100 MiB
file took 3203 x 32 KiB writes in 3.6s, and now takes 108 x ~1 MiB
writes in 0.47s.
Nick Craig-Wood
DhevenddraandNick Craig-Wood
CAOShurongandGitHub
Loi NguyenandNick Craig-Wood
Sune MølgaardandNick Craig-Wood
Vijay MisalandNick Craig-Wood
waterandNick Craig-Wood
machsixandNick Craig-Wood
Sanjay Kanth AandNick Craig-Wood
kingston125andNick Craig-Wood
Rohit BeheraandNick Craig-Wood
Rahman YilmazandGitHub
Dominik SanderandGitHub