Commit Graph
10302 Commits
Author SHA1 Message Date
Nick Craig-Wood 4369d16a1c test_all: pikpak: ignore TestRcatSizeChecksum/Corrupted
Pikpak never returns MD5 for uploads which causes this test to fail.

Perhaps Pikpak should not declare MD5 but that is a bigger decision
being discussed in #9826
2026-08-26 12:00:22 +01:00
Nick Craig-Wood f7c510af49 webdav: fix SetModTime failing and hashes missing on Nextcloud
Nextcloud only stores a checksum which is supplied in the OC-Checksum
header of an upload, and discards it again when the modification time
is set with PROPPATCH. Re-sending the checksum in the PROPPATCH (as is
done for ownCloud) is rejected by Nextcloud with 403 Forbidden which
made the whole PROPPATCH fail, so SetModTime returned an error on any
object which had a hash. Uploads from sources without hashes, eg
streamed uploads with `rclone rcat`, were stored with no hash at all.

Use the Nextcloud PATCH extension with the X-Recalculate-Hash header
to have the server calculate and store the SHA1 of an object after a
streamed upload and after setting the modification time. This gives
a server side hash of the stored data which also lets rclone verify
streamed uploads.
2026-08-26 12:00:22 +01:00
Nick Craig-Wood 8e744de5e6 pikpak: fix truncated single part uploads reported as ok when source ends early
If the source supplied fewer bytes than its declared size, the single
part upload path accepted the short body and stored a truncated file
recorded with the declared size, reporting a successful upload. The
multipart path already checks for this.

Count the bytes actually read and fail the upload if they do not match
the declared size, which cancels the partially created file.

This was found by the FsPutShortEOF integration test.
2026-08-26 12:00:22 +01:00
Nick Craig-Wood a893df601a Add machsix to contributors 2026-08-26 12:00:22 +01:00
machsixandNick Craig-Wood 8869a848f2 onedrive: fix 403 Forbidden for configuration personal onedrive 2026-08-25 09:32:52 +01:00
Nick Craig-Wood d3a71eea36 azureblob: fix test which didn't compile
We accidentally merged this commit with non compiling tests.

bee45bccfd azureblob: fix spurious vfs cache corruption errors during chunked reads #9782
2026-08-25 09:31:40 +01:00
Nick Craig-Wood d68ecd1717 Add Sanjay Kanth A to contributors 2026-08-25 09:31:40 +01:00
Sanjay Kanth AandNick Craig-Wood f3a7aaf635 dropbox: decode received shared-file names - fixes #9707
listSharedFolders already decoded shared-folder names with
f.opt.Enc.ToStandardName, but listReceivedFiles stored the raw name
returned by the Dropbox API unchanged. Names that require encoding
(e.g. a trailing space, which Dropbox itself rejects, so rclone
stores it as "name␠" via EncodeRightSpace) were therefore shown under
their raw, encoded form for received files instead of being decoded
back to the standard name, and findSharedFile could not resolve such
a file by its standard name.

Apply the same ToStandardName conversion listSharedFolders uses.
2026-08-25 09:26:13 +01:00
Nick Craig-Wood bee45bccfd azureblob: fix spurious vfs cache corruption errors during chunked reads - fixes #9782
On a ranged download the metadata decoder stored the response's
Content-Length (the length of the range, not the blob) in the object's
size and only corrected it from the Content-Range total afterwards.
Object.Size() is read concurrently by the VFS cache and chunked reader
while a download is in progress, so with --vfs-read-chunk-size a reader
could observe the chunk length (e.g. 67108864 for 64M chunks) as the
object size. The VFS cache then logged

    vfs cache: cached file (N) is unexpectedly larger than the remote
    object (67108864). The cached file is likely corrupted after an
    unclean shutdown; recovering ...

and truncated the read request against the bogus size, breaking
sequential reads of large blobs with --vfs-cache-mode full.

This applies the Content-Range correction before the size is stored so
the range length is never published as the object size.
2026-08-24 18:16:12 +01:00
Nick Craig-Wood 2a8d8afd0f lib/rest: make ParseContentRange public 2026-08-24 18:16:12 +01:00
kingston125andNick Craig-Wood 6ee1d851ec filelu: fix duplicate root path during multipart folder creation 2026-08-24 18:11:57 +01:00
Rohit BeheraandNick Craig-Wood 83b143103c huaweidrive: fix truncated files being uploaded successfully when the source ends early
The multipart upload copied the source into the request buffer without
checking how many bytes it had read, so a source that supplied fewer
bytes than its declared size was accepted by the server and reported as
a success with a truncated file stored.

Count the bytes actually read and fail the upload if they do not match
the declared size.

Signed-off-by: Rohit Behera <126186063+r0h1tb@users.noreply.github.com>
2026-08-24 18:10:19 +01:00
Nick Craig-Wood d9aa903358 protondrive: fix files uploaded with v1.75.0 not being readable in the Proton apps
rclone v1.75.0 started creating files in Proton Drive's new
crypto-refresh encryption format, following guidance from Proton that
new file node keys should use the v6/AEAD profile. It turns out the
official Proton web app cannot decrypt files whose node key is a v6
key (but the Android app can), so every file uploaded with v1.75.0 (or
a beta after 2026-07-13) shows 'Item cannot be decrypted' in the web
app, even though rclone itself reads the files fine. Inspecting a file
created by the web app shows Proton itself still creates v4 node keys,
using the new format only for the file content.

New files are now created with the same fully pre-crypto-refresh
format as v1.74.4 (v4 node key, v3 PKESK content key, v1 SEIPD
blocks), which every Proton client can read. Reading files in the new
format still works, new revisions of files which already use the new
content format keep it, and the auxiliary fields (name, node
passphrase, extended attributes, block signatures) are pinned to the
old format regardless of the recipient key's preferences, as Proton
requires.

Files already uploaded with v1.75.0 cannot be repaired in place -
uploading a new revision does not change the file's node key. To make
such a file readable by the Proton apps again, delete it from the
remote and upload it again with a fixed version of rclone.

This updates Proton-API-Bridge to v1.0.5 and go-proton-api to v1.0.4.

See: https://forum.rclone.org/t/proton-drive-unable-to-decrypt/54087
2026-08-24 17:47:13 +01:00
Nick Craig-Wood c7be826d7b Add Rahman Yilmaz to contributors 2026-08-24 17:47:13 +01:00
Nick Craig-Wood 3c1a265842 Add Rohit Behera to contributors 2026-08-24 17:47:13 +01:00
Rohit BeheraandNick Craig-Wood 64ab1ac322 box: fix truncated files being uploaded successfully when the source ends early
The single-shot upload path sent the source straight to Box as a multipart
body with no Content-Length, so a source that supplied fewer bytes than its
declared size produced a short request that Box accepted and stored, and the
upload was reported as a success.

Count the bytes actually read and fail the upload if they do not match the
declared size. The multipart path already reads each chunk with io.ReadFull
and so already fails in this case.
2026-08-21 17:53:41 +01:00
Rahman YilmazandGitHub 5eb5c01e36 walk: stop directory traversal when the context is cancelled - fixes #9788
The concurrent walker created by walk() only stopped when the callback
returned an error or the whole tree had been listed. Cancelling the
context (for example via the rc job/stop endpoint for an async
operations/size or recursive operations/list call) was therefore
ignored: the checkers kept pulling list jobs from the channel and kept
listing the entire tree, burning CPU and making job cancellation
useless for every backend without a native ListR implementation.

Make every checker select on ctx.Done() so a cancelled walk shuts down
promptly through the existing quit/drain path and reports the context
error. Also check the context between directory read chunks in the
local backend so a single huge directory does not block cancellation.
2026-08-21 17:48:29 +01:00
Rohit BeheraandNick Craig-Wood 1128693468 yandex: fix truncated files being uploaded successfully when the source ends early
Update already wrapped the source in a counting reader but never looked at the
count, so a source that supplied fewer bytes than its declared size was
uploaded as a chunked request, accepted by the server and reported as a
success with a truncated file stored.

Compare the bytes actually read against the declared size.
2026-08-21 17:46:05 +01:00
Dominik SanderandGitHub e2352201d1 local: speed up default checksummed copies by writing in larger blocks
When copying to the local backend with checksums enabled (the default),
rclone hashed the incoming data by wrapping the source reader in an
io.TeeReader. TeeReader has no WriteTo method, so io.Copy could not use
the source's fast path and fell back to its generic 32 KiB buffer loop.
The same wrapping also stopped the destination *os.File using
copy_file_range, since the source was no longer a raw fd.

This meant checksummed copies were written in 32 KiB chunks whereas
--ignore-checksum copies were written in much larger blocks (typically
1 MiB). On filesystems where small writes are expensive, such as FUSE
mounts like LucidLink, this made a big difference: copying a 100 MiB
file took 3203 x 32 KiB writes in 3.6s, and now takes 108 x ~1 MiB
writes in 0.47s.
2026-08-21 17:40:53 +01:00
Nick Craig-Wood be7f9b38b0 build: untap aws/tap to silence homebrew tap trust warnings on macOS 2026-08-21 12:52:12 +01:00
Nick Craig-Wood 0027678977 huaweidrive: simplify chunk size clamping found by "go fix -minmax" 2026-08-21 12:23:31 +01:00
Nick Craig-Wood a017a54bef build: modernize with "go fix -waitgroupgo": use WaitGroup.Go 2026-08-21 12:23:31 +01:00
Nick Craig-Wood 2d1a3386a8 build: modernize with "go fix -stringsseq": use SplitSeq iterators 2026-08-21 12:23:31 +01:00
Nick Craig-Wood aed06f9052 build: modernize with "go fix -stringscutprefix": use strings.CutPrefix 2026-08-21 12:23:31 +01:00
Nick Craig-Wood 7b002153bd build: modernize with "go fix -stringscut": use strings.Cut 2026-08-21 12:23:31 +01:00
Nick Craig-Wood 4e9577ed76 build: modernize with "go fix -stringsbuilder": use strings.Builder 2026-08-21 12:23:31 +01:00
Nick Craig-Wood 9f9fd82923 build: modernize with "go fix -slicescontains": use slices.Contains 2026-08-21 12:23:31 +01:00
Nick Craig-Wood 237719bb8d build: modernize with "go fix -slicesbackward": use slices.Backward 2026-08-21 12:23:31 +01:00
Nick Craig-Wood e20e18d2e0 build: modernize with "go fix -reflecttypefor": use reflect.TypeFor 2026-08-21 12:23:31 +01:00
Nick Craig-Wood a64c0a0fde build: modernize with "go fix -rangeint": use range over int 2026-08-21 12:23:31 +01:00
Nick Craig-Wood 67728ce37d huaweidrive: remove no-op omitempty found by "go fix -omitzero" 2026-08-21 12:23:31 +01:00
Nick Craig-Wood 53d9f7f956 build: modernize with "go fix -newexpr": use go1.26 new(expr)
Also inline and remove the now unneeded pointer helper functions.
2026-08-21 12:23:31 +01:00
Nick Craig-Wood b5bea683c5 build: modernize with "go fix -minmax": use min and max builtins 2026-08-21 12:23:31 +01:00
Nick Craig-Wood bfd0e3f3c2 build: modernize with "go fix -mapsloop": use maps.Copy 2026-08-21 12:23:31 +01:00
Nick Craig-Wood 33e9251b52 build: modernize with "go fix -errorsastype": use errors.AsType 2026-08-21 12:23:31 +01:00
Nick Craig-Wood 77f9c70cf6 build: modernize with "go fix -atomictypes": use sync/atomic types 2026-08-21 12:23:31 +01:00
Nick Craig-Wood 7929921ed8 build: modernize with "go fix -any": replace interface{} with any 2026-08-21 12:23:31 +01:00
Nick Craig-Wood 357c2a2b44 build: disable staticcheck SA4023 to fix lint job timeout
The dataflow analysis behind SA4023, new in the staticcheck 0.8.0
bundled with golangci-lint v2.13.0, makes linting large packages more
than 10x slower (89s vs 7s for backend/s3 alone) which took the CI
lint job past its 30 minute limit. golangci-lint no longer enforces
its run timeout during analysis so the job ran until cancelled, and
the cancellation meant the lint cache was never saved, making every
subsequent run cold and guaranteeing the timeout repeated.

The check also produces false positives (eg claiming operations.Delete
never returns nil).
2026-08-21 12:23:31 +01:00
Nick Craig-Wood 152adfcd2a serve s3: update to gofakes3 v0.0.8 which fixes 500 errors with go1.27 2026-08-21 12:23:31 +01:00
Nick Craig-Wood e189f38122 build: update to go1.27 and make go1.26 the minimum required version 2026-08-20 12:16:48 +01:00
Nick Craig-Wood ec3a95c279 s3: Mega: update endpoints 2026-08-20 12:12:00 +01:00
Nick Craig-Wood 2f7d4e3f9f rc: deprecate --rc-web-gui flag
This commit removes the workings of the old web ui which hasn't been
maintained for 6 years. If users supply --rc-web-gui then rclone will
exit with an error pointing users at the maintained `rclone gui`
command.
2026-08-19 12:15:00 +01:00
Nick Craig-Wood 00496f5dfb docs: take gui out of beta 2026-08-18 14:29:43 +01:00
Nick Craig-Wood 2c1174af0d docs: add assigned CVE numbers to the v1.75.0 security advisories in the changelog
Five of the advisories released with v1.75.0 now have CVEs assigned:

- GHSA-45pq-889g-fcgh serve restic path traversal: CVE-2026-71309
- GHSA-xhf4-832v-7xcr lib/proxy CONNECT header OOM: CVE-2026-71310
- GHSA-8c48-q9wj-3w37 ftp command injection: CVE-2026-71311
- GHSA-2m8m-jhrm-w6j2 sftp PowerShell command injection: CVE-2026-71312
- GHSA-7p4m-qxvv-g567 local file name escape: CVE-2026-71313

GHSA-6jcg-q3wp-x2f4 (squashfs) loses its CVE-PENDING marker as GitHub
declined to issue a CVE from the rclone repository - the vulnerable code
is in go-diskfs so any CVE must come from an advisory there.

GHSA-mfvx-7rcj-9m5g (pprof) keeps its CVE-PENDING marker as the CVE
request is still awaiting allocation.
2026-08-18 12:29:35 +01:00
Nick Craig-Wood b13f62237a Add VXNCXNX to contributors 2026-08-18 12:29:35 +01:00
Nick Craig-Wood 8da69d8775 Add Shantanav Mukherjee to contributors 2026-08-18 12:29:35 +01:00
Nick Craig-Wood 5b417ce5a1 Add Hakan İSMAİL to contributors 2026-08-18 12:29:35 +01:00
Nick Craig-Wood 47b1ac9540 Add max to contributors 2026-08-18 12:29:35 +01:00
VXNCXNXandNick Craig-Wood c667e53638 lib/transform: fix panic in truncate_keep_extension
Return error when extension is longer than truncation limit.
2026-08-18 11:13:23 +01:00
Shantanav MukherjeeandGitHub 4f22d62c66 docs: clarify VFS cache age semantics 2026-08-18 09:49:42 +01:00