Commit Graph
10284 Commits
Author SHA1 Message Date
Dominik SanderandGitHub e2352201d1 local: speed up default checksummed copies by writing in larger blocks
When copying to the local backend with checksums enabled (the default),
rclone hashed the incoming data by wrapping the source reader in an
io.TeeReader. TeeReader has no WriteTo method, so io.Copy could not use
the source's fast path and fell back to its generic 32 KiB buffer loop.
The same wrapping also stopped the destination *os.File using
copy_file_range, since the source was no longer a raw fd.

This meant checksummed copies were written in 32 KiB chunks whereas
--ignore-checksum copies were written in much larger blocks (typically
1 MiB). On filesystems where small writes are expensive, such as FUSE
mounts like LucidLink, this made a big difference: copying a 100 MiB
file took 3203 x 32 KiB writes in 3.6s, and now takes 108 x ~1 MiB
writes in 0.47s.
2026-08-21 17:40:53 +01:00
Nick Craig-Wood be7f9b38b0 build: untap aws/tap to silence homebrew tap trust warnings on macOS 2026-08-21 12:52:12 +01:00
Nick Craig-Wood 0027678977 huaweidrive: simplify chunk size clamping found by "go fix -minmax" 2026-08-21 12:23:31 +01:00
Nick Craig-Wood a017a54bef build: modernize with "go fix -waitgroupgo": use WaitGroup.Go 2026-08-21 12:23:31 +01:00
Nick Craig-Wood 2d1a3386a8 build: modernize with "go fix -stringsseq": use SplitSeq iterators 2026-08-21 12:23:31 +01:00
Nick Craig-Wood aed06f9052 build: modernize with "go fix -stringscutprefix": use strings.CutPrefix 2026-08-21 12:23:31 +01:00
Nick Craig-Wood 7b002153bd build: modernize with "go fix -stringscut": use strings.Cut 2026-08-21 12:23:31 +01:00
Nick Craig-Wood 4e9577ed76 build: modernize with "go fix -stringsbuilder": use strings.Builder 2026-08-21 12:23:31 +01:00
Nick Craig-Wood 9f9fd82923 build: modernize with "go fix -slicescontains": use slices.Contains 2026-08-21 12:23:31 +01:00
Nick Craig-Wood 237719bb8d build: modernize with "go fix -slicesbackward": use slices.Backward 2026-08-21 12:23:31 +01:00
Nick Craig-Wood e20e18d2e0 build: modernize with "go fix -reflecttypefor": use reflect.TypeFor 2026-08-21 12:23:31 +01:00
Nick Craig-Wood a64c0a0fde build: modernize with "go fix -rangeint": use range over int 2026-08-21 12:23:31 +01:00
Nick Craig-Wood 67728ce37d huaweidrive: remove no-op omitempty found by "go fix -omitzero" 2026-08-21 12:23:31 +01:00
Nick Craig-Wood 53d9f7f956 build: modernize with "go fix -newexpr": use go1.26 new(expr)
Also inline and remove the now unneeded pointer helper functions.
2026-08-21 12:23:31 +01:00
Nick Craig-Wood b5bea683c5 build: modernize with "go fix -minmax": use min and max builtins 2026-08-21 12:23:31 +01:00
Nick Craig-Wood bfd0e3f3c2 build: modernize with "go fix -mapsloop": use maps.Copy 2026-08-21 12:23:31 +01:00
Nick Craig-Wood 33e9251b52 build: modernize with "go fix -errorsastype": use errors.AsType 2026-08-21 12:23:31 +01:00
Nick Craig-Wood 77f9c70cf6 build: modernize with "go fix -atomictypes": use sync/atomic types 2026-08-21 12:23:31 +01:00
Nick Craig-Wood 7929921ed8 build: modernize with "go fix -any": replace interface{} with any 2026-08-21 12:23:31 +01:00
Nick Craig-Wood 357c2a2b44 build: disable staticcheck SA4023 to fix lint job timeout
The dataflow analysis behind SA4023, new in the staticcheck 0.8.0
bundled with golangci-lint v2.13.0, makes linting large packages more
than 10x slower (89s vs 7s for backend/s3 alone) which took the CI
lint job past its 30 minute limit. golangci-lint no longer enforces
its run timeout during analysis so the job ran until cancelled, and
the cancellation meant the lint cache was never saved, making every
subsequent run cold and guaranteeing the timeout repeated.

The check also produces false positives (eg claiming operations.Delete
never returns nil).
2026-08-21 12:23:31 +01:00
Nick Craig-Wood 152adfcd2a serve s3: update to gofakes3 v0.0.8 which fixes 500 errors with go1.27 2026-08-21 12:23:31 +01:00
Nick Craig-Wood e189f38122 build: update to go1.27 and make go1.26 the minimum required version 2026-08-20 12:16:48 +01:00
Nick Craig-Wood ec3a95c279 s3: Mega: update endpoints 2026-08-20 12:12:00 +01:00
Nick Craig-Wood 2f7d4e3f9f rc: deprecate --rc-web-gui flag
This commit removes the workings of the old web ui which hasn't been
maintained for 6 years. If users supply --rc-web-gui then rclone will
exit with an error pointing users at the maintained `rclone gui`
command.
2026-08-19 12:15:00 +01:00
Nick Craig-Wood 00496f5dfb docs: take gui out of beta 2026-08-18 14:29:43 +01:00
Nick Craig-Wood 2c1174af0d docs: add assigned CVE numbers to the v1.75.0 security advisories in the changelog
Five of the advisories released with v1.75.0 now have CVEs assigned:

- GHSA-45pq-889g-fcgh serve restic path traversal: CVE-2026-71309
- GHSA-xhf4-832v-7xcr lib/proxy CONNECT header OOM: CVE-2026-71310
- GHSA-8c48-q9wj-3w37 ftp command injection: CVE-2026-71311
- GHSA-2m8m-jhrm-w6j2 sftp PowerShell command injection: CVE-2026-71312
- GHSA-7p4m-qxvv-g567 local file name escape: CVE-2026-71313

GHSA-6jcg-q3wp-x2f4 (squashfs) loses its CVE-PENDING marker as GitHub
declined to issue a CVE from the rclone repository - the vulnerable code
is in go-diskfs so any CVE must come from an advisory there.

GHSA-mfvx-7rcj-9m5g (pprof) keeps its CVE-PENDING marker as the CVE
request is still awaiting allocation.
2026-08-18 12:29:35 +01:00
Nick Craig-Wood b13f62237a Add VXNCXNX to contributors 2026-08-18 12:29:35 +01:00
Nick Craig-Wood 8da69d8775 Add Shantanav Mukherjee to contributors 2026-08-18 12:29:35 +01:00
Nick Craig-Wood 5b417ce5a1 Add Hakan İSMAİL to contributors 2026-08-18 12:29:35 +01:00
Nick Craig-Wood 47b1ac9540 Add max to contributors 2026-08-18 12:29:35 +01:00
VXNCXNXandNick Craig-Wood c667e53638 lib/transform: fix panic in truncate_keep_extension
Return error when extension is longer than truncation limit.
2026-08-18 11:13:23 +01:00
Shantanav MukherjeeandGitHub 4f22d62c66 docs: clarify VFS cache age semantics 2026-08-18 09:49:42 +01:00
Hakan İSMAİLandNick Craig-Wood 30e79a017b serve, mountlib: test VFS release on shutdown and mount failure 2026-08-18 09:03:12 +01:00
Hakan İSMAİLandNick Craig-Wood 216d2a8c76 serve: fix VFS instance leaks on server startup failures and shutdown 2026-08-18 09:03:12 +01:00
Hakan İSMAİLandNick Craig-Wood 45ddf3a5f2 mountlib: fix VFS instance leaks on mount failures and unmounts 2026-08-18 09:03:12 +01:00
Hakan İSMAİLandNick Craig-Wood f425f8d466 serve: refactor VFS and proxy handling into Provider 2026-08-18 09:03:12 +01:00
maxandNick Craig-Wood a1f98b49df asyncreader: run the skip-bytes test under the race detector again
The test skipped every buffer count above 1 under -race, pointing at
golang/go#27070. That issue was closed in September 2018, so the
workaround outlived its cause: -race covered 137 of the 681 subtests.

Without the guard the race build runs all 681 and passes.
2026-08-17 08:13:21 +01:00
Nick Craig-Wood f0b210a886 build: fix multiple CVEs by upgrading to go1.26.6
- CVE-2026-56860: net/url: quadratic complexity in resolvePath
- CVE-2026-56858: html/template: JavaScript regexp context tracking
- CVE-2026-56862: crypto/tls: limit handshake messages accepted post-handshake
- CVE-2026-56853: net/http: apply ReadHeaderTimeout to unencrypted HTTP/2 check
- CVE-2026-56859: encoding/xml: recursion depth guard during decode
- CVE-2026-33818: encoding/asn1: enforce maximum recursion depth
- CVE-2026-46600: net: panic parsing an invalid SVCB or HTTPS RR in dnsmessage
- CVE-2026-39821: net/http: reject ASCII-only Punycode-encoded labels in idna

This also updates the go1.25 test job to go1.25.13 which contains the
same fixes.
2026-08-14 19:18:26 +01:00
Nick Craig-Wood 00593a96fe build: update golang.org/x/image to v0.45.0 to fix CVE-2026-46603
CVE-2026-46603: excessive memory allocation during VP8L decoding

This also updates golang.org/x/text to v0.41.0 as a dependency.
2026-08-14 19:18:26 +01:00
Nick Craig-Wood 64c9a28651 Add Dave to contributors 2026-08-14 19:18:26 +01:00
Nick Craig-Wood f81ccdf786 Add Shurong Cao to contributors 2026-08-14 19:18:26 +01:00
Nick Craig-Wood 8aba1c4378 Add Morax to contributors 2026-08-14 19:18:26 +01:00
DaveandNick Craig-Wood 6e0c71bd27 vfs/vfscache: fix reader deadlock when the item size drops below the read offset
_dispatchWaiters decided whether a waiter was satisfied by clipping its
range against dls.src.Size(), the size of the fs.Object snapshot taken
when the Downloaders was created. _ensureDownloader decided whether to
start a downloader from Item.FindMissing, which clips against
item.info.Size instead.

When item.info.Size dropped below the offset a waiter was parked on while
the source object still reported the full size, the two disagreed.
_ensureDownloader found nothing missing so it started no downloader, and
_dispatchWaiters found the range absent so it never released the waiter.
Nothing was downloaded and no error was produced, so the error count never
reached maxErrorCount and the waiter was never woken. The reader blocked
forever with nothing logged at any level.

Wake a waiter when FindMissing reports nothing left to download for it as
well as when its data has arrived. Since _ensureDownloader starts a
downloader only when FindMissing is non empty, a waiter with nothing
missing has nothing that could ever wake it.

Fixes #9769
2026-08-14 18:51:49 +02:00
Shurong CaoandNick Craig-Wood bb7c081ec5 sync: add pattern-based transfer ordering
Allow --order-by to rank files using comma-separated rclone path globs. Patterns are evaluated in order, unmatched files are placed last, and path ordering makes ties deterministic.

Fixes #3975
2026-08-14 18:47:15 +02:00
MoraxandNick Craig-Wood 9f93da3299 operations: fall back when source ignores ranges
When a multi-thread source ignores ranged reads, abort the partial
destination and retry the copy as a single stream. Reset accounting
before the fallback so transfer progress remains accurate.
2026-08-14 18:40:40 +02:00
MoraxandNick Craig-Wood 1003280bb1 webdav: reject ignored ranged responses
Validate ranged GET responses before returning their bodies. Close invalid
responses, retry malformed partial responses through the pacer, and return
without retrying when a server deterministically ignores Range.

Fixes #6980
2026-08-14 18:40:40 +02:00
MoraxandNick Craig-Wood 69e5aff2a9 lib/rest: validate ranged responses
Add response validation for calls made with Range open options. Verify
Content-Range, Content-Length, response status, and the complete
representation size before a backend accepts the response body.

Return a shared sentinel when a server ignores a partial range so callers
can avoid retrying the same unsupported request.
2026-08-14 18:40:40 +02:00
Nick Craig-Wood 5a0b7d6746 crypt: fix hash mismatches with no_data_encryption on backends which check upload hashes
Before this change, when no_data_encryption was set, uploads from
local disk advertised the hash of the encrypted data even though the
data was uploaded unencrypted.

On backends which check upload hashes (eg b2) this made uploads of
small files fail with errors like "Checksum did not match data
received", and made chunked uploads store an incorrect hash so the
files failed their checksum on download with "corrupted on transfer:
SHA1 hashes differ".

See: https://forum.rclone.org/t/sha1-mismatches-on-b2-with-no-data-encryption-true/54121
2026-08-14 09:54:00 +01:00
Nick Craig-Wood afb4fa5f2b Add Pastalikek65 to contributors 2026-08-14 09:54:00 +01:00
Nick Craig-Wood d22c4509aa Add Dean Chen to contributors 2026-08-14 09:54:00 +01:00