Commit Graph
344 Commits
Author SHA1 Message Date
Nick Craig-Wood 267f1560d2 azureblob: add arrowlist package for Apache Arrow blob listing
This adds a temporary subpackage implementing the experimental "Blob
Listing with Apache Arrow" feature on top of the released Azure azblob
SDK.

The Azure SDK for Go supports Arrow listing on its unreleased
feature/storage/bifrost branch (commit c6fa341ca22b) but that can't be
used in a release. This package exposes the same options and pager
interface as the experimental SDK, built on the released SDK's public
API plus verbatim copies of its Arrow decoder and of the two internal
auth policies (shared key signing and storage bearer challenge) which
the released SDK does not export. Copied files carry MIT attribution
headers pinned to their upstream source.

When Arrow listing ships in a released azblob the whole package will be
deleted and callers pointed back at the SDK - the interface is
deliberately source compatible to make that a mechanical change.

Also exports auth.Transporter so the new package's pipeline can use
rclone's fshttp transport, and adds github.com/apache/arrow-go/v18 as a
direct dependency for decoding the Arrow IPC stream.
2026-07-29 20:16:22 +01:00
Nick Craig-Wood a110fa5d47 build: update google.golang.org/grpc to fix multiple security problems
Multiple security vulnerabilities have been identified and addressed
in grpc-go affecting the xDS RBAC authorization engine
(internal/xds/rbac) and the HTTP/2 transport server implementation
(internal/transport).

This updates to master to add the fix which allows it to compile on
plan9.
2026-07-23 17:03:42 +01:00
maximilizeandNick Craig-Wood 2f3895fa3c archive/squashfs: fix reading images with no fragment or xattr table
Bump go-diskfs to v1.9.4, which contains diskfs/go-diskfs#413 handling
squashfs images that have no fragment table or no xattr table. On v1.9.3
these images fail to list at all (ReadDir errors, or a nil xattr-table
dereference inside the library).

Adapt the backend to the v1.9.4 io/fs API: ReadDir now returns
io/fs.DirEntry (metadata via DirEntry.Info), and files are opened via
FileSystem.OpenFile instead of the removed FileStat.Open. The path
conversion added in c10eb47 (toIOFS) is reused.

Add a regression test with two committed fixtures built from trivial
placeholder content: an empty directory (no fragment table) and a small
tree whose superblock has the NO_XATTRS flag set while inodes still carry
an xattr index.

Fixes #9004
2026-07-22 11:30:21 +01:00
Nick Craig-Wood cbbf588c47 build: update all dependencies
This fixes the code to compile with the updated dependencies:

- squashfs: implement the new Path method required by the go-diskfs
  backend.Storage interface, returning an empty string as there is no
  underlying path.
- dropbox: convert to and from the SDK's new DBXTime type (an alias
  for time.Time) for ClientModified, TimeInvited and Expires.
- dropbox: remove a stray debug fmt.Printf from the shared folder
  listing.
- pin go-systemd to v22.6.0 to fix netbsd builds
  go-systemd v22.7.0 uses unix.ClockGettime and unix.CLOCK_MONOTONIC
  which golang.org/x/sys does not define for netbsd, so the build fails.
  See: https://github.com/coreos/go-systemd/issues/512
- pin google.golang.org/grpc to v1.80.0 to fix plan9 builds
  grpc v1.81.0 and later use syscall.Errno, syscall.ECONNRESET and
  syscall.ECONNABORTED which do not exist on plan9, so the build fails
  See: https://github.com/grpc/grpc-go/issues/9253
2026-07-19 19:01:12 +01:00
Nick Craig-Wood f0d77d07a2 mega: fix files reappearing in listings after being renamed
This updates go-mega to fix the handling of the event stream when a
file is moved or renamed. The server reports a move as a delete of
the old node followed by an add of the same node, and go-mega was
losing track of the node's identity in the process, which could make
the old name reappear in listings and leave the backend unable to
remove it.
2026-07-17 18:29:39 +01:00
Nick Craig-Wood bb17d07eb7 protondrive: fix gopenpgp: invalid data: user ID signature with wrong type on custom-domain account
Update to github.com/ProtonMail/gopenpgp/v3 from v2 by updating to

- github.com/rclone/go-proton-api@v1.0.3
- github.com/rclone/Proton-API-Bridge@v1.0.4

Fixes #9434
2026-07-13 15:22:47 +01:00
jzunigax2andNick Craig-Wood 0ce75add5a internxt: add Move and DirMove methods for server-side file and directory operations 2026-07-09 12:42:21 +01:00
dithwickandGitHub bf2945bf40 smb: fix for IBM iSeries and signature verification
This commit updates go-smb2 to a version that fixes the following issues.

- Issue connecting to SMB share from IBM iSeries
  (see https://forum.rclone.org/t/smb-shares-on-ibm-iseries-update/53550)
- Fix for signature verification for async STATUS_PENDING response which
  resolves a transfer issue with Lenovo DSS-G.
  (see https://github.com/CloudSoda/go-smb2/pull/48)

It also provides additional security hardening, fixing panics on malicious
server input
(see https://github.com/CloudSoda/go-smb2/pull/51)
2026-07-01 12:00:26 +01:00
Nick Craig-Wood b59a17e5e8 mega: fix hard deleted files reappearing in listings - fixes #9554
The mega backend keeps the whole account as an in-memory tree. go-mega's
Delete with destroy=true (hard_delete) removed the node from its lookup
table but left it in its parent's children, so a hard deleted file kept
showing up in directory listings until the tree was reloaded. In a
long-running mount or serve the file reappeared once the VFS directory
cache expired and re-read the backend.

Update go-mega to pick up the fix, and let the ghost test exercise
whichever delete mode the remote is configured with.
2026-07-01 10:06:53 +01:00
Nick Craig-Wood fee55edd15 build: update golang.org/x/image to v0.43.0 to fix image decoding vulnerabilities
govulncheck reported four vulnerabilities in golang.org/x/image v0.41.0,
all reachable via the internxt backend's thumbnail/image upload path:

- CVE-2026-46604 (GO-2026-5066): panic decoding a TIFF image with an
  out-of-bounds strip offset
- CVE-2026-46602 (GO-2026-5062): unbounded memory use from lack of a
  limit on TIFF tile sizes
- CVE-2026-46601 (GO-2026-5061): panic on a WEBP VP8 alpha channel size
  mismatch
- CVE-2026-33813 (GO-2026-4961): panic decoding a large WEBP image on
  32-bit platforms

Updating to v0.43.0 fixes all four.
2026-06-30 10:56:49 +01:00
Nick Craig-Wood d204b29cce ncdu: fix duplicated keystrokes on Windows by pinning tcell to v2.9.0
Since v1.74.0, ncdu doubled every keystroke on Windows: pressing Enter
entered a directory and immediately descended a level deeper, "n" sorted
by name and then reversed the order, and so on. Arrow keys were
unaffected.

The cause is an upstream regression in github.com/gdamore/tcell/v2
pulled in by the v2.9.0 -> v2.13.8 bump in commit 23917555. The same
regression affects lazygit and micro.

There is no fixed tcell release, so pin tcell back to v2.9.0, the last
version before that commit, until the input handling is fixed upstream.

See: https://github.com/gdamore/tcell/issues/1124
See: https://github.com/jesseduffield/lazygit/issues/5344
Fixes #9539
2026-06-22 12:16:27 +01:00
Nick Craig-Wood 3d246a2aea serve s3: stream multipart uploads to the backend instead of buffering in memory
Previously serve s3 buffered every part of a multipart upload in memory
(in the gofakes3 S3 library) and concatenated them when the upload
completed, so memory use grew with the size of the upload.

serve s3 now streams the parts, in part-number order, into a single
PutStream upload to the underlying remote, which performs its own upload
with bounded memory. The whole file is never held in memory - memory use
is bounded by the parts in flight. This works for any remote that
supports PutStream (nearly all, including crypt) and for any part size,
so clients that don't produce uniform-sized parts (for example
PostgreSQL backup tools such as pgBarman and pgBackRest) work too.

Parts must arrive in ascending, contiguous part-number order; parts
uploaded out of order are buffered until their turn, and there is no
per-part retry (a failure aborts the whole upload). These trade-offs are
documented.

Passing --disable-multipart-streaming, or using a remote without
PutStream, reverts to buffering the parts in memory (the previous
behaviour); a one-off NOTICE is logged the first time this happens.

Fixes #7453
2026-06-11 12:30:19 +01:00
Nick Craig-Wood 79f42d37ff serve s3: fix multipart ListParts pagination returning wrong part numbers - fixes #9460
Listing the parts of a multipart upload with a MaxParts smaller than
the number of parts returned wrong PartNumber values and duplicate
parts on pages after the first, and the part number marker regressed
instead of advancing, causing clients to loop forever.

This bumps github.com/rclone/gofakes3 to v0.0.6 pick up the fix.
2026-05-29 17:04:01 +01:00
Nick Craig-Wood a03070c960 build: update golang.org/x dependencies to address multiple CVEs
Upgrade to v0.55.0 of golang.org/x/net in order to address:

- CVE-2026-42506: html: incorrect handling of namespaced elements in foreign content
- CVE-2026-39821: idna: failure to reject ASCII-only Punycode-encoded labels
- CVE-2026-42502: html: incorrect handling of HTML elements in foreign content
- CVE-2026-25680: html: denial of service when parsing arbitrary HTML
- CVE-2026-25681: html: incorrect handling of character references in DOCTYPE nodes
- CVE-2026-27136: html: duplicate attributes can cause XSS

Upgrade to version v0.52.0 of golang.org/x/crypto to address:

- CVE-2026-46598: ssh/agent: pathological inputs can lead to client panic
- CVE-2026-46597: ssh: byte arithmetic causes underflow and panic
- CVE-2026-39828: ssh: bypass of certificate restrictions
- CVE-2026-39835: ssh: server panic during CheckHostKey/Authenticate
- CVE-2026-39833: ssh/agent: key constraints not enforced
- CVE-2026-39832: ssh/agent: agent constraints dropped when forwarding keys
- CVE-2026-39827: ssh: memory leak when rejecting channels can lead to DoS
- CVE-2026-39830: ssh: client can cause server deadlock on unexpected responses
- CVE-2026-39829: ssh: pathological RSA/DSA parameters may cause DoS
- CVE-2026-39831: ssh: bypass of FIDO/U2F security keys physical interaction
- CVE-2026-39834: ssh: infinite loop on large channel writes
- CVE-2026-42508: ssh/knownhosts: auth bypass via unenforced @revoked status
- CVE-2026-46595: ssh: VerifiedPublicKeyCallback permissions skip enforcement

Upgrade to version v0.41.0 of golang.org/x/image to address:

- CVE-2026-42500: bmp: panic when reading out of bound palette index
- CVE-2026-33809: tiff: excessive resource consumption in PackBits decompression

Upgrade to version v0.45.0 of golang.org/x/sys to address:

- CVE-2026-39824: windows: integer overflow in NewNTUnicodeString
2026-05-22 11:54:29 +01:00
dependabot[bot]Nick Craig-Wooddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
72fcef5183 build: bump github.com/go-git/go-billy/v5 to 5.9.0 to fix CVE-2026-44740 (again)
The previous commit had no content due to copilot deleting it as I was
merging it!

Bumps [github.com/go-git/go-billy/v5](https://github.com/go-git/go-billy) from 5.8.0 to 5.9.0.
- [Release notes](https://github.com/go-git/go-billy/releases)
- [Commits](https://github.com/go-git/go-billy/compare/v5.8.0...v5.9.0)

---
updated-dependencies:
- dependency-name: github.com/go-git/go-billy/v5
  dependency-version: 5.9.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-13 20:01:04 +01:00
Nick Craig-Wood fc50f3e731 build: update golang.org/x/net to v0.53.0 to fix CVE-2026-33814 2026-05-08 10:59:20 +01:00
Nick Craig-Wood 0737599cd4 protondrive: fix segfault when copying files missing revision metadata
When a Proton Drive file has no active revision attributes,
readMetaDataForLink returns a nil FileSystemAttrs and Object.originalSize
is left as nil. Object.Open then dereferenced this nil pointer when
calling fs.FixRangeOption, causing a SIGSEGV during copy.

Use Object.Size() instead, which already implements the correct fallback
to the link size when originalSize is unavailable.

This updates the github.com/rclone/Proton-API-Bridge package to fix a
segfault when reading files with no metadata.

Fixes #9377
Fixes #9117
2026-05-05 15:02:34 +01:00
Nick Craig-Wood ef26e6d26d protondrive: route HTTP through rclone's transport
The Proton Drive backend constructed the upstream Proton-API-Bridge
without ever passing rclone's HTTP transport. As a result none of
rclone's HTTP flags reached Proton: --dump headers, --dump bodies,
--no-check-certificate, --user-agent, --bind, --ca-cert, --header,
--tpslimit etc. all silently did nothing for this remote, and HTTP
traffic was invisible to -vv.

Pass fshttp.NewTransport(ctx) through the new Config.Transport hook on
the bridge, which forwards it to the updated go-proton-api's
WithTransport option and so to the underlying resty client.
2026-05-05 09:43:39 +01:00
Nick Craig-Wood 0812cee9e9 protondrive: fix server-side moveto and DirMove against current API
Bump go-proton-api and Proton-API-Bridge to versions that send the new
NameSignatureEmail field and omit NodePassphraseSignature/SignatureEmail
for ordinary nodes, matching the schema accepted by the Proton Drive
API. Without this rclone moveto, --backup-dir, server-side rename and
DirMove all failed with "value cannot be empty" / "outdated app" 422.

Fixes #8512
2026-04-30 12:40:52 +01:00
Nick Craig-Wood 7447f23d66 build: Update all packages with pseudo versions which aren't v0.0.0
These don't get updated by `go get` and were missed until recently. It
turns out `github.com/jlaffaye/ftp` was the only one that needed
updating.
2026-04-28 09:24:51 +01:00
Nick Craig-Wood 7c56eff1a7 rc: add user directories to core/disks and filter mounts better 2026-04-27 15:07:33 +01:00
dlaumenandNick Craig-Wood a71c6f0459 protondrive: update to latest go-proton-api to use new host 2026-04-27 14:33:47 +01:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1f44b2ee93 build: bump github.com/Azure/go-ntlmssp to 0.1.1 to fix CVE-2026-32952
Fix CVE-2026-32952: A malicious NTLM challenge message can causes an slice out
of bounds panic, which can crash any Go process using ntlmssp.Negotiator as an
HTTP transport.

This is in use in rclone in the webdav backend to access sharepoint.

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-24 16:18:05 +01:00
Nick Craig-Wood 9dedb12b9d build: update golang.org/x/image/webp to v0.39.0 to fix CVE-2026-33813
Parsing a WEBP image with an invalid, large size panics on 32-bit platforms.

This only affects users on 32 bit platforms using the Internxt backend.

See: https://pkg.go.dev/vuln/GO-2026-4961
2026-04-23 18:46:18 +01:00
Nick Craig-Wood 513d2fc17e serve nfs: fix EOF flag in READ response not being set when read reaches end of file
This was fixed in the upstream library

See: https://github.com/willscott/go-nfs/pull/149
2026-04-15 17:53:00 +01:00
Nick Craig-Wood 0fa16e8638 build: update all dependencies
This also streamlines the update process in RELEASE.md and the Makefile
2026-04-08 12:08:28 +01:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
f757e6c0a7 build: fix Denial of Service due to Panic in AWS SDK for Go v2 SDK EventStream Decoder
Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.96.0 to 1.97.3.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.96.0...service/s3/v1.97.3)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-version: 1.97.3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-08 09:03:49 +01:00
SuyunandGitHub e7deebdf40 build: fix loong64 and s390x build 2026-04-06 10:54:17 +01:00
EndurielandNick Craig-Wood da7a3d8d64 filen: update SDK version
- increase timeout for http requests
- only use a single URL per request type
2026-03-30 22:56:32 +01:00
dependabot[bot]andNick Craig-Wood 2362b4ca50 build(deps): bump golang.org/x/image from 0.36.0 to 0.38.0
Bumps [golang.org/x/image](https://github.com/golang/image) from 0.36.0 to 0.38.0.
- [Commits](https://github.com/golang/image/compare/v0.36.0...v0.38.0)

---
updated-dependencies:
- dependency-name: golang.org/x/image
  dependency-version: 0.38.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-30 22:55:35 +01:00
dependabot[bot]andNick Craig-Wood 589d45a0bf build(deps): bump github.com/buger/jsonparser from 1.1.1 to 1.1.2
Bumps [github.com/buger/jsonparser](https://github.com/buger/jsonparser) from 1.1.1 to 1.1.2.
- [Release notes](https://github.com/buger/jsonparser/releases)
- [Commits](https://github.com/buger/jsonparser/compare/v1.1.1...v1.1.2)

---
updated-dependencies:
- dependency-name: github.com/buger/jsonparser
  dependency-version: 1.1.2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-23 17:05:35 +00:00
dependabot[bot]andNick Craig-Wood 7dfc5b3076 build(deps): bump google.golang.org/grpc from 1.79.1 to 1.79.3
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.79.1 to 1.79.3.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.79.1...v1.79.3)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-version: 1.79.3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-19 11:02:37 +00:00
nielash 72c561d209 bisync: auto-generate rc help docs
This adds a go generate ./cmd/bisync command to autogenerate the bisync rc docs,
including the list of params.
2026-03-03 16:13:00 -05:00
Nick Craig-Wood 056adc97d6 build: update to golang.org/x/net v0.51.0 to fix CVE-2026-27141 #9220
See: https://pkg.go.dev/vuln/GO-2026-4559
2026-02-27 11:24:16 +00:00
jzunigax2andNick Craig-Wood 7eed0b6825 internxt: fix Entry doesn't belong in directory errors on windows 2026-02-26 11:13:16 +00:00
Nick Craig-Wood cf97f250df build: update all dependencies
Could not update github.com/coreos/go-systemd as the new version doesn't build under freebsd

See: https://github.com/coreos/go-systemd/issues/509
2026-02-18 11:33:48 +00:00
Nick Craig-Wood f14945f9c1 build: update to go1.26 and make go1.25 the minimum required version 2026-02-18 11:33:48 +00:00
José ZúnigaandGitHub 33859568d6 internxt: implement re-login under refresh logic, improve retry logic - fixes #9174 2026-02-13 19:18:51 +00:00
albertonyandNick Craig-Wood 54c2078f25 build: bump github.com/go-chi/chi/v5 from 5.2.3 to 5.2.5 to fix GO-2026-4316 2026-02-11 11:34:29 +01:00
EndurielandNick Craig-Wood ed5bd327c0 filen: fix 32 bit targets not being able to list directories Fixes #9142
or do pretty much anything,
this was caused by timestamps not being read to 64 bit integers
2026-02-05 12:06:20 +00:00
Nick Craig-Wood fac7ba4bd0 internxt: remove use of CVE laden github.com/disintegration/imaging 2026-01-30 17:28:52 +00:00
85d13c4e34 internxt: add Internxt backend - fixes #7610
Co-Authored-By: lullius <lullius@users.noreply.github.com>
Co-Authored-By: StarHack <StarHack@users.noreply.github.com>
2026-01-30 16:02:50 +00:00
Nick Craig-Wood 824257583c protondrive: update to use forks of upstream modules
This updates rclone to use forks of the upstream proton drive modules
in preparation for making changes.

The go-proton-api modules has had changes from master merged so rclone
and Proton-API-Bridge are using the same version.
2026-01-28 11:38:38 +00:00
Nick OwensandGitHub 264e75d892 cmount: fix OpenBSD mount support.
this pulls in https://github.com/winfsp/cgofuse/pull/100 to fix OpenBSD
mount support.

part of #1727
2026-01-27 13:47:00 +00:00
EndurielandNick Craig-Wood 63cfe260a2 filen: swap to blake3 hashes 2026-01-22 00:11:46 +00:00
EndurielandNick Craig-Wood 9bef7f0dbf filen: add Filen backend - Fixes #6728 2026-01-13 12:50:27 +00:00
Vladislav TropnikovandGitHub a99d155fd4 s3: The ability to specify an IAM role for cross-account interaction 2025-11-29 13:53:00 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
bd99e05ff0 build: bump golang.org/x/crypto from 0.43.0 to 0.45.0 to fix CVE-2025-58181
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-11-20 13:09:29 +00:00
Nick Craig-Wood 31df39d356 build: fix tls: failed to verify certificate: x509: negative serial number
Before Go 1.23, x509.ParseCertificate accepted certificates with
negative serial numbers. Rejecting these certificates caused a small
number of users to see this error.

From Go 1.23 debug flags can be added to go.mod so this change adds a
debug flag to ensure negative serial numbers are still allowed since
this is a spec violation, not a security issue.

See: https://forum.rclone.org/t/ssl-validation-broken-between-v1-69-1-latest-version/
2025-11-14 12:51:17 +00:00
Nick Craig-Wood 2ebfedce85 build: update all dependencies 2025-11-12 10:36:30 +00:00