Commit Graph
424 Commits
Author SHA1 Message Date
Nick Craig-Wood b5bea683c5 build: modernize with "go fix -minmax": use min and max builtins 2026-08-21 12:23:31 +01:00
VXNCXNXandNick Craig-Wood c667e53638 lib/transform: fix panic in truncate_keep_extension
Return error when extension is longer than truncation limit.
2026-08-18 11:13:23 +01:00
MoraxandNick Craig-Wood 69e5aff2a9 lib/rest: validate ranged responses
Add response validation for calls made with Range open options. Verify
Content-Range, Content-Length, response status, and the complete
representation size before a backend accepts the response body.

Return a shared sentinel when a server ignores a partial range so callers
can avoid retrying the same unsupported request.
2026-08-14 18:40:40 +02:00
Nick Craig-Wood 5b7cc097e3 multipart: fix chunked uploads storing truncated objects when the source ends early
If the source supplied fewer bytes than its declared size, the
multipart upload was completed anyway, storing a truncated object and
reporting a successful upload.

Check the number of bytes read from the source against the declared
size before finalising and abort the upload with an error if they do
not match.

This affects all backends which use lib/multipart for chunked uploads:
azureblob, b2, drime, internxt, oracleobjectstorage, s3 and shade.

This was found by the new FsPutShortEOF and TestRcatSizeShortEOF
integration tests.
2026-08-11 19:32:12 +01:00
Loi NguyenandNick Craig-Wood 7f6207fae2 lib/batcher: prevent commits racing shutdown - fixes #9687 2026-08-01 12:29:44 +01:00
am-at-enrollvbandGitHub 5dd34275dc serve: pass the client IP address to the auth proxy - fixes #4499
The auth proxy was only given the user and their password or public
key, so a proxy program had no way to restrict logins to particular
networks, or to record where an authentication attempt came from.

The JSON sent to the program now has a client_ip key holding the bare
IP the client connected from, with the port stripped so IPv6 arrives
as 2001:db8::1 rather than [2001:db8::1]:52344. An IPv4-mapped IPv6
address is reported as plain IPv4 so that a client arriving over a
dual-stack listener still matches IPv4 networks. The key is omitted
when the client has no IP address.

The IP is also mixed into the backend cache key. That is needed as the
program is only run on a cache miss, so a client from a
non-allowlisted address presenting valid credentials within the 5
minute cache lifetime would get a cache hit and be let in without the
program being consulted at all.
2026-08-01 12:25:06 +01:00
Nick Craig-Wood 9ee9d0a0ca Version v1.75.0 2026-07-31 16:56:33 +01:00
Nick Craig-Wood 043e58b83c lib/http: use TLS on all --addr listeners when --cert and --key are set GHSA-mfvx-7rcj-9m5g
When --cert and --key were supplied TLS was only applied to the listener if
exactly one --addr was given. With two or more --addr flags every listener
without an explicit tls:// prefix silently served cleartext HTTP, so adding a
second --addr to an HTTPS server quietly disabled TLS on both.

Now when TLS is configured every listener serves TLS. An individual listener
can be prefixed with http:// to serve unencrypted HTTP on that address, and
tls:// still marks a listener as TLS explicitly. Using a tls:// address
without --cert and --key is now an error instead of silently serving
cleartext with an https:// URL.

Addresses GHSA-mfvx-7rcj-9m5g finding 3.
2026-07-31 13:21:59 +01:00
Nick Craig-Wood 21d8cd3b92 lib/proxy: fix unbounded HTTP CONNECT headers causing OOM GHSA-xhf4-832v-7xcr CVE-PENDING
Before this change rclone read a proxy response with http.ReadResponse
over an unrestricted buffered reader. A malicious or compromised
configured proxy, or an active on-path actor controlling a plaintext
HTTP-proxy hop, can grow memory until the process fails.

This fixes the problem by restrincting the read to 1MB maximum.
2026-07-31 13:21:59 +01:00
Nick Craig-Wood 59b513b0e7 webdav: fix HTTPS to HTTP redirects leaking credentials GHSA-h4mf-4v27-hggj
A server that redirects an HTTPS request to a plaintext HTTP URL on the
same host would cause Go's http.Client to replay the configured
credentials (Basic Authorization, cookies, secret headers) over the
network in cleartext.

Refuse to follow such downgrade redirects by default in lib/rest and wire
the webdav backend's client to use it. The `auth_redirect` option remains
the opt-in escape hatch for servers that legitimately need auth preserved
across redirects.

Fixes GHSA-h4mf-4v27-hggj
2026-07-31 13:21:59 +01:00
31604f623f http: add Prefer to CORS Access-Control-Allow-Headers header
The rclone web GUI uses the Prefer header for sync/copy operations,
but the CORS middleware's preflight response did not include it in
Access-Control-Allow-Headers. This caused the browser to block
remote GUI copy requests with a CORS policy error:

  Request header field prefer is not allowed by
  Access-Control-Allow-Headers in preflight response.

Add 'Prefer' to the allowed headers list so remote GUI
access works correctly for file copy operations.

Fixes #9614

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 11:55:33 +01:00
Nick Craig-Wood 8b812fff28 fs: fix passwords and tokens appearing in the debug log during rclone config
Previously running rclone config (or driving it via the rc API or web
GUI) with -vv would write secrets to the debug log.

This was dangerous as users debugging a failing config flow often
paste their -vv logs into the forum or GitHub issues.

These values are now redacted from the log as "XXX". Values whose
option is known are only redacted if the option is marked IsPassword
or Sensitive, so normal answers remain visible.

Use --dump auth to see the unredacted values when debugging a config
flow - rclone prints a warning that secrets will appear in the log
when this is in effect.

This was discovered by CodeQL: https://github.com/rclone/rclone/security/code-scanning/182
2026-07-16 16:11:22 +01:00
Nick Craig-Wood b5a81dab76 drive, googlephotos: warn in config wizard before using the shared client_id #9580
rclone's shared Google Drive and Google Photos client_id is being
retired and will stop working during 2026. When creating a new remote
that would use it, the config wizard now warns the user and asks the
user to enter their own client_id and secret instead. Service account
and environment auth are unaffected as they don't use the shared
client_id.

See: https://forum.rclone.org/t/google-drive-and-google-photos-users-action-required/54005
2026-07-07 12:35:37 +01:00
Nick Craig-Wood d03eb58586 drive, googlephotos: warn when using rclone's shared client_id #9580
The shared Google Drive and Google Photos client_id is being retired and
will stop working during 2026. Warn users who rely on it (ie who have not
configured their own client_id) so they can create their own in advance.

The warning is only shown for auth flows that actually use the shared
client_id, not for service account, environment or anonymous auth.

See: https://forum.rclone.org/t/google-drive-and-google-photos-users-action-required/54005
2026-07-07 12:35:37 +01:00
Amit MishraandNick Craig-Wood cb41e42d04 fs: fix negative offset when a suffix Range request exceeds object size
A Range header requesting a suffix longer than the object (e.g.
"bytes=-90407" against a 5 byte object) caused RangeOption.Decode to
compute a negative offset (size - End), which serve.Object then used
directly as a slice/seek offset and panicked with "slice bounds out of
range". FixRangeOption (used by backends like OneDrive/Box that lack
native suffix-range support) had the same root cause: it produced a
RangeOption with a negative Start, which Header() silently dropped,
turning the request into the wrong byte range instead of erroring or
serving the whole object.

Per RFC 7233 section 2.1, when the suffix-length exceeds the
representation size, the entire representation should be served.
Clamp the computed offset/start to 0 in both places.

Fixes #6310
2026-07-04 09:11:52 +01:00
Hakan İSMAİLandGitHub f2832217aa rc: include OAuth authorization URL in rc config/oauthstatus response 2026-05-28 17:51:58 +01:00
FTCHDandGitHub 35b95aa2a3 rc: add config/oauthstop and config/oauthstatus to control oauth listener
rclone starts a http listener if the user is adding a remote that requires
oauth. The problem is that sometimes this breaks or the user stops midway with
no way of us knowing that. Once that happens, clients of the api break. In the
best scenario, new remotes cannot be configured without a restart.

This change adds 2 oauth related endpoints, one to get the status of the listener
and one to stop it. This allows clients to go back to normal without having to
restart rclone.

- config/oauthstop - cancels an in-progress OAuth flow and releases the port.
- config/oauthstatus - reports "running" / "stopped".
2026-05-25 19:59:09 +01:00
Nick Craig-Wood ad8a108453 lib/http: replace deprecated h2c.NewHandler with http.Server.Protocols
The golang.org/x/net/http2/h2c package was deprecated in v0.54.0 in
favour of setting the http.Server Protocols field to enable unencrypted
HTTP/2.

This replaces the h2c.NewHandler wrapping added in e863f751f with
http.Server.Protocols, which is supported by the standard library
since Go 1.24.

Note that the stdlib only supports HTTP/2 prior-knowledge on cleartext
connections, not HTTP/1.1 Upgrade: h2c negotiation. In practice clients
use prior-knowledge or require TLS, so this should not affect users.
2026-05-22 12:20:24 +01:00
d86b72c405 serve: support custom http response headers
Co-authored-by: Tim Schumacher <tim@tschumacher.net>
2026-05-06 12:41:15 +01:00
KTibowandNick Craig-Wood 7200e377dd oauthutil: clarify token replacement prompt wording
The previous wording "Already have a token - refresh?" was misleading
because answering yes triggers a full re-authorization flow, not an
OAuth2 refresh token grant. Updated to "Token already configured -
replace it?" to accurately describe what happens.

Also updated the SugarSync backend which has its own copy of the prompt,
and the docs for box, drive, and onedrive that reference it.
2026-05-06 10:51:16 +01:00
Nick Craig-Wood aa031c51cc Version v1.74.0 2026-05-01 15:56:56 +01:00
albertony f1a5e44d50 docs: fix code comment regarding cmount tag 2026-04-20 20:09:52 +02:00
TheBabuandNick Craig-Wood e863f751f0 lib/http: Add HTTP/2 cleartext support in server configuration
Enable HTTP/2 cleartext support for non-TLS listeners.
2026-04-09 17:00:18 +01:00
Andrew FurmanandGitHub 639bd88959 lib/rest: fix URLPathEscapeAll breaking WebDAV servers (eg nzbdav) with strict path matching
URLPathEscapeAll was only passing [A-Za-z0-9/] through unencoded, causing
it to percent-encode RFC 3986 unreserved characters (-, ., _, ~). Per RFC
3986 §2.3, unreserved characters MUST NOT be percent-encoded, and a URI
that unnecessarily encodes them is not equivalent to one that does not.

Servers that perform strict path matching without normalising
percent-encoded characters will reject the over-encoded form with a 404.

Before: /files/my-report.pdf → /files/my%2Dreport%2Epdf
After:  /files/my-report.pdf → /files/my-report.pdf

Reserved characters (spaces, semicolons, colons, etc.) continue to be
encoded as before.
2026-03-12 21:58:22 +00:00
Nick Craig-Wood 307f1edaf4 operations: add method to real time account server side copy
Before this change server side copies would show at 0% until they were
done then show at 100%.

With support from the backend, server side copies can now be accounted
in real time. This will only work for backends which have been
modified and themselves get feedback about how copies are going.

If the transfer fails, the bytes accounted will be reversed.
2026-03-03 14:01:11 +00:00
Leon BrocardandGitHub 9be4fc8c2b serve http: add gzip compression
Add gzip compression for directory listings and text assets served over HTTP.

This reduces the rclone repository file listing from 40 kB to 8 kB and reduces
the rclone MANUAL.txt from 2.7 MB to 700 kB.

This makes listings and assets served across the network load faster.

The compression level of 5 should be a good balance between size and speed.
2026-02-26 17:18:52 +00:00
Brian BockelmanandNick Craig-Wood cf240b6c0f webdav: permit redirects on PROPFIND for metadata
The WebDAV implementation already permits redirects on PROPFIND for
listing paths in the `listAll` method but does not permit this for
metadata in `readMetaDataForPath`.  This results in a strange experience
for endpoints that heavily use redirects -

```
rclone lsl endpoint:
```

functions and lists `hello_world.txt` in its output but

```
rclone lsl endpoint:hello_world.txt
```

Fails with a HTTP 307.

The git history for this setting indicates this was done to avoid
an issue where redirects cause a verb change to GET in the Go HTTP
client; it does not appear to be problematic with HTTP 307.

To fix, a new `CheckRedirect` function is added in the `rest` library
to force the client to use the same verb across redirects, forcing this
for the PROPFIND case.
2026-02-26 16:23:52 +00:00
Romāns PotašovsandNick Craig-Wood 6159ea9cf5 webdav: add missing headers for CORS
fix #7492
2026-02-26 16:23:07 +00:00
FTCHDandNick Craig-Wood 77892a8289 http: dark mode for browser 2026-02-26 12:29:40 +00:00
Nick Craig-Wood b2866f0291 build: modernize Go code with go fix for go1.25 2026-02-18 12:11:52 +00:00
Nick Craig-Wood 627b763d4b lib/rest: remove go1.24 workaround now go1.25 is the minimum 2026-02-18 11:33:48 +00:00
Nick Craig-Wood 15a9c0fd36 pacer: re-read the sleep time as it may be stale
Before this change we read sleepTime before acquiring the pacer token
and uses that possibly stale value to schedule the token return. When
many goroutines enter while sleepTime is high (e.g., 10s), each
goroutine caches this 10s value. Even if successful calls rapidly
decay the pacer state to 0, the queued goroutines still schedule 10s
token returns, so the queue drains at 1 req/10s for the entire herd.
This can create multi‑minute delays even after the pacer has dropped
to 0.

After this change we refresh the sleep time after getting the token.

This problem was introduced by the desire to skip reading the pacer
token entirely when sleepTime is 0 in high performance backends (eg
s3, azure blob).
2026-02-17 12:27:17 +00:00
Nick Craig-Wood 8b85ffbf03 pacer: fix deadlock between pacer token and --max-connections
It was possible in the presence of --max-connections and recursive
calls to the pacer to deadlock it leaving all connections waiting on
either a max connection token or a pacer token.

This fixes the problem by making sure we return the pacer token on
schedule if we take it.

This also short circuits the pacer token if sleepTime is 0.
2026-02-17 12:27:17 +00:00
Nick Craig-Wood f42aa5a586 Version v1.73.0 2026-01-30 20:53:56 +00:00
yyandGitHub b728929f44 docs: fix typos in comments and messages 2026-01-18 20:48:33 +01:00
Nicolas DessartandNick Craig-Wood a64a8aad0e sftp,ftp: add http proxy authentication support
This change supports the `http://user:pass@host:port` syntax for the
http_proxy setting.
2026-01-08 16:31:11 +00:00
Nick Craig-Wood d9895fef9d lib/rest: add opts.MultipartContentType to explicitly set Content-Type of attachements
Before this the standard library set it to application/octet-stream for some reason
2026-01-08 12:05:37 +00:00
dougalandNick Craig-Wood 8c7b7ac891 dircache: allow empty string as root parent id
This was causing an internal error with the drime backend which has the
root parent id as an empty string. This shouldn't affect anything else.
2026-01-08 12:05:37 +00:00
Nick Craig-Wood 38ab3dd5b1 Version v1.72.0 2025-11-21 17:10:17 +00:00
ecea0cd6f9 http: add basic metadata and provide it via serve
Co-authored-by: dougal <147946567+roucc@users.noreply.github.com>
2025-11-17 16:52:30 +00:00
Nick Craig-Wood b5e4d39b05 pacer: factor call stack searching into its own package 2025-11-12 12:22:04 +00:00
albertony 059ad47336 docs: change syntax hightlighting for command examples from sh to console 2025-11-01 15:33:38 +01:00
Lakshmi-SurekhaandGitHub ed87f82d21 build: enable support for aix/ppc64
* Adds "aix/ppc64" to the cross-compile target list.
* Including AIX in the build tag of "metadata_other.go".
* Excluding AIX from the main ncdu build tags.
* Marking AIX as an unsupported platform for ncdu.
* Excluding AIX from the fallback redirect implementation.
* Excluding AIX from unix build tags to avoid undefined unix.WNOHANG.
2025-10-27 13:34:58 +00:00
viochaandNick Craig-Wood 4f60f8915d webdav: Add Access-Control-Max-Age header for CORS preflight caching - fixes #5078 2025-10-24 10:19:22 +01:00
Oleksandr RedkoandGitHub c8a834f0e8 build: enable all govet checks (except fieldalignment and shadow) and fix issues. 2025-10-22 18:37:58 +01:00
vastonusandGitHub c4eef3065f build: remove obsolete build tag 2025-10-21 18:56:06 +01:00
albertonyanddougal 484d955ea8 lib/http: cleanup indentation and other whitespace in http serve template 2025-10-20 11:53:55 +01:00
albertonyanddougal 8fa9f255a0 docs: improve formatting of http serve template parameters 2025-10-20 11:53:55 +01:00
albertony de8e9d4693 oauthutil: improved debug logs from token refresh 2025-10-10 20:10:21 +02:00
iTroozandGitHub 4510aa679a docs: fix variants of --user-from-header 2025-10-04 08:10:49 +02:00