f-denkena/rclone
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
10,326 Commits 1 Branch 0 Tags
13084df67c7ef96eba20eefb628af39b170fdcd0
Commit Graph
24 Commits
Author SHA1 Message Date
Nick Craig-Wood e8e883c35e serve s3: fix auth proxy accepting any request signed with an empty secret GHSA-xwwr-4h3p-r22c CVE-PENDING 
With --auth-proxy set and --auth-key unset, serve s3 registered every client
supplied access key ID with an empty secret and verified the SigV4 signature
against that, so anyone could sign a request for an arbitrary access key ID with
an empty secret and be let in. The proxy program was only ever given the access
key ID (as both user and pass) so it had nothing with which to authenticate the
client either.

An S3 client never sends its secret, only a signature made with it, so the
server has to know the secret to check the request. The auth proxy protocol as
been changed to handle this. For serve s3 the proxy program is given just the
access key ID as the user (no pass or public_key) and must return the matching
secret as _secret_access_key in its output. rclone verifies the request's
signature against that secret, refusing the request if the proxy rejects the
access key ID, doesn't return a secret or returns an empty one, or the signature
doesn't match. The secret is only used for this server's own verification and is
never registered with gofakes3, so other serve s3 instances in the same process
don't honour it.

The proxy's answers are cached. If a signature fails against a cached secret the
proxy is consulted again so a rotated secret takes effect immediately - but only
for a signature mismatch, and at most once every 10 seconds per access key ID
and client IP, so a stream of bad signatures can't make the proxy program run
for every request. A rotation never shuts down the cached backend under requests
still using it. A cached answer is checked with the proxy again once it is 5
minutes old even if in constant use, so revoking an access key ID takes effect
within 5 minutes.

This means --auth-key is no longer needed with --auth-proxy: it is ignored and a
warning is given at startup if both are set. The proxy is the source of truth
for both the credentials and the backend they map to. Presigned URLs (credential
in the query string) are now recognised by the proxy middleware too. The auth
proxy docs are added to serve s3.

Note that the serve s3 auth proxy protocol has changed. The proxy program is now
given the access key ID as "user" (it was previously given an MD5 hash of it,
with the access key ID as "pass") and must return the matching secret as
"_secret_access_key".

This needs gofakes3 v0.0.9 for signature.V4SignVerifyWithSecret.
 2026-09-04 19:00:22 +01:00
Hakan İSMAİLandNick Craig-Wood f425f8d466 serve: refactor VFS and proxy handling into Provider 2026-08-18 09:03:12 +01:00
am-at-enrollvbandGitHub 5dd34275dc serve: pass the client IP address to the auth proxy - fixes #4499 
The auth proxy was only given the user and their password or public
key, so a proxy program had no way to restrict logins to particular
networks, or to record where an authentication attempt came from.

The JSON sent to the program now has a client_ip key holding the bare
IP the client connected from, with the port stripped so IPv6 arrives
as 2001:db8::1 rather than [2001:db8::1]:52344. An IPv4-mapped IPv6
address is reported as plain IPv4 so that a client arriving over a
dual-stack listener still matches IPv4 networks. The key is omitted
when the client has no IP address.

The IP is also mixed into the backend cache key. That is needed as the
program is only run on a cache miss, so a client from a
non-allowlisted address presenting valid credentials within the 5
minute cache lifetime would get a cache hit and be let in without the
program being consulted at all.
 2026-08-01 12:25:06 +01:00
Nick Craig-Wood df9935d71e serve: fix auth proxy using stale config parameters when making a backend 
Before this change, if the user changed their password or public-key
and the auth proxy script returned updated config parameters for the
backend (eg a rotated api_key) rclone would continue to re-use the old
backend with the old config parameters out of the fscache.

This was because both the VFS cache and the fs/cache key were derived
from the user name only, so a change in the user's password or
public-key did not invalidate the cached backend.

Fix this by deriving the cache key from the user plus a hash of the
password/public-key, so a credential change forces a fresh backend.
The hash uses a per-process random HMAC key so the fragment that
appears in logs cannot be brute-forced offline.
 2026-06-08 16:10:20 +01:00
Nick Craig-Wood 7b8994ab32 vfs: add context parameter to New() for config propagation 
Add a ctx parameter to vfs.New() so callers can pass in context
carrying ConfigInfo and FilterInfo. The context is stripped of
cancellation but config and filter values are preserved into a fresh
background context.
 2026-04-13 12:48:38 +01:00
Nick Craig-Wood 71b9b4ad7a build: use sequence Split introduced in go1.24 2025-09-15 12:45:57 +01:00
albertony 2e02d49578 docs: fix markdown lint issues in command docs 2025-08-26 12:04:00 +02:00
Nick Craig-Wood 6054c4e49d auth proxy: convert options to new style 2025-04-09 11:12:07 +01:00
Nick Craig-Wood 028316ba5d auth proxy: add VFS options parameter for use for default VFS 
This is for use from the RC API.
 2025-04-09 11:12:07 +01:00
Nick Craig-Wood 401cf81034 build: modernize Go usage 
This commit modernizes Go usage. This was done with:

go run golang.org/x/tools/gopls/internal/analysis/modernize/cmd/modernize@latest -fix -test ./...

Then files needed to be `go fmt`ed and a few comments needed to be
restored.

The modernizations include replacing

- if/else conditional assignment by a call to the built-in min or max functions added in go1.21
- sort.Slice(x, func(i, j int) bool) { return s[i] < s[j] } by a call to slices.Sort(s), added in go1.21
- interface{} by the 'any' type added in go1.18
- append([]T(nil), s...) by slices.Clone(s) or slices.Concat(s), added in go1.21
- loop around an m[k]=v map update by a call to one of the Collect, Copy, Clone, or Insert functions from the maps package, added in go1.21
- []byte(fmt.Sprintf...) by fmt.Appendf(nil, ...), added in go1.19
- append(s[:i], s[i+1]...) by slices.Delete(s, i, i+1), added in go1.21
- a 3-clause for i := 0; i < n; i++ {} loop by for i := range n {}, added in go1.22
 2025-02-28 11:31:14 +00:00
albertonyandNick Craig-Wood 33bff6fe71 build: fix gocritic lint issue wrapperfunc 2024-08-15 22:08:34 +01:00
Nick Craig-Wood a28287e96d vfs: convert vfs options to new style 
This also
- move in use options (Opt) from vfsflags to vfscommon
- change os.FileMode to vfscommon.FileMode in parameters
- rework vfscommon.FileMode and add tests
 2024-07-15 11:09:54 +01:00
albertony e739ee2c27 docs: ensure empty line between text and a following heading 2024-04-05 21:39:44 +02:00
Vincent MurphyandNick Craig-Wood 41b8935a6c docs: Fix broken test_proxy.py link again 
The previous fix fixed the auto generated output - this fixes the source.
 2024-01-08 11:54:02 +00:00
albertony ec117593f1 Fix lint issues reported by staticcheck 
Used staticcheck 2022.1.2 (v0.3.2)

See: staticcheck.io
 2022-06-13 21:13:50 +02:00
Nick Craig-Wood e43b5ce5e5 Remove github.com/pkg/errors and replace with std library version 
This is possible now that we no longer support go1.12 and brings
rclone into line with standard practices in the Go world.

This also removes errors.New and errors.Errorf from lib/errors and
prefers the stdlib errors package over lib/errors.
 2021-11-07 11:53:30 +00:00
Nick Craig-Wood d846210978 fs: Add context to NewFs #3257 #4685 
This adds a context.Context parameter to NewFs and related calls.

This is necessary as part of reading config from the context -
backends need to be able to read the global config.
 2020-11-09 18:05:54 +00:00
Josh SorefandNick Craig-Wood d0888edc0a Spelling fixes 
Fix spelling of: above, already, anonymous, associated,
authentication, bandwidth, because, between, blocks, calculate,
candidates, cautious, changelog, cleaner, clipboard, command,
completely, concurrently, considered, constructs, corrupt, current,
daemon, dependencies, deprecated, directory, dispatcher, download,
eligible, ellipsis, encrypter, endpoint, entrieslist, essentially,
existing writers, existing, expires, filesystem, flushing, frequently,
hierarchy, however, implementation, implements, inaccurate,
individually, insensitive, longer, maximum, metadata, modified,
multipart, namedirfirst, nextcloud, obscured, opened, optional,
owncloud, pacific, passphrase, password, permanently, persimmon,
positive, potato, protocol, quota, receiving, recommends, referring,
requires, revisited, satisfied, satisfies, satisfy, semver,
serialized, session, storage, strategies, stringlist, successful,
supported, surprise, temporarily, temporary, transactions, unneeded,
update, uploads, wrapped

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-14 15:21:31 +01:00
Martin MichlmayrandNick Craig-Wood 4aee962233 doc: fix typos throughout docs and code 2020-05-20 15:54:51 +01:00
Nick Craig-Wood de93852512 docs: restore lost auth proxy logs 
These came from f2a789ea98 which unfortunately added the docs to
the auto generated files.
 2020-02-10 15:29:39 +00:00
Nick Craig-Wood b88dec51e5 proxy: replace use of bcrypt with sha256 
Unfortunately bcrypt only hashes the first 72 bytes of a given input
which meant that using it on ssh keys which are longer than 72 bytes
was incorrect.

This swaps over to using sha256 which should be adequate for the
purpose of protecting in memory passwords where the unencrypted
password is likely in memory too.
 2020-01-15 16:23:57 +00:00
Paul TinsleyandNick Craig-Wood f2a789ea98 serve sftp: Add support for public key with auth proxy - fixes #3572 2020-01-15 16:23:57 +00:00
Nick Craig-Wood 9115752679 proxy: reduce the internal bcrypt strength to fix race tests 
Before this change the race tests were taking too long.  The bcrypt
function went from about 20ms to 1s under the race detector and this
is called for every transaction on webdav.

This change reduces the bcrypt strength so it takes 1ms non race so
the race tests pass and still has adequate security for in memory only
storage.
 2019-11-11 16:20:28 +00:00
Nick Craig-Wood e6ab237fcd serve: add auth proxy infrastructure 2019-08-06 11:43:42 +01:00