Before this change the environment variable getters in fs/configmap.go
logged the option value with %q, so a password set via
RCLONE_CONFIG_remote_pass (or RCLONE_remote_pass) was printed in full
to the debug log. Values from the config file were already redacted,
which made the leak easy to miss.
This change routes both getters through fs.RedactOptionValue, which
looks up the option in the backend's option list: options marked
IsPassword or Sensitive log as XXX, unknown options are conservatively
redacted, and --dump auth still shows the value for debugging.
Fixes#5794
A backend flag set on the command line to a value that happened to
equal its default was silently ignored, letting the config file win
instead.
For example --sftp-user defaults to the current user, so connecting as
that same user with --sftp-user=USER had no effect and caused rclone
to use the value from the config file.
See: https://forum.rclone.org/t/sftp-user-parsing-as-cli-argument-broken/53955
After re-organising the config it became apparent that there was a bug
in the config system which hadn't manifested until now.
This was the default config overriding the main config and was fixed
by noting when the defaults had actually changed.
Before this fix, saving a :backend config gave the error
Can't save config "token" = "XXX" for on the fly backend ":backend"
Even when using the in-memory config `--config ""`
This fixes the problem by
- always using the in memory config if it is configured
- moving the check for a :backend config save to the file config backend
It also removes the contents of the config items being saved from the
log which saves confidential tokens being logged.
Fixes#5451
Some environment variables didn’t behave like their corresponding
command line flags. The affected flags were --stats, --log-level,
--separator, --multi-tread-streams, --rc-addr, --rc-user and --rc-pass.
Example:
RCLONE_STATS='10s'
rclone check remote: remote: --progress
# Expected: rclone check remote: remote: --progress –-stats=10s
# Actual: rclone check remote: remote: --progress
Remote specific options set by environment variables was overruled by
less specific backend options set by environment variables. Example:
RCLONE_DRIVE_USE_TRASH='false'
RCLONE_CONFIG_MYDRIVE_USE_TRASH='true'
rclone deletefile myDrive:my-test-file
# Expected: my-test-file is recoverable in the trash folder
# Actual: my-test-file is permanently deleted (not recoverable)
Backend specific options set by environment variables was overruled by
general backend options set by environment variables. Example:
RCLONE_SKIP_LINKS='true'
RCLONE_LOCAL_SKIP_LINKS='false'
rclone lsd local:
# Expected result: Warnings when symlinks are skipped
# Actual result: No warnings when symlinks are skipped
# That is RCLONE_SKIP_LINKS takes precedence
The above issues have been fixed.
The debug logging (-vv) has been enhanced to show when flags are set by
environment variables.
The documentation has been enhanced with details on the precedence of
configuration options.
See pull request #5341 for more information.
Nothing is added or removed and no package is renamed by this change.
Just rearrange definitions between source files in the fs directory.
New source files:
- types.go Filesystem types and interfaces
- features.go Features and optional interfaces
- registry.go Filesystem registry and backend options
- newfs.go NewFs and its helpers
- configmap.go Getters and Setters for ConfigMap
- pacer.go Pacer with logging and calculator
The final fs.go contains what is left.
Also rename options.go to open_options.go
to dissociate from registry options.
Pastalikek65andNick Craig-Wood
Ole FrostandNick Craig-Wood
Ivan AndreevandGitHub