The dataflow analysis behind SA4023, new in the staticcheck 0.8.0
bundled with golangci-lint v2.13.0, makes linting large packages more
than 10x slower (89s vs 7s for backend/s3 alone) which took the CI
lint job past its 30 minute limit. golangci-lint no longer enforces
its run timeout during analysis so the job ran until cancelled, and
the cancellation meant the lint cache was never saved, making every
subsequent run cold and guaranteeing the timeout repeated.
The check also produces false positives (eg claiming operations.Delete
never returns nil).
JSON logs reported "source":"slog/logger.go:256" instead of the real
caller. getCaller skips logging-machinery frames by file path, but
release builds use -trimpath which rewrites the standard library slog
frame's path to "log/slog/logger.go" - matching neither the "/log/" nor
the "log.go" check, so it was reported as the source. Also skip frames
whose function belongs to the log/slog package, which is immune to
-trimpath.
Avoid side effects by using own logger instance
- Importing fs/log only sets rclone's private logger via fs.SetLogger,
so internal rclone logging works from the moment the package is
imported but the process-wide slog default is left untouched.
- slog.SetDefault and slog.SetLogLoggerLevel move into InitLogging,
which is called explicitly from the CLI (cmd/cmd.go), the librclone
wrapper and the integration test framework. So rclone-as-a-program
keeps capturing log.Print/log.Fatal and slog.Default() output as
before.
Library consumers that import fs/log without calling InitLogging now
keep their own slog default and can safely route rclone output back
into it via log.Handler.SetOutput without recursing.
Fixes#8907
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
The format field was read in Handle() without synchronization while
setFormat() could write it concurrently from InitLogging(). This
caused a data race detected by the race detector, failing
TestListBucketsAuthProxy in cmd/serve/s3.
Fix by protecting all access to format with the existing mutex.
Before the log re-organisation in:
8d353039a6 log: add log rotation to --log-file
rclone would write any backtraces to the --log-file which was very
convenient for users.
This got accidentally disabled due to a typo which meant backtraces
started going to stderr even if --log-file was supplied.
This fixes the problem.
* Adds "aix/ppc64" to the cross-compile target list.
* Including AIX in the build tag of "metadata_other.go".
* Excluding AIX from the main ncdu build tags.
* Marking AIX as an unsupported platform for ncdu.
* Excluding AIX from the fallback redirect implementation.
* Excluding AIX from unix build tags to avoid undefined unix.WNOHANG.
We were seeing a (non-fatal) error in our logs:
```
Failed to seek log file to end: seek /proc/1/fd/1: illegal seek
```
Because we open the log file with O_APPEND,
we don't need to manually seek to the end.
As https://pkg.go.dev/os#File.Seek also confirms
that the behavior of `Seek` is not specified
if the file has been opened with O_APPEND,
remove the `Seek` call.
In this commit the logging system was re-worked
dfa4d94827 fs: Remove github.com/sirupsen/logrus and replace with log/slog
Unfortunately the systemd logging was still using the plain log
package and this caused a deadlock as it was recursively calling the
logging package.
The fix was to use the dedicated systemd journal logging routines in
the process removing a TODO!
This removes logrus which is not developed any more and replaces it
with the new log/slog from the Go standard library.
It implements its own slog Handler which is backwards compatible with
all of rclone's previous logging modes.
This commit modernizes Go usage. This was done with:
go run golang.org/x/tools/gopls/internal/analysis/modernize/cmd/modernize@latest -fix -test ./...
Then files needed to be `go fmt`ed and a few comments needed to be
restored.
The modernizations include replacing
- if/else conditional assignment by a call to the built-in min or max functions added in go1.21
- sort.Slice(x, func(i, j int) bool) { return s[i] < s[j] } by a call to slices.Sort(s), added in go1.21
- interface{} by the 'any' type added in go1.18
- append([]T(nil), s...) by slices.Clone(s) or slices.Concat(s), added in go1.21
- loop around an m[k]=v map update by a call to one of the Collect, Copy, Clone, or Insert functions from the maps package, added in go1.21
- []byte(fmt.Sprintf...) by fmt.Appendf(nil, ...), added in go1.19
- append(s[:i], s[i+1]...) by slices.Delete(s, i, i+1), added in go1.21
- a 3-clause for i := 0; i < n; i++ {} loop by for i := range n {}, added in go1.22
This changes log statements from log to fs package, which is required for --use-json-log
to properly make log output in JSON format. The recently added custom linting rule,
handled by ruleguard via gocritic via golangci-lint, warns about these and suggests
the alternative. Fixing was therefore basically running "golangci-lint run --fix",
although some manual fixup of mainly imports are necessary following that.
This adds an additional parameter to the creation of each flag. This
specifies one or more flag groups. This **must** be set for global
flags and **must not** be set for local flags.
This causes flags.md to be built with sections to aid comprehension
and it causes the documentation pages for each command (and the
`--help`) to be built showing the flags groups as specified in the
`groups` annotation on the command.
See: https://forum.rclone.org/t/make-docs-for-mortals-not-only-rclone-gurus/39476/
This is done by making fs.Config private and attaching it to the
context instead.
The Config should be obtained with fs.GetConfig and fs.AddConfig
should be used to get a new mutable config that can be changed.
The purpose of this is to make it easier to maintain and eventually to
allow the rclone backends to be re-used in other projects without
having to use the rclone configuration system.
The new code layout is documented in CONTRIBUTING.
Nick Craig-Wood
yyandGitHub
dougalandGitHub
Tingsong Xuanddougal
Lakshmi-SurekhaandGitHub
Aneesh AgrawalandNick Craig-Wood
nielash
albertonyandNick Craig-Wood
Anagh Kumar BaranwalandNick Craig-Wood
Ivan Andreev
Benjamin GustinandGitHub
Josh SorefandNick Craig-Wood
HekmonandNick Craig-Wood
Martin MichlmayrandNick Craig-Wood
Patryk JakuszewandNick Craig-Wood
justinalinandNick Craig-Wood
dcpuandNick Craig-Wood
Filip BartodziejandNick Craig-Wood