Before this change, if the server returned a 302 redirect message when
opening a file rclone would do the redirect but drop the
Authorization: header. This is a sensible thing to do for security
reasons but breaks some setups.
This patch adds the --webdav-auth-redirect flag which makes it
preserve the auth just for this kind of request.
See: https://forum.rclone.org/t/webdav-401-unauthorized-when-server-redirects-to-another-domain/39292
Before this change the output of `rclone size -P` would get corrupted
by the progress printing.
This is fixed by using operations.SyncPrintf instead of fmt.Printf.
Fixes#7912
In this commit
aaadb48d48 vfs: keep virtual directory status accurate and reduce deadlock potential
We reworked the virtual directory detection to use an atomic bool so
that we could run part of the cache forgetting only with a read lock.
Unfortunately this had a bug which meant that directories with virtual
items could be forgotten.
This commit changes the boolean into a count of virtual entries which
should be more accurate.
Fixes#8082
Before this change, this code implemented an ad-hoc parser for a
subset of vfs and mount options.
After the config re-organization it can use the same parsing code as
the rest of rclone which simplifies the code and exposes all the VFS
and mount options.
Before this fix the smb backend could panic if a stat call failed.
This fix makes it return an error instead.
It should have the side effect that we do one less stat call on upload
too.
Fixes#8106
In 6ba3e24853
serve nfs: fix incorrect user id and group id exported to NFS #7973
We updated the stat function to output uid and gid. However this set
the inode numbers of everything to -1. This causes a problem with
doing `ls -laR` giving "not listing already-listed directory" as it
uses inode numbers to see if it has listed a directory or not.
This patch reads the inode number from the vfs.Node and sets it in the
Stat output.
`--nfs-cache-type symlink` is similar to `--nfs-cache-type disk` in
that it uses an on disk cache, but the cache entries are held as
symlinks. Rclone will use the handle of the underlying file as the NFS
handle which improves performance.
Before this change when setting up client credentials flow manually,
rclone would fail with this error message on first run despite the
fact that no existing token is needed.
empty token found - please run "rclone config reconnect remote:"
This fixes the problem by ignoring token loading problems for client
credentials flow.
Before this change, creating a new directory would write a DEBUG log
but removing it would write an INFO log.
This change makes both write an INFO log for consistency.
An attacker can craft an input to the Parse functions that would be
processed non-linearly with respect to its length, resulting in
extremely slow parsing. This could cause a denial of service.
This only affects users running rclone servers exposed to untrusted
networks.
See: https://pkg.go.dev/vuln/GO-2024-3333
See: https://github.com/advisories/GHSA-w32m-9786-jp63
According to the SDK docs
> FileRequestIntent is required when using TokenCredential for
> authentication. Acceptable value is backup.
This sets the correct option in the SDK. It does it for all types of
authentication but the SDK seems clever enough not to supply it when
it isn't needed.
This fixes the error
> MissingRequiredHeader An HTTP header that's mandatory for this
> request is not specified. x-ms-file-request-intent
Fixes#8241
Before this change the --links flag when using the VFS override the
--links flag for the local backend which meant the local backend
needed explicit config to use links.
This fixes the problem by making the --links flag global and adding a
new --local-links flag and --vfs-links flags to control the features
individually if required.
This is somewhat limited in that it only resolves symlinks when files
are opened. This will work fine for the intended use in rclone mount,
but is inadequate for the other servers probably.
An incorrect nil check was spotted while reviewing the code for
CVE-2024-45337.
The nil check failing has never happened as far as we know. The
consequences would be a nil pointer exception.
This now compiles rclone with CGO_ENABLED=0 which is closer to the
release compile.
It also removes pikpak if testing s3 as the two depend on each
other.
CEPH uses a special bucket form `tenant:bucket` for multitentant
access using S3 as documented here:
https://docs.ceph.com/en/reef/radosgw/multitenancy/#s3
However when doing multipart uploads, in the reply from
`CreateMultipart` the `tenant:` was missing from the `Bucket` response
rclone was using to build the `UploadPart` request. This caused a 404
failure return. This may be a CEPH bug, but it is easy to work around.
This changes the code to use the `Bucket` and `Key` that we used in
`CreateMultipart` in `UploadPart` rather than the one returned from
`CreateMultipart` which fixes the problem.
See: https://forum.rclone.org/t/rclone-zcat-does-not-work-with-a-multitenant-ceph-backend/48618
Before this change attempting to download a file with
`Content-Encoding: gzip` from Cloudflare R2 gave this error
corrupted on transfer: sizes differ src 0 vs dst 999
This was caused by the SDK v2 overriding our attempt to set
`Accept-Encoding: gzip`.
This fixes the problem by disabling the middleware that does that
overriding.
Before this change, if writing to a local backend with --metadata and
--links, if the incoming metadata contained mode or ownership
information then rclone would apply the mode/ownership to the
destination of the link not the link itself.
This fixes the problem by using the link safe sycall variants
lchown/fchmodat when --links and --metadata is in use. Note that Linux
does not support setting permissions on symlinks, so rclone emits a
debug message in this case.
This also fixes setting times on symlinks on Windows which wasn't
implemented for atime, mtime and was incorrectly setting the target of
the symlink for btime.
See: https://github.com/rclone/rclone/security/advisories/GHSA-hrxh-9w67-g4cv
Before this change, if writing to a local backend with --metadata and
--links, if the incoming metadata contained mode or ownership
information then rclone would apply the mode/ownership to the
destination of the link not the link itself.
This fixes the problem by using the link safe sycall variants
lchown/fchmodat when --links and --metadata is in use. Note that Linux
does not support setting permissions on symlinks, so rclone emits a
debug message in this case.
This also fixes setting times on symlinks on Windows which wasn't
implemented for atime, mtime and was incorrectly setting the target of
the symlink for btime.
See: https://github.com/rclone/rclone/security/advisories/GHSA-hrxh-9w67-g4cv
We changed the precision of the onedrive personal backend in
c053429b9c from 1mS to 1S.
However the tests did not get updated. This changes the time tests to
use `fstest.AssertTimeEqualWithPrecision` which compares with
precision so hopefully won't break again.
Before this change, upgrading to v1.13.7 caused a deadlock in the tests.
This was caused by additional locking in the sftp package exposing a
bad choice by the rclone code.
See https://github.com/pkg/sftp/issues/603 and thanks to @puellanivis
for the fix suggestion.
The Mailru backend integration tests have been failing due to new rate
limits on the backend.
This patch
- Removes Mailru from the chunker tests
- Adds the flag so we only run one Mailru test at once
Before this testing any backend which implemented the OpenChunkWriter
gave this error:
ERROR : writer-at-subdir/writer-at-file: Don't know how to set key "chunkSize" on upload
This was due to the ChunkOption incorrectly rendering into HTTP
headers which weren't understood by the backend.
Nick Craig-Wood