The file is created at the declared size and the data then written
with ranged writes, so if the source supplied fewer bytes than
declared, the remainder of the file was left as zeroes and the upload
reported success.
Count the bytes actually read from the source and if they do not match
the declared size, delete the partially uploaded file (if newly
created) and return an error.
This was found by the new FsPutShortEOF and TestRcatSizeShortEOF
integration tests.
The server stores the SMB last write time with 100 ns (FILETIME)
precision and returns it in full in listings, so advertise that as
the precision instead of one second. This makes syncs preserve
sub-second modtimes when comparing and copying files.
Note that this means rclone will consider files with modtimes
differing by less than a second as needing their modtime set where
before it did not.
Update and SetModTime kept the caller's full nanosecond precision
modtime in memory while the server stores SMB LastWriteTime with
100 ns (FILETIME) precision, so until the object was re-read the
modtime did not match what a fresh listing would report. This broke
wrappers which compare exact modtimes, such as the hasher backend's
fingerprint and the VFS cache. Truncate the modtime to 100 ns
precision to match what the server stores.
The fix for #8241 set FileRequestIntent=Backup on
service.ClientOptions so the azfile SDK emits the
x-ms-file-request-intent header was inadvertently dropped in this
commit (released in v1.73.0)
846f193806 azureblob,azurefiles: factor the common auth into a library
This broke azurefiles with OAuth (service principal secret,
certificate, MSI, etc.) with:
400 MissingRequiredHeader: x-ms-file-request-intent
This restores it in the azurefiles SetClientOptions callback. The SDK
only emits the header for TokenCredential auth, so shared-key and SAS
paths are unaffected.
Before this change, setting an object's modtime with o.SetModTime() (without
updating the file's content) would inadvertently erase its md5 hash.
The documentation notes: "If this property isn't specified on the request, the
property is cleared for the file. Subsequent calls to Get File Properties won't
return this property, unless it's explicitly set on the file again."
https://learn.microsoft.com/en-us/rest/api/storageservices/set-file-properties#common-request-headers
This change fixes the issue by setting ContentMD5 (and ContentType), to the
extent we have it, during SetModTime.
Discovered on bisync integration tests such as TestBisyncRemoteRemote/resolve
According to the SDK docs
> FileRequestIntent is required when using TokenCredential for
> authentication. Acceptable value is backup.
This sets the correct option in the SDK. It does it for all types of
authentication but the SDK seems clever enough not to supply it when
it isn't needed.
This fixes the error
> MissingRequiredHeader An HTTP header that's mandatory for this
> request is not specified. x-ms-file-request-intent
Fixes#8241
- use rclone's http Transport
- fix handling of 0 length files
- combine into one file and remove uneeded abstraction
- make `chunk_size` and `upload_concurrency` settable
- make auth the same as azureblob
- set the Features correctly
- implement `--azurefiles-max-stream-size`
- remove arbitrary sleep on Mkdir
- implement `--header-upload`
- implement read and write MimeType for objects
- implement optional methods
- About
- Copy
- DirMove
- Move
- OpenWriterAt
- PutStream
- finish documentation
- disable build on plan9 and js
Fixes#365Fixes#7378
Nick Craig-Wood
Vikas BhansaliandGitHub
nielash
jinjingroadandalbertony
b-wimmerandGitHub
Dimitri Papadopoulosandalbertony
OksanaandGitHub
