f-denkena/rclone
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5,396 Commits 1 Branch 0 Tags
60dcafe04dbb86267a48b8f18a9a2d63a5ae67dc
Commit Graph
111 Commits
Author SHA1 Message Date
Nick Craig-Wood 9393225a1d link: use "off" value for unset expiry 2021-04-07 13:58:37 +01:00
Nick Gaya 1fd9b483c8 onedrive: add list_chunk option 
Add --onedrive-list-chunk option similar to existing options for azureblob, drive, and s3.

Suggested as a workaround for a OneDrive pagination bug

See: https://forum.rclone.org/t/unexpected-duplicates-on-onedrive-with-0s-in-filename/23164/8
 2021-04-04 09:08:16 +01:00
Nick Craig-Wood f2c0f82fc6 backends: Add context checking to remaining backends #4504 
This is a follow up to 4013bc4a4c which missed some backends.

It adds a ctx parameter to shouldRetry and checks it.
 2021-03-16 16:17:22 +00:00
Nick Craig-Wood 46078d391f config: make config file reads reload the config file if needed #4996 
Before this change the config file needed to be explicitly reloaded.
This coupled the config file implementation with the backends
needlessly.

This change stats the config file to see if it needs to be reloaded on
every config file operation.

This allows us to remove calls to

- config.SaveConfig
- config.GetFresh

Which now makes the the only needed interface to the config file be
that provided by configmap.Map when rclone is not being configured.

This also adds tests for configfile
 2021-03-14 16:03:35 +00:00
Nick Craig-Wood 0ad38dd6fa dropbox,ftp,onedrive,yandex: make --timeout 0 work properly 
See: https://forum.rclone.org/t/an-issue-about-ftp-backend-in-2-different-systems/22551
 2021-03-01 12:08:58 +00:00
NyaMisty 15da53696e onedrive: add support for china region operated by 21vianet #4963 (#4963) 
fixes #3804
obsoletes #3973
obsoletes #4072
 2021-01-29 23:04:21 +03:00
kice ef2bfb9718 onedrive: Support addressing site by server-relative URL (#4761) 2021-01-09 03:26:42 +08:00
Alex Chen 78a76b0d29 onedrive: remove % and # from the set of encoded characters (#4909) 
onedrive: remove % and # from the set of encoded characters

This fixes #4700, fixes #4184, fixes #2920.
 2021-01-08 12:07:17 +00:00
Nick Craig-Wood d58fdb10db onedrive: enhance link creation with expiry, scope, type and password 
This change makes the --expire flag in `rclone link` work.

It also adds the new flags

    --onedrive-link-type
    --onedrive-link-scope
    --onedrive-link-password

See: https://forum.rclone.org/t/create-share-link-within-the-organization-only/21498
 2021-01-08 09:22:50 +00:00
Alex Chen c66b901320 onedrive: (business only) workaround to replace existing file on server-side copy (#4904) 2021-01-06 10:50:37 +08:00
Cnly 00bf40a8ef onedrive: fix server-side copy completely disabled on OneDrive for Business 
This fixes a little problem in PR #4903, which is a fix for #4342
 2021-01-06 02:57:51 +08:00
Alex Chen b594cb9430 onedrive: fall back to normal copy if server-side copy unavailable (#4903) 
Fixes #4342 by:
* Disabling server-side copy if either drive isn't OneDrive Personal
* Falling back to normal copy if server-side copy fails
 2021-01-05 21:26:00 +08:00
Nick Craig-Wood 2e21c58e6a fs: deglobalise the config #4685 
This is done by making fs.Config private and attaching it to the
context instead.

The Config should be obtained with fs.GetConfig and fs.AddConfig
should be used to get a new mutable config that can be changed.
 2020-11-26 16:40:12 +00:00
Nick Craig-Wood f7efce594b config: add context.Context #3257 #4685 
This add config to the Config callback in the backends and the related
config functions.
 2020-11-09 18:05:54 +00:00
Nick Craig-Wood 8b96933e58 fs: Add context to fs.Features.Fill & fs.Features.Mask #3257 #4685 2020-11-09 18:05:54 +00:00
Nick Craig-Wood d846210978 fs: Add context to NewFs #3257 #4685 
This adds a context.Context parameter to NewFs and related calls.

This is necessary as part of reading config from the context -
backends need to be able to read the global config.
 2020-11-09 18:05:54 +00:00
Nick Craig-Wood bedf6e90d2 onedrive: warn on gateway timeout errors 
It seems that when doing chunked uploads to onedrive, if the chunks
take more than 3 minutes or so to upload then they may timeout with
error 504 Gateway Timeout.

This change produces an error (just once) suggesting lowering
`--onedrive-chunk-size` or decreasing `--transfers`.

This is easy to replicate with:

    rclone copy -Pvv --bwlimit 0.05M 20M onedrive:20M

See: https://forum.rclone.org/t/default-onedrive-chunk-size-does-not-work/20010/
 2020-11-02 16:53:35 +00:00
Josh Soref e4a87f772f docs: spelling: e.g. 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Josh Soref bbe7eb35f1 docs: spelling: server-side 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Josh Soref d0888edc0a Spelling fixes 
Fix spelling of: above, already, anonymous, associated,
authentication, bandwidth, because, between, blocks, calculate,
candidates, cautious, changelog, cleaner, clipboard, command,
completely, concurrently, considered, constructs, corrupt, current,
daemon, dependencies, deprecated, directory, dispatcher, download,
eligible, ellipsis, encrypter, endpoint, entrieslist, essentially,
existing writers, existing, expires, filesystem, flushing, frequently,
hierarchy, however, implementation, implements, inaccurate,
individually, insensitive, longer, maximum, metadata, modified,
multipart, namedirfirst, nextcloud, obscured, opened, optional,
owncloud, pacific, passphrase, password, permanently, persimmon,
positive, potato, protocol, quota, receiving, recommends, referring,
requires, revisited, satisfied, satisfies, satisfy, semver,
serialized, session, storage, strategies, stringlist, successful,
supported, surprise, temporarily, temporary, transactions, unneeded,
update, uploads, wrapped

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-14 15:21:31 +01:00
Nick Craig-Wood 0906f8dd3b onedrive: fix disk usage for sharepoint 
Some onedrive sharepoints appear to return all 0s for quota

    "quota":{"deleted":0,"remaining":0,"total":0,"used":0}

This commit detects this and returns unknown for all quota parts.

See: https://forum.rclone.org/t/zero-size-volume-when-mounting-onedrive-sharepoint/19597
 2020-10-09 14:11:56 +01:00
Tim Gallant 30eb094f28 oauthutil: adds SharedOptions for OAuth backends 
1. adds SharedOptions data structure to oauthutil
2. adds config.ConfigToken option to oauthutil.SharedOptions
3. updates the backends that have oauth functionality

Fixes #2849
 2020-08-07 16:32:01 +01:00
Nick Craig-Wood b401a727f7 onedrive: add --onedrive-no-versions flag to remove old versions - fixes #4106 2020-08-07 15:58:30 +01:00
Nick Craig-Wood 8eb16ce89c onedrive: implement rclone cleanup #4106 2020-08-07 15:58:30 +01:00
Nick Craig-Wood a2afa9aadd fs: Add directory to optional Purge interface - fixes #1891 
- add a directory to the optional Purge interface
- fix up all the backends
- add an additional integration test to test for the feature
- use the new feature in operations.Purge

Many of the backends had been prepared in advance for this so the
change was trivial for them.
 2020-07-31 17:43:17 +01:00
Nick Craig-Wood 50e36fb482 onedrive: Fix reverting to Copy when Move would have worked 
For some objects the onedrive backend has been doing a server side
copy and a delete when a server side move would have worked OK.

This was caused by not detecting the home drive correctly (when it was
an empty string) and assuming that these transfers were cross drive.

This is fixed by comparing canonicalizing drive IDs before comparing them.
 2020-07-02 10:55:36 +01:00
Nick Craig-Wood 2c10ce64aa onedrive: rework cancel of multipart uploads on rclone exit #4300 
This now uses the atexit.OnError framework rather than a home grown one.
 2020-06-25 15:22:53 +01:00
Nick Craig-Wood 2121c0fa23 dircache: factor DirMove code out of backends into dircache 
Before this change there was lots of duplicated code in all the
dircache using backends to support DirMove.

This change factors this code into the dircache library.
 2020-06-25 09:41:36 +01:00
Nick Craig-Wood a8652e2252 dircache: simplify interface, fix corner cases and apply to backends 
Dircache was changed to:

- Remove special cases for the root directory
- Remove Fatal errors
- Call FindRoot on behalf of the user wherever possible
- Bring up to modern Go standards

Backends were changed to:

- Remove calls to FindRoot
- Change calls to FindRootAndPath to FindPath
- Don't make special cases for the root

This fixes several corner cases, for example removing a non existent
directory if FindRoot hasn't been called.
 2020-06-25 09:41:36 +01:00
Roman Kredentser 55ad1354b6 link: Add --expire and --unlink flags 
This adds expire and unlink fields to the PublicLink interface.

This fixes up the affected backends and removes unlink parameters
where they are present.
 2020-06-05 14:51:05 +01:00
Nick Craig-Wood 49ba4eeb86 oauthutil: tidy interface to Config to add Options struct 
The interface was getting so that a new function was needed for every
Config variant. Adding an Options struct fixes this.
 2020-05-26 11:27:01 +01:00
Martin Michlmayr 4aee962233 doc: fix typos throughout docs and code 2020-05-20 15:54:51 +01:00
Tim Gallant c390fc8100 onedrive: pass options to rest.Opts for Put and Update 2020-04-23 11:07:21 +01:00
harry 9aa32bc269 onedrive: make error quotaLimitReached to be fatal - Fixes #4089 2020-03-26 16:19:50 +00:00
Nick Craig-Wood d9c8c47e02 onedrive: add missing drive on config - fixes #4068 
Before this change we queries /me/drives for a list of the users
drives and asked the user to choose. Sometimes this does not return
the users main drive for reasons unknown.

After this change we query /me/drives first then /me/drive and add
that to the list of drives if it wasn't already there.
 2020-03-24 08:44:10 +00:00
Nick Craig-Wood 77e94be280 onedrive: implement --onedrive-server-side-across-configs - fixes #4058 2020-03-15 21:10:23 +00:00
Harry fdb07f2f89 onedrive: Added maximum chunk size limit warning in the docs 
If chunk size is more than 250M (262,144,000 bytes) then API throws the following error:

Microsoft.SharePoint.Client.InvalidClientQueryException: The request message is too big. The server does not allow messages larger than 262144000 bytes.
 2020-03-10 15:14:08 +00:00
Nick Craig-Wood 3dfa63b85c onedrive: fix occasional 416 errors on multipart uploads 
Before this change, when uploading multipart files, onedrive would
sometimes return an unexpected 416 error and rclone would abort the
transfer.

This is usually after a 500 error which caused rclone to do a retry.

This change checks the upload position on a 416 error and works how
much of the current chunk to skip, then retries (or skips) the current
chunk as appropriate.

If the position is before the current chunk or after the current chunk
then rclone will abort the transfer.

See: https://forum.rclone.org/t/fragment-overlap-error-with-encrypted-onedrive/14001

Fixes #3131
 2020-02-01 21:15:07 +00:00
Motonori IWAMURO 7662f15939 onedrive: add support "Retry-After" header 2020-01-29 12:16:18 +00:00
Benjamin Richter 77fa8194f2 onedrive: add Sites.Read.All permission - Fixes #1770 2020-01-20 12:30:19 +00:00
Nick Craig-Wood bafe7d5a73 backends: move encoding definitions from fs/encodings 2020-01-16 14:40:36 +00:00
Nick Craig-Wood 3c620d521d backend: adjust backends to have encoding parameter 
Fixes #3761
Fixes #3836
Fixes #3841
 2020-01-16 14:40:36 +00:00
Xiaoxing Ye 191cfb79d1 onedrive: no trailing slash reading metadata... 
No trailing slash when reading metadata of an item given item ID.

This should fix #3664.
 2019-10-29 13:33:11 +00:00
Alex Chen 41ed33b08e docs: update onedrive/sharepoint docs on some known issues 2019-10-12 12:08:22 +01:00
Fionera 1dc8bcd48c Remove backend dependency from fs/hash 2019-10-01 16:29:58 +01:00
Fabian Möller dac20093c5 onedrive: use lib/encoder 2019-09-30 14:05:49 +01:00
Nick Craig-Wood 58a531a203 rest: add context propagation to rest library #3257 
This fixes up the calling and propagates the contexts for the backends
which use lib/rest.
 2019-09-09 23:27:07 +01:00
Nick Craig-Wood 0edbc9578d googlephotos,onedrive: fix crash on error response - fixes #3491 
This fixes a crash on the google photos backend when an error is
returned from the rest.Call function.

This turned out to be a mis-understanding of the rest docs so
- improved rest.Call docs
- fixed mis-understanding in google photos backend
- fixed similar mis-understading in onedrive backend
 2019-08-28 12:11:03 +01:00
Nick Craig-Wood 57d5de6fba build: fix up package paths after repo move 
git grep -l github.com/ncw/rclone | xargs -d'\n' perl -i~ -lpe 's|github.com/ncw/rclone|github.com/rclone/rclone|g'
goimports -w `find . -name \*.go`
 2019-07-28 18:47:38 +01:00
Aleksandar Jankovic f78cd1e043 Add context propagation to rclone 
- Change rclone/fs interfaces to accept context.Context
- Update interface implementations to use context.Context
- Change top level usage to propagate context to lover level functions

Context propagation is needed for stopping transfers and passing other
request-scoped values.
 2019-06-19 11:59:46 +01:00