f-denkena/rclone
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
10,343 Commits 1 Branch 0 Tags
b957d264dfee4ad192de60e2307718af0383a4c0
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Fabian MöllerandNick Craig-Wood b9bd15a8c9 koofr: use lib/encoder 
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
 2019-09-30 22:00:25 +01:00
Fabian MöllerandNick Craig-Wood 3304bb7a56 googlecloudstorage: use lib/encoder 
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
 2019-09-30 22:00:24 +01:00
Fabian MöllerandNick Craig-Wood 7689bd7e21 amazonclouddrive: use lib/encoder 
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
 2019-09-30 22:00:24 +01:00
Fabian MöllerandNick Craig-Wood 33f129fbbc s3: use lib/encoder 
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
 2019-09-30 22:00:24 +01:00
Fabian MöllerandNick Craig-Wood 00b2c02bf4 pcloud: use lib/encoder 
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
 2019-09-30 22:00:24 +01:00
Fabian MöllerandNick Craig-Wood 33aea5d43f mega: use lib/encoder 
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
 2019-09-30 22:00:24 +01:00
Fabian MöllerandNick Craig-Wood 13d8b7979d b2: use lib/encoder 
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
 2019-09-30 22:00:24 +01:00
Fabian MöllerandNick Craig-Wood f0c2249086 encoder: add edge control characters and fix edge test generation 2019-09-30 14:05:49 +01:00
Fabian MöllerandNick Craig-Wood 6ba08b8612 info: rewrite invalid character test and reporting 2019-09-30 14:05:49 +01:00
Fabian MöllerandNick Craig-Wood c8d3e57418 encodings: add . and .. to all backends, except Drive 2019-09-30 14:05:49 +01:00
Fabian MöllerandNick Craig-Wood d5cd026547 encoder: add option to encode . and .. names 2019-09-30 14:05:49 +01:00
Fabian MöllerandNick Craig-Wood 6c0a749a42 crypt: remove checkValidString 
Remove the usage of checkValidString in decryptSegment to allow all
paths which can be created by encryptSegment to be decryptable.
 2019-09-30 14:05:49 +01:00
Fabian MöllerandNick Craig-Wood 4b9fdb8475 opendrive: use lib/encoder 2019-09-30 14:05:49 +01:00
Fabian MöllerandNick Craig-Wood dac20093c5 onedrive: use lib/encoder 2019-09-30 14:05:49 +01:00
Fabian MöllerandNick Craig-Wood d211347d46 dropbox: use lib/encoder 2019-09-30 14:05:49 +01:00
Fabian MöllerandNick Craig-Wood 4837bc3546 jottacloud: use lib/encoder 2019-09-30 14:05:49 +01:00
Fabian MöllerandNick Craig-Wood 69c51325bb drive: use lib/encoder 2019-09-30 14:05:49 +01:00
Fabian MöllerandNick Craig-Wood 05e4f10436 box: use lib/encoder 2019-09-30 14:05:49 +01:00
Fabian MöllerandNick Craig-Wood a98a750fc9 local: use lib/encoder 2019-09-30 14:05:49 +01:00
Fabian MöllerandNick Craig-Wood c09b62a088 encodings: add all known backend encodings 2019-09-30 14:05:49 +01:00
Fabian MöllerandNick Craig-Wood a56c9ab61d docs: add section for restricted filenames 2019-09-30 14:05:48 +01:00
Fabian MöllerandNick Craig-Wood 97a218903c fstest: remove WinPath from fstest.Item 2019-09-30 14:05:48 +01:00
Fabian MöllerandNick Craig-Wood d04b0b856a fserrors: use errors.Walk for the wrapped error types 2019-05-01 16:56:08 +01:00
Fabian Möller 48c09608ea fix spelling 2019-04-30 14:12:18 +02:00
Fabian Möller 8f4ea77c07 fs: remove unnecessary pacer warning 2019-02-18 08:42:36 +01:00
Fabian MöllerandNick Craig-Wood 61616ba864 pacer: make pacer more flexible 
Make the pacer package more flexible by extracting the pace calculation
functions into a separate interface. This also allows to move features
that require the fs package like logging and custom errors into the fs
package.

Also add a RetryAfterError sentinel error that can be used to signal a
desired retry time to the Calculator.
 2019-02-16 14:38:07 +00:00
Fabian MöllerandNick Craig-Wood 9ed721a3f6 errors: add lib/errors package 2019-02-16 14:38:07 +00:00
Fabian MöllerandNick Craig-Wood ef5e1909a0 encoder: add lib/encoder to handle character subsitution and quoting 2019-02-09 18:23:47 +00:00
Fabian MöllerandNick Craig-Wood bca5d8009e onedrive: return errors instead of panic for invalid uploads 2019-02-09 18:23:47 +00:00
Fabian MöllerandNick Craig-Wood 334f19c974 info: improve allowed character testing 2019-02-09 18:23:47 +00:00
Fabian MöllerandNick Craig-Wood 42a5bf1d9f golangci: enable lints excluded by default 2019-02-09 18:18:22 +00:00
Fabian Möller a0d4c04687 backend: fix misspellings 2019-02-07 19:51:03 +01:00
Fabian Möller f3874707ee drive: fix ListR for items with multiple parents 
Fixes #2946
 2019-02-07 19:46:50 +01:00
Fabian Möller f8c2689e77 drive: improve ChangeNotify support for items with multiple parents 2019-02-07 19:46:50 +01:00
Fabian Möller 1433558c01 sftp: perform environment variable expansion on key-file 2019-01-09 10:11:33 +01:00
Fabian Möller 0458b961c5 sftp: add option to force the usage of an ssh-agent 
Also adds the possibility to specify a specific key to request from the
ssh-agent.
 2019-01-09 10:11:33 +01:00
Fabian Möller c1998c4efe sftp: add support for PEM encrypted private keys 2019-01-09 10:11:33 +01:00
Fabian Möller 15ce0ae57c fstests: fix maximum tested size in TestFsPutChunked 
Before this it was possible hat maxChunkSize was incorrectly set to 200.
 2018-10-16 11:50:47 +02:00
Fabian MöllerandNick Craig-Wood 57273d364b fstests: add TestFsPutChunked 2018-10-11 14:47:58 +01:00
Fabian MöllerandNick Craig-Wood 84289d1d69 readers: add NewPatternReader 2018-10-11 14:47:58 +01:00
Fabian MöllerandNick Craig-Wood 98e2746e31 backend: add fstests.ChunkedUploadConfig 
- azureblob
- b2
- drive
- dropbox
- onedrive
- s3
- swift
 2018-10-11 14:47:58 +01:00
Fabian MöllerandNick Craig-Wood c00ec0cbe4 fstests: add ChunkedUploadConfig 2018-10-11 14:47:58 +01:00
Fabian MöllerandNick Craig-Wood 1a40bceb1d backend: unify NewFs path handling for wrapping remotes 
Use the same function to join the root paths for the wrapping remotes
alias, cache and crypt.
The new function fspath.JoinRootPath is equivalent to path.Join, but if
the first non empty element starts with "//", this is preserved to allow
Windows network path to be used in these remotes.
 2018-10-10 17:50:27 +01:00
Fabian Möller 1e2676df26 union: fix ChangeNotify to support multiple remotes 
To correctly support multiple remotes, each remote has to receive a
value on the input channel.
 2018-10-07 11:13:37 +02:00
Fabian MöllerandNick Craig-Wood 1d14972e41 vfs: reduce directory cache cleared by poll-interval 
Reduce the number of nodes purged from the dir-cache when ForgetPath is
called. This is done by only forgetting the cache of the received path
and invalidating the parent folder cache by resetting *Dir.read.

The parent will read the listing on the next access and reuse the
dir-cache of entries in *Dir.items.
 2018-10-02 10:21:14 +01:00
Fabian MöllerandNick Craig-Wood 05fa9cb379 drive: improve directory notifications in ChangeNotify 
When moving a directory in drive, most of the time only a notification
for the directory itself is created, not the old or new parents.

This tires to find the old path in the dirCache and the new path with
the dirCache of the new parent, which can result in two notifications
for a moved directory.
 2018-10-02 10:14:14 +01:00
Fabian Möller 527c0af1c3 drive: cleanup changeNotifyRunner 2018-09-25 17:54:48 +02:00
Fabian MöllerandNick Craig-Wood a20fae0364 drive: code cleanup 2018-09-25 15:20:23 +01:00
Fabian MöllerandNick Craig-Wood 15b1a1f909 drive: add support for apps-script to json export 2018-09-25 15:20:23 +01:00
Fabian MöllerandNick Craig-Wood 80b25daac7 drive: add support for multipart document extensions 2018-09-25 15:20:23 +01:00
Fabian MöllerandNick Craig-Wood 70b30d5ca4 drive: add document links 2018-09-25 15:20:23 +01:00
Fabian MöllerandNick Craig-Wood 0b2fc621fc drive: restructure Object type 2018-09-25 15:20:23 +01:00
Fabian MöllerandNick Craig-Wood 171e39b230 drive: add --drive-import-formats 
Add a new flag to the drive backend to allow document conversions oni upload.
The existing --drive-formats flag has been renamed to --drive-export-formats.
The old flag is still working to be backward compatible.
 2018-09-25 15:20:23 +01:00
Fabian MöllerandNick Craig-Wood 690a44e40e drive: rewrite mime type and extension handling 
Make use of the mime package to find matching extensions and mime types.
For simplicity, all extensions are now prefixed with "." to match the
mime package requirements.

Parsed extensions get converted if needed.
 2018-09-25 15:20:23 +01:00
Fabian Möller d9a3b26e47 vfs: add vfs/poll-interval rc command 
This command can be used to query the current status of the
poll-interval option and also update the value.
 2018-09-25 14:01:13 +02:00
Fabian Möller 1eec59e091 fs: update ChangeNotifier interface 
This introduces a channel to the ChangeNotify function, which can be
used to update the poll-interval and cleanly exit the polling function.
 2018-09-25 14:01:13 +02:00
Fabian MöllerandNick Craig-Wood 03ea05b860 drive: add workaround for slow downloads 
Add --drive-v2-download-min-size flag to allow downloading files via the
drive v2 API. If files are greater than this flag, a download link is
generated when needed. The flag is disabled by default.
 2018-09-18 15:55:50 +01:00
Fabian MöllerandNick Craig-Wood b8678c9d4b vendor: add google.golang.org/api/drive/v2 2018-09-18 15:55:50 +01:00
Fabian MöllerandNick Craig-Wood 13823a7743 drive: fix escaped chars in documents during list 
Fixes #2591
 2018-09-18 15:53:44 +01:00
Fabian MöllerandNick Craig-Wood 3c44ef788a cache: add plex_insecure option to skip certificate validation 
Fixes #2215
 2018-09-10 21:19:25 +01:00
Fabian MöllerandNick Craig-Wood e5663de09e docs: add section for .plex.direct urls to cache 2018-09-10 21:15:18 +01:00
Fabian MöllerandNick Craig-Wood f6ee0795ac cache: preserve leading / in wrapped remote path 
When combining the remote value and the root path, preserve the absence
or presence of the / at the beginning of the wrapped remote path.

e.g. a remote "cloud:" and root path "dir" becomes "cloud:dir" instead
of "cloud:/dir".

Fixes #2553
 2018-09-10 20:35:50 +01:00
Fabian MöllerandNick Craig-Wood eb5a95e7de crypt: preserve leading / in wrapped remote path 
When combining the remote value and the root path, preserve the absence
or presence of the / at the beginning of the wrapped remote path.

e.g. a remote "cloud:" and root path "dir" becomes "cloud:dir" instead
of "cloud:/dir".
 2018-09-10 20:35:50 +01:00
Fabian MöllerandNick Craig-Wood cc23ad71ce ncdu: return error instead of log.Fatal in Show 2018-09-07 10:06:37 +01:00
Fabian MöllerandNick Craig-Wood 6b8b9d19f3 googlecloudstorage: fix service_account_file been ignored - Fixes #2523 2018-09-04 15:31:20 +01:00
Fabian Möller deda093637 cache: fix error return value of cache/fetch rc method 2018-09-03 17:32:11 +02:00
Fabian Möller a0b3fd3a33 cache: fix worker scale down 
Ensure that calling scaleWorkers will create/destroy the right amount of
workers.
 2018-09-03 12:29:35 +02:00
Fabian Möller cdbe3691b7 cache: add cache/fetch rc function 2018-09-03 12:29:35 +02:00
Fabian Möller 3a0b3b0f6e drive: reformat long API call lines 2018-09-03 12:22:05 +02:00
Fabian MöllerandNick Craig-Wood f818df52b8 config: add List type 2018-08-20 17:38:51 +01:00
Fabian MöllerandNick Craig-Wood 6dc5aa7454 docs: clearify buffer-size is per transfer/filehandle 2018-08-17 18:11:40 +01:00
Fabian MöllerandNick Craig-Wood 552eb8e06b vfs: try to seek buffer on read only files 2018-08-17 18:10:28 +01:00
Fabian MöllerandNick Craig-Wood 2e80d4c18e vfs: update vfs/refresh rc command documentation 2018-08-07 09:31:12 +01:00
Fabian MöllerandNick Craig-Wood 6349147af4 vfs: add non recursive mode to vfs/refresh rc command 2018-08-07 09:31:12 +01:00
Fabian MöllerandNick Craig-Wood 782972088d vfs: add the vfs/refresh rc command 
vfs/refresh will walk the directory tree for the given paths and
freshen the directory cache. It will use the fast-list capability
of the remote when enabled.
 2018-08-07 09:31:12 +01:00
Fabian MöllerandNick Craig-Wood 38381d3786 lsjson: add option to show the original object IDs 2018-08-07 09:28:55 +01:00
Fabian MöllerandNick Craig-Wood eb6aafbd14 cache: implement fs.ObjectUnWrapper 2018-08-07 09:28:55 +01:00
Fabian MöllerandNick Craig-Wood dc5a734522 drive: implement ListR 2018-08-06 21:31:47 +01:00
Fabian MöllerandNick Craig-Wood dd670ad1db drive: handle gdocs when filtering file names in list 
Fixes #2399
 2018-07-30 13:01:16 +01:00
Fabian MöllerandNick Craig-Wood 7983b6bdca vfs: enable vfs-read-chunk-size by default 2018-07-29 18:17:05 +01:00
Fabian MöllerandNick Craig-Wood 9815b09d90 fs: add multipliers for SizeSuffix 2018-07-29 18:17:05 +01:00
Fabian Möller 9c90b5e77c stats: use appropriate Lock func's 2018-07-22 11:33:19 +02:00
Fabian Möller 6791cf7d7f atexit: prevent Run from being called on nil signal 2018-05-12 18:59:25 +02:00
Fabian MöllerandNick Craig-Wood d022c81d99 mount: ensure atexit gets run on interrupt 
When running `rclone mount`, there were 2 signal handlers for `os.Interrupt`.

Those handlers would run concurrently and in some cases cause either unmount or `atexit.Run()` being skipped.

In addition `atexit.Run()` will get called in `resolveExitCode` to ensure cleanup on errors.
 2018-05-12 10:40:44 +01:00
Fabian MöllerandNick Craig-Wood 98bf65c43b vfs: fix ChangeNotify for new or changed folders 
Fixes #2251
 2018-05-05 12:54:03 +01:00
Fabian MöllerandNick Craig-Wood 3b5e70c8c6 drive: fix ChangeNotify for folders 2018-05-05 12:54:03 +01:00
Fabian MöllerandNick Craig-Wood bd3ad1ac3e vfs: add option to read source files in chunks 2018-05-05 12:49:42 +01:00
Fabian MöllerandNick Craig-Wood 9fdf273614 fs: improve ChunkedReader 
- make Close permanent and return errors afterwards
- use RangeSeek from the wrapped reader if present
- add a limit to chunk growth
- correct RangeSeek interface behavior
- add tests
 2018-05-05 12:49:42 +01:00
Fabian Möller 21faac6e6c Add David0rk to contributors 2018-02-27 10:06:56 +01:00
Fabian Möller 167a4396c7 drive: remove debug binary 2018-02-27 09:59:06 +01:00
Fabian MöllerandGitHub 00adf40f9f cryptdecode: use Cipher instead of NewFs (#2087) 
* crypt: extract NewCipher out of NewFs
* cryptdecode: make use of crypt.NewCipher

Fixes #2075
 2018-02-25 12:57:14 +01:00
Fabian MöllerandGitHub aeefa34f62 fstests: add TestInternal (#2085) 
TestInternal allows to perform a custom test on the backend using the
optional InternalTester interface.
 2018-02-25 10:58:06 +01:00
Fabian MöllerandNick Craig-Wood 451cd6d971 fs: add ChunkedReader 2018-02-19 15:45:24 +00:00
Fabian MöllerandNick Craig-Wood a647c54888 fs: add RangeSeeker interface 2018-02-19 15:45:24 +00:00
Fabian MöllerandGitHub 62e72801be vfs: fix race between multiple RWFileHandle (#2052) 
Fixes #2034
 2018-02-18 14:12:26 +01:00
Fabian MöllerandNick Craig-Wood b183bd7f00 alias: add new backend to create aliases for remote names #1049 
The alias backend is a wrapper for an existing remote.
It allows you to name a "remote:path" as an "alias:".
 2018-02-06 18:23:47 +00:00
Fabian MöllerandNick Craig-Wood 3c7a755631 lsjson: explain the Path value in the docs 2018-01-31 20:06:01 +00:00
Fabian MöllerandGitHub cf6d522d2f drive: fix upload to existing file (#2032) 
This fixes uploads to existing files for Google Drive introduced by #2007.
Instead of updating the old file a new "Untitled" file would be created
in the root folder.
 2018-01-30 14:37:06 +01:00
Fabian MöllerandGitHub 29d428040c cache: clean root path (#2023) 
Trim "/" from the root path to fix "slice bounds out of range" panic
in cache.go:1272.

Fixes #1945
 2018-01-30 14:35:40 +01:00
Fabian MöllerandGitHub 1aa482c333 drive: fix chunked upload (#2030) 2018-01-29 23:36:39 +01:00