f-denkena/rclone
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,044 Commits 1 Branch 0 Tags
118a8b949e902791508260c392b3fe9c5f76db70
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Nick Craig-Wood 27714e29c3 s3: note incompatibility with CEPH Jewel - fixes #3015 2019-03-06 11:50:37 +00:00
Nick Craig-Wood 9f8e1a1dc5 drive: fix imports of text files 
Before this change text file imports were ignored.  This was because
the mime type wasn't matched.

Fix this by adjusting the keys in the mime type maps as well as the
values.

See: https://forum.rclone.org/t/how-to-upload-text-files-to-google-drive-as-google-docs/9014
 2019-03-05 17:20:31 +00:00
Nick Craig-Wood 1692c6bd0a vfs: shorten the locking window for vfs/refresh 
Before this change we locked the root directory, recursively fetched
the listing, applied it then unlocked the root directory.

After this change we recursively fetch the listing then apply it with
the root directory locked which shortens the time that the root
directory is locked greatly.

With the original method and the new method the subdirectories are
left unlocked and so potentially could be changed leading to
inconsistencies.  This change makes the potential for inconsistencies
slightly worse by leaving the root directory unlocked at a gain of a
much more responsive system while runing vfs/refresh.

See: https://forum.rclone.org/t/rclone-rc-vfs-refresh-locking-directory-being-refreshed/9004
 2019-03-05 14:17:42 +00:00
Nick Craig-Wood d233efbf63 Add marcintustin to contributors 2019-03-01 17:10:26 +00:00
Nick Craig-Wood f6eb5c6983 lib/pacer: fix test on macOS 2019-03-01 12:27:33 +00:00
Nick Craig-Wood 2bf19787d5 Add Dr.Rx to contributors 2019-03-01 12:25:16 +00:00
Nick Craig-Wood b353c730d8 vfs: make tests work on remotes which don't support About 2019-02-28 14:05:21 +00:00
Nick Craig-Wood 173dfbd051 vfs: read directory and check for a file before mkdir 
Before this change when doing Mkdir the VFS layer could add the new
item to an unread directory which caused confusion.

It could also do mkdir on a file when run on a bucket based remote
which would temporarily overwrite the file with a directory.

Fixes #2993
 2019-02-28 14:05:17 +00:00
Nick Craig-Wood e3bceb9083 operations: fix Overlapping test for Windows native paths 2019-02-28 11:39:32 +00:00
Nick Craig-Wood 52c6b373cc Add calisro to contributors 2019-02-28 10:20:35 +00:00
Nick Craig-Wood 5240f9d1e5 sync: fix integration tests to check correct error 2019-02-27 22:05:16 +00:00
Nick Craig-Wood 997654d77d ncdu: fix display corruption with Chinese characters - #2989 2019-02-27 09:55:28 +00:00
Nick Craig-Wood f1809451f6 docs: add more examples of config-less usage 2019-02-27 09:41:40 +00:00
Nick Craig-Wood 84c650818e sync: don't allow syncs on overlapping remotes - fixes #2932 2019-02-26 19:25:52 +00:00
Nick Craig-Wood c5775cf73d fserrors: don't panic on uncomparable errors 2019-02-26 15:39:16 +00:00
Nick Craig-Wood dca482e058 Add Alexandru Bumbacea to contributors 2019-02-26 15:39:16 +00:00
Nick Craig-Wood 6943169cef Add Six to contributors 2019-02-26 15:38:25 +00:00
Nick Craig-Wood 63bb6de491 build: update to use go1.12 for the build 2019-02-26 13:18:31 +00:00
Nick Craig-Wood 0a56a168ff bin/get-github-release.go: scrape the downloads page to avoid the API limit 
This should fix pull requests build failures which can't use the
github token.
 2019-02-25 21:34:59 +00:00
Nick Craig-Wood 88e22087a8 Add Nestar47 to contributors 2019-02-25 21:34:59 +00:00
Nick Craig-Wood c7ecccd5ca mount: remove an obsolete EXPERIMENTAL tag from the docs 2019-02-25 17:53:53 +00:00
Nick Craig-Wood 0b9d7fec0c lsf: add 'e' format to show encrypted names and 'o' for original IDs 
This brings it up to par with lsjson.

This commit also reworks the framework to use ListJSON internally
which removes duplicated code and makes testing easier.
 2019-02-14 14:45:35 +00:00
Nick Craig-Wood 240c15883f accounting: fix total ETA when --stats-unit bits is in effect 2019-02-14 07:56:52 +00:00
Nick Craig-Wood 38864adc9c cmd: Use private custom func to fix clash between rclone and kubectl 
Before this change, rclone used the `__custom_func` hook to control
the completions of remote files.  However this clashes with other
cobra users, the most notable example being kubectl.

Upgrading cobra to master allows us to use a namespaced function
`__rclone_custom_func` which fixes the problem.

Fixes #1529
 2019-02-13 23:02:22 +00:00
Nick Craig-Wood 5991315990 vendor: update github.com/spf13/cobra to master 2019-02-13 23:02:22 +00:00
Nick Craig-Wood 73f0a67d98 s3: Update Dreamhost endpoint - fixes #2974 2019-02-13 21:10:43 +00:00
Nick Craig-Wood ffe067d6e7 azureblob: fix SAS URL support - fixes #2969 
This was broken accidentally in 5d1d93e163 as part of #2654
 2019-02-13 17:36:14 +00:00
Nick Craig-Wood b5f563fb0f vfs: Ignore Truncate if called with no readers and already the correct size 
This fixes FreeBSD which seems to call SetAttr with a size even on
read only files.

This is probably a bug in the FreeBSD FUSE implementation as it
happens with mount and cmount.

See: https://forum.rclone.org/t/freebsd-question/8662/12
 2019-02-12 17:27:04 +00:00
Nick Craig-Wood 9310c7f3e2 build: update to use go1.12rc1 for the build 2019-02-12 16:23:08 +00:00
Nick Craig-Wood 1c1a8ef24b webdav: allow IsCollection property to be integer or boolean - fixes #2964 
It turns out that some servers emit "true" or "false" rather than "1"
or "0" for this property, so adapt accordingly.
 2019-02-12 12:33:08 +00:00
Nick Craig-Wood 2cfbc2852d docs: move --no-traverse docs to the correct section 2019-02-12 12:26:19 +00:00
Nick Craig-Wood b167d30420 Add client side TLS/SSL flags --ca-cert/--client-cert/--client-key 
Fixes #2966
 2019-02-12 12:26:19 +00:00
Nick Craig-Wood ec59760d9c pcloud: remove duplicated UserInfo.Result field spotted by go vet 2019-02-12 11:53:26 +00:00
Nick Craig-Wood 076d3da825 operations: resume downloads if the reader fails in copy - fixes #2108 
This puts a shim on the reader opened by Copy so that if an error is
returned, the reader is re-opened at the correct seek point.

This should make downloading very large files more reliable.
 2019-02-12 11:47:57 +00:00
Nick Craig-Wood c3eecbe933 dropbox: retry blank errors to fix long listings 
Sometimes dropbox returns blank errors in listings - retry this

See: https://forum.rclone.org/t/bug-sync-dropbox-to-gdrive-failing-for-large-files-50gb-error-unexpected-eof/8595
 2019-02-10 20:55:16 +00:00
Nick Craig-Wood d8e5b19ed4 build: switch to semvar compliant version tags 
Fixes #2960
 2019-02-10 20:55:16 +00:00
Nick Craig-Wood 43bc381e90 vendor: update all dependencies 2019-02-10 20:55:16 +00:00
Nick Craig-Wood fb5ee22112 Add Vince to contributors 2019-02-10 20:55:16 +00:00
Nick Craig-Wood 71d1890316 build: ignore testbuilds when uploading to github 2019-02-09 12:22:06 +00:00
Nick Craig-Wood d29c545627 Start v1.46-DEV development 2019-02-09 12:21:57 +00:00
Nick Craig-Wood eb85ecc9c4 Version v1.46 2019-02-09 10:42:57 +00:00
Nick Craig-Wood 0dc08e1e61 Add James Carpenter to contributors 2019-02-09 09:00:22 +00:00
Nick Craig-Wood 60a4a8a86d genautocomplete: add remote path completion for bash - fixes #1529 
Thanks to:
- Christopher Peterson (@cspeterson) for the original script
- Danil Semelenov (@sgtpep) for many refinements
 2019-02-08 19:03:30 +00:00
Nick Craig-Wood 8ec55ae20b Fix broken flag type tests 
Introduced in fc1bf5f931
 2019-02-07 16:42:26 +00:00
Nick Craig-Wood fc1bf5f931 Make flags show up with their proper names, eg SizeSuffix rather than int 2019-02-07 11:57:26 +00:00
Nick Craig-Wood 578d00666c test_all: make -clean not give up on the first error 2019-02-07 11:29:52 +00:00
Nick Craig-Wood f5c853b5c8 Add Jonathan to contributors 2019-02-07 11:29:16 +00:00
Nick Craig-Wood 8217f361cc webdav: if MKCOL fails with 423 Locked assume the directory exists 
This fixes the integration tests with owncloud
 2019-02-07 11:00:28 +00:00
Nick Craig-Wood a0016e00d1 mega: return error if an unknown length file is attempted to be uploaded 
This fixes the integration test created in #2947 to attempt to flush
out non-conforming backends.
 2019-02-07 10:43:31 +00:00
Nick Craig-Wood 99c37028ee build: disable go modules for travis build 2019-02-06 21:25:32 +00:00
Nick Craig-Wood cfba337ef0 lib/pool: fix memory leak by freeing buffers on flush 2019-02-06 17:20:54 +00:00
Nick Craig-Wood fd370fcad2 vendor: update github.com/t3rm1n4l/go-mega to add new error codes 2019-02-05 17:22:28 +00:00
Nick Craig-Wood c680bb3254 box: document how to use rclone with Enterprise SSO 
Thanks to Lorenzo Grassi for help with this.
 2019-02-05 14:29:13 +00:00
Nick Craig-Wood 7d5d6c041f vendor: update github.com/t3rm1n4l/go-mega to fix v2 account login 
Fixes #2771
 2019-02-04 17:33:15 +00:00
Nick Craig-Wood bdc638530e walk: make NewDirTree always use ListR #2946 
This fixes vfs/refresh with recurse=true needing the --fast-list flag
 2019-02-04 10:37:27 +00:00
Nick Craig-Wood 315cee23a0 http: add an example with username and password 2019-02-04 10:30:05 +00:00
Nick Craig-Wood 2135879dda lsjson: use exactly the correct number of decimal places in the seconds 2019-02-03 20:03:23 +00:00
Nick Craig-Wood da90069462 lib/pool: only flush buffers if they are unused between flush intervals 2019-02-03 19:07:50 +00:00
Nick Craig-Wood 08c4854e00 webdav: fix identification of directories for Bitrix Site Manager - #2716 
Bitrix Site Manager emits `<D:resourcetype><collection/></D:resourcetype>`
missing the namespace on the `collection` tag.  This causes the item
to be identified as a file instead of a directory.

To work around this look at the Microsoft extension prop
`iscollection` which seems to be emitted as well.
 2019-02-03 12:34:18 +00:00
Nick Craig-Wood a838add230 fstests: skip chunked uploading tests with -short 2019-02-03 12:28:44 +00:00
Nick Craig-Wood d68b091170 hubic: make error message more informative if authentication fails 2019-02-03 12:25:19 +00:00
Nick Craig-Wood d809bed438 Add weetmuts to contributors 2019-02-03 12:19:08 +00:00
Nick Craig-Wood 6e153781a7 rc: add help to show how to set log level with options/set 2019-02-03 11:47:57 +00:00
Nick Craig-Wood f27c2d9760 vfs: make cache tests more reliable 2019-02-02 16:26:55 +00:00
Nick Craig-Wood eb91356e28 fs/asyncreader: optionally user mmap for memory allocation with --use-mmap #2200 
This replaces the `sync.Pool` allocator with lib/pool.  This
implements a pool of buffers of up to 64MB which can be re-used but is
flushed every 5 seconds.

If `--use-mmap` is set then rclone will use mmap for memory
allocations which is much better at returning memory to the OS.
 2019-02-02 14:35:56 +00:00
Nick Craig-Wood bed2971bf0 lib/pool: a buffer recycling library which can be optionally be used with mmap 2019-02-02 14:35:56 +00:00
Nick Craig-Wood f0696dfe30 lib/mmap: library to do memory allocation with anonymous memory maps 2019-02-02 14:35:56 +00:00
Nick Craig-Wood a43ed567ee vfs: implement --vfs-cache-max-size to limit the total size of the cache 2019-02-02 12:30:10 +00:00
Nick Craig-Wood fffdbb31f5 bin/get-github-release.go: Use GOPATH/bin by preference to place binary 2019-02-02 11:45:07 +00:00
Nick Craig-Wood cacefb9a82 bin/get-github-release.go: automatically choose the right os/arch 
This fixes the install of golangci-lint on non Linux platforms
 2019-02-02 11:45:07 +00:00
Nick Craig-Wood d966cef14c build: fix problems found with unconvert 2019-02-02 11:45:07 +00:00
Nick Craig-Wood a551978a3f build: fix problems found with structcheck linter 2019-02-02 11:45:07 +00:00
Nick Craig-Wood 97752ca8fb build: fix problems found with ineffasign linter 2019-02-02 11:45:07 +00:00
Nick Craig-Wood 8d5d332daf build: fix problems found with golint 2019-02-02 11:45:07 +00:00
Nick Craig-Wood 6b3a9bf26a build: fix problems found by the deadcode linter 2019-02-02 11:45:07 +00:00
Nick Craig-Wood c1d9a1e174 build: use golangci-lint for code quality checks 2019-02-02 11:45:07 +00:00
Nick Craig-Wood 98120bb864 bin/get-github-release.go: enable extraction of binary not in root of tar 
Also fix project name regexp to allow -
 2019-02-02 11:34:51 +00:00
Nick Craig-Wood f8ced557e3 mount: print more things in seek_speed test 2019-02-02 11:30:49 +00:00
Nick Craig-Wood c496efe9a4 Add Wojciech Smigielski to contributors 2019-02-01 17:12:43 +00:00
Nick Craig-Wood cf583e0237 Add Rémy Léone to contributors 2019-02-01 17:12:43 +00:00
Nick Craig-Wood be643ecfbc sftp: don't error on dangling symlinks 2019-02-01 16:43:26 +00:00
Nick Craig-Wood 0c4ed35b9b build: improve beta tidy script 2019-02-01 16:40:55 +00:00
Nick Craig-Wood 4e4feebf0a drive: fix google docs in rclone mount in some circumstances #1732 
Before this change any attempt to access a google doc in an rclone
mount would give the error "partial downloads are not supported while
exporting Google Documents" as the mount uses ranged requests to read
data.

This implements ranged requests for a limited number of scenarios,
just enough so that Google docs can be cat-ed from an rclone mount.
When they are cat-ed then they receive their correct size also.
 2019-01-31 10:39:13 +00:00
Nick Craig-Wood f799be1d6a local: fix symlink tests under Windows 2019-01-29 15:40:49 +00:00
Nick Craig-Wood 74297a0c55 local: make sure we close file handle in local tests 
...as Windows can't remove a directory with an open file handle in
 2019-01-29 15:23:42 +00:00
Nick Craig-Wood 7e13103ba2 Add kayrus to contributors 2019-01-29 14:43:25 +00:00
Nick Craig-Wood 6f25e48cbb ftp: fix docs to note ftp_proxy isn't supported 2019-01-29 14:38:26 +00:00
Nick Craig-Wood 7e99abb5da Add Matt Robinson to contributors 2019-01-29 14:38:26 +00:00
Nick Craig-Wood 629019c3e4 Add yair@unicorn to contributors 2019-01-29 14:38:26 +00:00
Nick Craig-Wood b26276b416 union: fix poll-interval not working - fixes #2837 
Before this change the union remote was using whether the writable
union could poll for changes to decide whether the union mount could
poll for changes.

The fix causes the union backend to signal it can poll for changes if
**any** of the remotes can poll for changes.
 2019-01-28 14:43:12 +00:00
Nick Craig-Wood e317f04098 local: make using -l/--links with -L/--copy-links throw an error #1152 2019-01-28 13:47:27 +00:00
Nick Craig-Wood 65ff330602 local: add tests for -l feature #1152 2019-01-28 13:47:27 +00:00
Nick Craig-Wood 52763e1918 local: when using -l fix setting modification times of symlinks #1152 
Before this change it was setting the modification times of the things
that the symlinks pointed to.

Note that this is only implemented for unix style OSes.  Other OSes
will not attempt to set the modification time of a symlink.
 2019-01-28 13:47:27 +00:00
Nick Craig-Wood c294068780 webdav: support About - fixes #2937 
This means that `rclone about` will work with Webdav backends and `df`
will be correct when using them in `rclone mount`.
 2019-01-28 13:45:09 +00:00
Nick Craig-Wood 8a774a3dd4 webdav: support MD5 and SHA1 hashes with Owncloud and Nextcloud - fixes #2379 2019-01-28 13:45:09 +00:00
Nick Craig-Wood 53a8b5a275 vfs: Implement renaming of directories for backends without DirMove #2539 
Previously to this change, backends without the optional interface
DirMove could not rename directories.

This change uses the new operations.DirMove call to implement renaming
directories which will fall back to Move/Copy as necessary.
 2019-01-27 21:26:56 +00:00
Nick Craig-Wood bbd03f49a4 operations: Implement DirMove for moving a directory #2539 
This does the equivalent of sync.Move but is specialised for moving
files in one backend.
 2019-01-27 21:26:56 +00:00
Nick Craig-Wood e31578e03c s3: Auto detect region for buckets on operation failure - fixes #2915 
If an incorrect region error is returned while using a bucket then the
region is updated, the session is remade and the operation is retried.
 2019-01-27 21:22:49 +00:00
Nick Craig-Wood 0855608bc1 drive: add --drive-pacer-burst config to control bursting of the rate limiter 2019-01-27 21:19:20 +00:00
Nick Craig-Wood f8dbf8292a pacer: control the minSleep with a rate limiter and allow burst 
This will mean rclone tracks the minimum sleep values more precisely
when it isn't rate limiting.

Allowing burst is good for some backends (eg Google Drive).
 2019-01-27 21:19:20 +00:00