f-denkena/rclone
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,313 Commits 1 Branch 0 Tags
6fc88ff32e5f4fad26b3ab25ff0534296077aa6a
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Nick Craig-Wood 6fc88ff32e Use --fast-list flag for sync/copy/move - fixes #1277 
Redo test framework to take a -fast-list flag and test remotes with that flag.
 2017-06-14 16:49:40 +01:00
Nick Craig-Wood 50928a5027 Implement --fast-list flag. 
This is supported remotes which can do a recursive listing.  It will
use more memory.

This is related to #1277 but doesn't fix that issue yet.
 2017-06-14 16:49:40 +01:00
Nick Craig-Wood 3a431056e2 gcs, swift: increase directory listing chunk to 1000 to increase performance 2017-06-14 16:49:40 +01:00
Nick Craig-Wood 53c3e5f0ab Add placeholder support for ListR interface. 
The ListR interface will be implemented by remotes that can do a
recursive directory listing more efficiently than just recursing
through the directories.  These include the bucket based remotes.
 2017-06-14 16:49:40 +01:00
Nick Craig-Wood 0edb025257 Fixup tests with dirs vs bucket based fs 2017-06-14 16:49:40 +01:00
Nick Craig-Wood fded4dbea2 yandex: correct error return for listing empty directory 2017-06-14 16:49:40 +01:00
Nick Craig-Wood 7e20e16cff core: Implement Walk directory listing and use in place of Lister 
This is in preparation for removing the Lister code and replacing the
fundamental operation in the Fs with listing a single directory.
 2017-06-14 16:49:40 +01:00
Nick Craig-Wood 1e88f0702a dropbox: fix oauth configuration 
This was broken in c59a292719
 2017-06-14 16:46:46 +01:00
Nick Craig-Wood 68333d34a1 dropbox: make setting mod time on existing files work properly 
This is a fix left over from the v2 conversion.  Dropbox ignores the
client modification on an incoming file if it was identical to the
existing file.  This change deletes the existing file first before
re-uploading the new one.
 2017-06-13 13:58:39 +01:00
Nick Craig-Wood 740b3f6ae2 Fix problems found with ineffassign 2017-06-13 11:52:36 +01:00
Nick Craig-Wood 28fcc53e45 mount test: retry umount as it fails occasionally 
This is because of the background releasing of files which happens
after all the files are closed.
 2017-06-13 10:52:10 +01:00
Nick Craig-Wood 2ca477c57f swift: make sensible error if the user forgets the container - fixes #1470 2017-06-10 14:44:56 +01:00
Nick Craig-Wood 9a11d3efd9 Revert "Start Cat tests from 2 as onedrive doesn't support ranging from 1" 
Now that https://github.com/OneDrive/onedrive-api-docs/issues/543 is
fixed, this can be reverted.

This reverts commit 320c53eab0.
 2017-06-10 13:48:00 +01:00
Nick Craig-Wood 10d5377ed8 acd: remove revoked credentials, allow oauth proxy config and update docs 2017-06-10 12:02:34 +01:00
Nick Craig-Wood ee14efd3c2 config: fix menu selection when no remotes 2017-06-10 11:39:40 +01:00
Nick Craig-Wood b4be7d65a6 Update build to go1.8.3 2017-06-09 12:06:28 +01:00
Nick Craig-Wood 52e1bfae2a oauth: Allow auth_url and token_url to be set in the config file 
If set in the config file, these override the ones configured into the
remote.  This enables alternative oauth servers to be used for all
oauth remotes.  This can only be altered by editing the config file
for the moment.
 2017-06-08 20:35:32 +01:00
Nick Craig-Wood 9c1e703777 swift, b2, gcs, s3: Fix moveto and copyto 
We now make sure the container/bucket is created before creating any objects.
 2017-06-07 14:34:59 +01:00
Nick Craig-Wood b49821956a Fix copyto/moveto test error (see #1261) 2017-06-07 14:08:46 +01:00
Nick Craig-Wood a61ba1e7c4 moveto, copyto: report transfers and checks as per move and copy 2017-06-07 13:02:21 +01:00
Nick Craig-Wood d30cc1e119 Factor RemoteSplit into fs 2017-06-07 12:27:33 +01:00
Nick Craig-Wood 74a3dfc4e1 Fix TestHashSums 2017-06-06 23:21:47 +01:00
Nick Craig-Wood 3fe9448229 drive, acd, onedrive: Cache the directory IDs when reading the parent directory 
This makes directory listings much more efficient (one less
transaction needed) and also fixes #1439 (which was caused by having
to look up a directory name with quotes in which isn't dealt with well
by the list routine) by not doing a directory lookup at all.
 2017-06-05 12:26:30 +01:00
Nick Craig-Wood a5cfdfd233 drive: add team drive support - fixes #885 2017-06-04 22:38:29 +01:00
Nick Craig-Wood bdc19b7c8a fstests: fix -remote flag to override test target 2017-06-04 22:38:29 +01:00
Nick Craig-Wood e92cc8fe2b Add Edward Q. Bridges to contributors 2017-06-04 22:38:10 +01:00
Nick Craig-Wood b047402294 config: Fix save of temp file under Windows - fixes #1458 2017-06-01 16:38:19 +01:00
Nick Craig-Wood 7693cecd17 Add Fabian Möller to contributors 2017-06-01 16:23:48 +01:00
Nick Craig-Wood 48508cb5b7 Add Ruwbin to contributors 2017-06-01 09:03:56 +01:00
Nick Craig-Wood 71028e0f06 dropbox/dbhash: fix errcheck warning 2017-05-30 22:08:49 +01:00
Nick Craig-Wood 52e96bc0e2 dropbox: add missing dbhashsum command 
This was missed from 6381959850
 2017-05-30 19:26:06 +01:00
Nick Craig-Wood 178ff62d6a vendor: add github.com/ncw/dropbox-sdk-go-unofficial and remove github.com/stacktic/dropbox 
In due course this will become github.com/dropbox/dropbox-sdk-go-unofficial
when the fate of https://github.com/dropbox/dropbox-sdk-go-unofficial/pull/14
has been decided.
 2017-05-30 15:49:29 +01:00
Nick Craig-Wood 9d335eb5cb dropbox: add low level retries 2017-05-30 14:49:09 +01:00
Nick Craig-Wood 20da3e6352 Add options to Put, PutUnchecked and Update, add HashOption and speed up local 
* Add options to Put, PutUnchecked and Update for all Fses
  * Use these to create HashOption
  * Implement this in local
  * Pass the option in fs.Copy

This has the effect that we only calculate hashes we need to in the
local Fs which speeds up transfers significantly.
 2017-05-29 12:04:52 +01:00
Nick Craig-Wood 6381959850 dropbox: support Dropbox content hashing scheme - fixes #1302 
* add support to hashing module
  * add dbhashsum to list the hashes
  * add support to dropbox module

This means objects up and downloaded to/from Dropbox will have their
hashes checked.

Note after this change local objects are calculating MD5, SHA1 and
DBHASH which is excessive and needs to be fixed.
 2017-05-29 12:04:44 +01:00
Nick Craig-Wood 8916455e4f dropbox: implement dropbox hasher #1302 2017-05-29 12:04:34 +01:00
Nick Craig-Wood 8e214e838e dropbox: Update dropbox to use the v2 API #349 
This is feature complete with the old version but now supports modification times.
 2017-05-29 12:04:33 +01:00
Nick Craig-Wood 23acd3ce01 oauthutil: Don't expect tokens to have refresh URL 2017-05-29 12:04:33 +01:00
Nick Craig-Wood 5455d34f8c Fix ssh agent on Windows - fixes #1279 2017-05-26 10:21:07 +01:00
Nick Craig-Wood 84512ac77d vendor: add github.com/xanzy/ssh-agent for #1279 2017-05-26 10:21:06 +01:00
Nick Craig-Wood 1ec0327ed7 vendor: update cgofuse (because dep wanted to!) 2017-05-26 10:15:14 +01:00
Nick Craig-Wood 0f07b63fd1 ftp: convert the old config style to the new config style 2017-05-25 10:16:51 +01:00
Nick Craig-Wood 88ef475629 config: allow keys to be deleted from the config file 2017-05-25 10:15:22 +01:00
Nick Craig-Wood cfc5f7bb2d Document another file to edit when making a remote 2017-05-25 09:28:18 +01:00
Nick Craig-Wood ae9f8304fa Attempt to make async buffer test more reliable 2017-05-24 16:24:06 +01:00
Nick Craig-Wood 55755a8e5b Add Sjur Fredriksen to contributors 2017-05-24 15:59:49 +01:00
Nick Craig-Wood a243ea6353 sftp: fix under Windows #1432 
This was caused by erroneous use of filepath to parse unix standard paths
 2017-05-24 15:39:17 +01:00
Nick Craig-Wood 51d2174c0b ftp: check connection before returning it to the pool #1435 
If the last FTP command caused an error, and if the error wasn't a
regular FTP error code, then we check the connection is working using
a NOOP call before returning it to the connection pool.
 2017-05-24 14:47:13 +01:00
Nick Craig-Wood e75db0b14d Add Steven Lu to contributors 2017-05-24 08:44:42 +01:00
Nick Craig-Wood be5b8b8dff Add Bob Potter to contributors 2017-05-24 07:36:38 +01:00
Nick Craig-Wood a9d29c2264 ftp: don't pool the connection if file download failed 2017-05-19 17:45:22 +01:00
Nick Craig-Wood 8f54dc06a2 Use build tags to control when and where cmount is built 2017-05-19 17:08:04 +01:00
Nick Craig-Wood 7daf97f90a Add CircleCI badge to README 2017-05-19 16:06:43 +01:00
Nick Craig-Wood 2cae017738 mountlib: fix race condition in cache clear 2017-05-19 15:47:52 +01:00
Nick Craig-Wood e172f00e0e ftp: fix errors from Close of a stream which hasn't been fully read 2017-05-19 12:28:47 +01:00
Nick Craig-Wood 412dacf8be Add a test for partial reads to all remotes 2017-05-19 12:28:47 +01:00
Nick Craig-Wood cdacf026e4 ftp: implement server side move and directory move 2017-05-18 20:49:36 +01:00
Nick Craig-Wood 0ca6408580 ftp: rework mkdir to be more efficient 2017-05-18 20:49:36 +01:00
Nick Craig-Wood 9627a6142d ftp: support --contimeout 2017-05-18 20:49:36 +01:00
Nick Craig-Wood 6cc783f20b ftp: stop rmdir being recursive 2017-05-18 20:49:36 +01:00
Nick Craig-Wood 3136a75f4d ftp: add connection pool and remove excess locking 2017-05-18 20:49:36 +01:00
Nick Craig-Wood a9101f8608 ftp: Fix for go1.6 and go1.7 2017-05-18 20:49:36 +01:00
Nick Craig-Wood af043eda15 Vendor github.com/jlaffaye/ftp for ftp backend 2017-05-18 20:49:36 +01:00
Nick Craig-Wood 35c210d36f ftp: fix remaining issues to make tests work 
* fix root
  * factor ftpConnection
  * fix path munging
  * fix recursive dir loops after update
  * use fs.Trace and comment out debugs
  * re-arrange and supplement docs
 2017-05-18 20:49:36 +01:00
Nick Craig-Wood 3ed0440bd2 ftp: use path instead of filepath 2017-05-18 20:49:36 +01:00
Nick Craig-Wood c13cff37ef ftp: replace URL parser with url.URL 2017-05-18 20:49:36 +01:00
Nick Craig-Wood fce734662f ftp: fix golint/go vet/errchk errors and move methods into standard order 2017-05-18 20:49:36 +01:00
Nick Craig-Wood e0ba1a2cd2 ftp: fix bitrot 2017-05-18 20:49:36 +01:00
Nick Craig-Wood ae17d88518 Add Bill Zissimopoulos to contributors 2017-05-18 20:48:47 +01:00
Nick Craig-Wood 7ee3cfd7c9 Add Igor Kharin to contributors 2017-05-15 21:03:16 +01:00
Nick Craig-Wood 77cd93ef89 Fix tag to 8 digits of commit to make Appveyor and Travis consistent 2017-05-15 20:58:48 +01:00
Nick Craig-Wood 5b063679b5 travis: install libfuse for cmount build and disable on OS X 2017-05-15 17:41:16 +01:00
Nick Craig-Wood 09093a9954 Use appveyor to build the Windows beta releases 2017-05-15 17:41:16 +01:00
Nick Craig-Wood df0cfa9735 Add -no-clean flag to cross-compile.go 2017-05-15 17:41:16 +01:00
Nick Craig-Wood 64d7489fd2 Add -include, -exclude -cgo to cross-compile.go 2017-05-15 17:41:16 +01:00
Nick Craig-Wood ecedcd0e7f cmount: stop failing tests on Windows 2017-05-15 17:40:44 +01:00
Nick Craig-Wood 3dff91d691 mount: add missing build constraint to fix Windows build 2017-05-15 17:40:15 +01:00
Nick Craig-Wood e131ef3714 Fix appveyor tests after vendor update 2017-05-15 16:56:47 +01:00
Nick Craig-Wood ea0bc278ba cmount: Vendor github.com/billziss-gh/cgofuse 2017-05-15 16:56:47 +01:00
Nick Craig-Wood b553c23d5b Automate production of zip files for Windows 2017-05-15 16:56:47 +01:00
Nick Craig-Wood 4f954896a8 appveyor: make build include WinFsp and test cmount 2017-05-15 16:56:47 +01:00
Nick Craig-Wood b259f8b752 cmount, mount, mountlib: make --read-only reject modify operations 
Normally mount/cmount use `-o ro` to get the kernel to mark the fs as
read only.  However this is ignored by WinFsp, so in addition if
`--read-only` is in effect then return EROFS ("Read only File System")
from all methods which attempt to modify something.
 2017-05-15 16:56:47 +01:00
Nick Craig-Wood 8be8a8e41b mountlib: on read only open of file, make open pending until first read 
This fixes a problem with Windows which seems fond of opening files
just to read their attributes and closing them again.
 2017-05-15 16:56:47 +01:00
Nick Craig-Wood 79aa060e21 win-build.bat example bat file for building with WinFsp under Windows 2017-05-15 16:56:46 +01:00
Nick Craig-Wood f9500729b7 mountlib: fix cross platform tests 2017-05-15 16:56:46 +01:00
Nick Craig-Wood 204a19e67f cmount: Wait for mountpoint to appear on Windows before declaring mounted 2017-05-15 16:56:46 +01:00
Nick Craig-Wood e6ffe3464c cmount: check for filesystem blowing up before Init is called 2017-05-15 16:56:46 +01:00
Nick Craig-Wood 0384364c3e cmount: pass --FileSystemName under windows 2017-05-15 16:56:46 +01:00
Nick Craig-Wood 763facfd78 cmount: implement --fuse-flag to pass commands to fuse library directly 
Useful for `--fuse-flag -h` to see exactly which options the library supports.
 2017-05-15 16:56:46 +01:00
Nick Craig-Wood bc88f1dafa cmount: fix openFile leak 2017-05-15 16:56:46 +01:00
Nick Craig-Wood 0c055a1215 cmount: Statfs: reduce max size of volume for Windows 2017-05-15 16:56:46 +01:00
Nick Craig-Wood 938d7951ab cmount: allow extra options to pass to fuse with -o 2017-05-15 16:56:45 +01:00
Nick Craig-Wood b4466bd9b1 Add -o uid=-1 -o gid=-1 for Windows/WinFsp 2017-05-15 16:56:45 +01:00
Nick Craig-Wood 31f76aa464 cmount: implement no-ops for Fsync, Chmod, Chown, Access, Fsyncdir and stop using fuse.FileSystemBase 2017-05-15 16:56:45 +01:00
Nick Craig-Wood c887c164dc cmount: add function tracing 2017-05-15 16:56:45 +01:00
Nick Craig-Wood 115ac00222 mount, mountlib: move function tracing into mount 2017-05-15 16:56:45 +01:00
Nick Craig-Wood 50e79bc087 fs: Implement fs.Trace for tracing entry and exit of functions 2017-05-15 16:56:45 +01:00
Nick Craig-Wood abda616f84 mountlib: make Nodes also be fmt.Stringer so they debug nicely 2017-05-15 16:56:45 +01:00
Nick Craig-Wood 9c3048580a cmount: fix code quality warnings 2017-05-15 16:56:45 +01:00
Nick Craig-Wood c1d5faa32a mountlib: fix code quality warnings 2017-05-15 16:56:45 +01:00