f-denkena/rclone
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
763 Commits 1 Branch 0 Tags
31fe800d6aad386b7c0f59d8edfcffc6ce2a9bb7
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Nick Craig-Wood 31fe800d6a Add crypt to the docs index plus a few docs tweaks 2016-08-24 23:48:37 +01:00
Nick Craig-Wood 3996bbb8cb Version v1.33 2016-08-24 23:02:05 +01:00
Nick Craig-Wood c2599cb116 Fix crypt tests on Windows 2016-08-24 22:21:34 +01:00
Nick Craig-Wood 2c13074f6c drive: document how to make your own client_id - fixes #560 2016-08-24 22:06:41 +01:00
Nick Craig-Wood 059743a1b0 crypt: add to integration tests 2016-08-23 17:45:37 +01:00
Nick Craig-Wood 73cd1f4e88 crypt: Implement DirMover 2016-08-23 17:45:37 +01:00
Nick Craig-Wood a54806e5c1 Fix Move when underlying remote returns ErrorCantMove 2016-08-23 17:45:37 +01:00
Nick Craig-Wood e6a0521ca2 Make it possible to test Fs multiple times and use this with crypt 
We test both the filename encryption modes for crypt.
 2016-08-23 17:45:37 +01:00
Nick Craig-Wood 43eadf278c Remove flattening and replace with {off, standard} name encryption 2016-08-23 17:45:37 +01:00
Nick Craig-Wood 5f375a182d Create TestCrypt remote 2016-08-23 17:45:37 +01:00
Nick Craig-Wood 663dd6ed8b crypt: ask for a second password for the salt 2016-08-23 17:45:37 +01:00
Nick Craig-Wood 226c2a0d83 Implement crypt for encrypted remotes - #219 2016-08-23 17:45:37 +01:00
Nick Craig-Wood b4b4b6cb1c Allow Fs tests to declare new config items 2016-08-23 17:45:37 +01:00
Nick Craig-Wood 9985fc40f4 Make Password parameters obey Optional flag and offer to generate random ones 2016-08-23 17:45:37 +01:00
Nick Craig-Wood b1de4c8cba Implement password Option and re-implement editing 
Editing now shows all the options for the fs and asks one at a time
whether they should be changed.
 2016-08-23 17:45:37 +01:00
Nick Craig-Wood 6a4e424630 Re-implement Obscure/Reveal so they use AES-CTR encryption 2016-08-23 17:45:37 +01:00
Nick Craig-Wood ebb67c135e Fix listToChan passing nil objects to DeleteFile 2016-08-23 17:45:37 +01:00
Nick Craig-Wood 326dcf2470 Add more troublesome symbols to test cases 
These are from #623 #620 #218
 2016-08-23 14:28:05 +01:00
Nick Craig-Wood 86eb80ecdc Add Radek Šenfeld to contributors. 2016-08-23 12:25:39 +01:00
Nick Craig-Wood 037a000cc8 b2: fix stats accounting for upload - fixes #602 2016-08-22 21:19:38 +01:00
Nick Craig-Wood 8a771450d2 docs: Add hover over links on headings 2016-08-22 17:21:06 +01:00
Nick Craig-Wood 1e7dc06ab8 Fix file encoding 2016-08-22 16:47:06 +01:00
Nick Craig-Wood ca841c56a8 Disable smart dashes so --flag shows properly in the docs - fixes #632 2016-08-22 16:46:08 +01:00
Nick Craig-Wood 79eebf1993 onedrive: fix URL escaping in file names - eg uploading files with + in them. 
Fixes #620
Fixes #218
 2016-08-22 10:58:49 +01:00
Nick Craig-Wood bbccf4acd5 Update go versions 
Remove tip for the moment
 2016-08-20 14:14:48 +01:00
Nick Craig-Wood 9e7ddd5efc Fix tests when FUSE isn't present 2016-08-20 14:11:21 +01:00
Nick Craig-Wood 6089f443b9 Fix windows build - fixes #628 
Try to make clearer the distinction between OS paths and rclone paths
(remotes) so it is harder to muddle them up.
 2016-08-20 12:29:54 +01:00
Nick Craig-Wood 84eb7031bb Implement the rclone cat command 2016-08-18 22:45:32 +01:00
Nick Craig-Wood f22029bf3d Add mount command to implement FUSE mounting of remotes #494 
This enables any rclone remote to be mounted and used as a filesystem
with some limitations.

Only supported for Linux, FreeBSD and OS X
 2016-08-18 21:54:54 +01:00
Nick Craig-Wood d7b79b4481 Mark the compiled from source version with -DEV - fixes #627 2016-08-18 21:31:10 +01:00
Nick Craig-Wood b5faaf7116 Fix double close of abort channel - fixes #592 2016-08-18 18:56:57 +01:00
Nick Craig-Wood b4f2ada820 b2: on cleanup delete hide marker if it is the current file #604 2016-08-18 18:36:00 +01:00
Nick Craig-Wood 8a66930bd7 acd: document --acd-upload-wait-time 2016-08-18 17:49:49 +01:00
Nick Craig-Wood 2ebeed6753 acd: Fix token expiry during large uploads 
When rclone is busy doing lots of very long uploads it doesn't refresh
the token. Amazon will fail uploads if they finish when the token is
more than 1 Hour past expiry.

Fix this by keeping track of the number of uploads and refreshing the
token when the token expires if there is an upload in progress.
 2016-08-18 17:39:23 +01:00
Nick Craig-Wood 23d8ba41d5 oauthuil: implement a timer for token expiry 2016-08-18 17:39:23 +01:00
Nick Craig-Wood 4f9e805d44 acd: Work around 408 REQUEST_TIMEOUT and 504 GATEWAY_TIMEOUT errors 
Amazon Drive sometimes returns errors at the end of large uploads

  * 408 REQUEST_TIMEOUT
  * 504 GATEWAY_TIMEOUT
  * 500 Internal error

The file may have been uploaded correctly though, so, on error, wait
for up to 2 minutes for it to appear if it was fully
uploaded (configure timeout with --acd-upload-wait-time).

Issues: #601 #605 #606
 2016-08-18 17:39:23 +01:00
Nick Craig-Wood 3f7107839e Add Per Cederberg to contributors 2016-08-18 17:10:50 +01:00
Nick Craig-Wood ae6018355c Correct parameter order for copy/sync etc 2016-08-06 00:07:36 +01:00
Nick Craig-Wood 0805ec051f Add BasicInfo interface shared between Dir and Object 2016-08-05 17:45:27 +01:00
Nick Craig-Wood e27b91ffb8 Factor each commmand into its own package 2016-08-05 17:13:54 +01:00
Nick Craig-Wood 0a7b34eefc Move internals of rclone command into cmd so it can be imported externally 2016-08-04 22:33:46 +01:00
Nick Craig-Wood 549cac90af Use cobra autogenerated docs 
* put the most up to date docs into the code
  * generate command docs using rclone gendocs
  * put command docs into own directory
  * remake them into MANUAL.md
 2016-08-04 21:47:14 +01:00
Nick Craig-Wood ba0b41dd92 Add gendocs command to rclone 2016-08-04 21:47:14 +01:00
Nick Craig-Wood 2df261e42b Add genautocomplete command to make bash completion script. 2016-08-04 21:47:14 +01:00
Nick Craig-Wood 38adb35abe Make dedupe take an optional mode parameter 2016-08-04 21:47:14 +01:00
Nick Craig-Wood 520ded60e3 Add memtest command for debugging purposes 2016-08-04 21:47:14 +01:00
Nick Craig-Wood ae56df7d4f Add --dedupe-mode only to dedupe command 2016-08-04 21:47:14 +01:00
Nick Craig-Wood 412591dfaf Make rclone use cobra for command line parsing 2016-08-03 17:16:27 +01:00
Nick Craig-Wood 57f8f1ec92 b2: set maximum backoff to 5 Minutes #597 2016-08-01 22:57:52 +01:00
Nick Craig-Wood f0434789cf Encourage using the latest version before submitting an issue. 2016-07-28 10:38:16 +01:00
Nick Craig-Wood c2f6decb9c swift: note that tenant isn't optional for > v1 auth - fixes #563 2016-07-15 18:25:59 +01:00
Nick Craig-Wood 9eeed25418 local: fix filenames with invalid UTF-8 not being uploaded #568 2016-07-15 14:18:09 +01:00
Nick Craig-Wood 67562081f7 Version v1.32 2016-07-13 17:32:39 +01:00
Nick Craig-Wood 41917eb1f2 b2: Fix upload of files large files not in root - fixes #582 2016-07-13 15:28:39 +01:00
Nick Craig-Wood c3e996f10f b2 doc fixes 2016-07-13 14:50:47 +01:00
Nick Craig-Wood 63f6827a0d Version v1.31 2016-07-13 12:28:01 +01:00
Nick Craig-Wood 96e2271cce Factor commands into Makefile 2016-07-13 12:25:19 +01:00
Nick Craig-Wood ac3c83f966 Fix integration tests for drive 2016-07-12 21:38:15 +01:00
Nick Craig-Wood b9c8e61d39 Explicitly check the state in tests after writing files 
...otherwise Amazon Drive will fail.
 2016-07-12 21:36:39 +01:00
Nick Craig-Wood a6056408dd Fix move command - stop it running for overlapping fses - fixes #577 
* Make move command check for overlapping remotes and refuse to run
  * Do copy/delete rather than all the copies then all the deletes
  * Doesn't purge the source - this was unexpected behaviour see #512 and #416
  * Add -list-retries flag to test suite to control retries

This changes the semantics of `move` slightly.  However it now errs on
the side of not deleting stuff.
 2016-07-12 10:49:37 +01:00
Nick Craig-Wood b9479cf7ab Implement --no-update-modtime flag - fixes #511 2016-07-12 10:46:45 +01:00
Nick Craig-Wood 452a5badc1 Add Stefan Weichinger to contributors 2016-07-11 15:32:58 +01:00
Nick Craig-Wood 50addaa91e Add Antonio Messina to contributors 2016-07-11 15:22:17 +01:00
Nick Craig-Wood a20d80565b Tidy stats output - fixes #541 2016-07-11 13:04:30 +01:00
Nick Craig-Wood 56adb52a21 Rename Amazon Cloud Drive to Amazon Drive - fixes #532 2016-07-11 12:42:44 +01:00
Nick Craig-Wood 8c2fc6daf8 s3: Add instructions on how to use rclone with minio 2016-07-11 12:12:28 +01:00
Nick Craig-Wood 4bd9932703 Fix wording in verbose copy logs - fixes #574 2016-07-09 10:11:57 +01:00
Nick Craig-Wood 2a1d4b7563 s3: Add ap-northeast-2 (Seoul) and ap-south-1 (Mumbai) regions - fixes #567 2016-07-06 11:14:59 +01:00
Nick Craig-Wood b394431f18 Improve --files-from docs - fixes #547 2016-07-05 12:33:59 +01:00
Nick Craig-Wood cc628717d8 b2: Add --b2-versions flag so old versions can be listed and retreived. #420 2016-07-05 11:27:04 +01:00
Nick Craig-Wood f3e00133a0 dropbox: Don't retry 461 errors - fixes #551 
461 errors from dropbox indicate some sort of copyright violation.
 2016-07-04 13:45:53 +01:00
Nick Craig-Wood 606961f49d b2: Treat 403 errors (eg cap exceeded) as fatal #420 2016-07-04 13:45:53 +01:00
Nick Craig-Wood 13591c7c00 Redo error handling for sync/copy/move 
* Factor sync/copy/move into its own file
  * Make fatal errors abort the sync
  * Make Copy return errors
  * Make Sync/Copy/Move return the last Copy error if there was one
  * Prioritise returning Fatal errors
  * NoRetry errors are returned if no other types of errors
 2016-07-04 13:45:53 +01:00
Nick Craig-Wood 28f4061892 Add two more classes of error Fatal and NoRetry 
These are for remotes to signal that they have a fatal error and don't
want to continue (eg cap exceeded) or that a particular file shouldn't
be retried for some reason.
 2016-07-04 13:45:52 +01:00
Nick Craig-Wood 018fe80bcb b2: cleanup old file versions - fixes #462 2016-07-02 17:03:08 +01:00
Nick Craig-Wood 0a43ff9c13 Modify interface for accounting to take a string not an fs.Object 2016-07-02 16:58:50 +01:00
Nick Craig-Wood 9aae143833 Implement cleanup command for emptying trash / removing old versions of files 2016-07-01 16:35:36 +01:00
Nick Craig-Wood c8e2531c8b b2: make error handling compliant 2016-07-01 16:23:23 +01:00
Nick Craig-Wood 9290004bb8 pacer: make sleep get-able and set-able 2016-07-01 16:22:51 +01:00
Nick Craig-Wood cbebefebc4 b2: Fix handling of token expiry #420 
Found with --b2-test-mode expire_some_account_authorization_tokens
 2016-07-01 11:47:42 +01:00
Nick Craig-Wood 6f3897ce2c b2: implement --b2-test-mode to set X-Bz-Test-Mode header #420 2016-07-01 11:30:09 +01:00
Nick Craig-Wood ea5878f590 b2: set cutoff for chunked upload to 200MB #420 
This is the value recommended in the b2 integration checklist:

https://www.backblaze.com/b2/docs/integration_checklist.html
 2016-07-01 10:08:09 +01:00
Nick Craig-Wood 46f8e50614 b2: Make upload multi-threaded - fixes #531 2016-07-01 10:04:52 +01:00
Nick Craig-Wood 70dc97231e Convert more tests to use assert/require 2016-06-30 15:45:30 +01:00
Nick Craig-Wood f6a053df6e Automatically set --no-traverse when copying a single file 2016-06-29 17:38:56 +01:00
Nick Craig-Wood af4ef8ad8d Implement --no-traverse flag to stop copy traversing the destination remote. 
Refactor sync/copy/move
  * Don't load the src listing unless doing a sync and --delete-before
  * Don't load the dst listing if doing copy/move and --no-traverse is set

`rclone --no-traverse copy src dst` now won't load either of the
listings into memory so will use the minimum amount of memory.

This change will reduce the amount of memory rclone uses dramatically
too as in normal operations (copy without --notraverse or sync) as it
no longer loads the source file listing into memory at all.

Fixes #8
Fixes #544
Fixes #546
 2016-06-29 17:38:50 +01:00
Nick Craig-Wood 13797a1fb8 Make retry logs be debug in main copy routine 2016-06-28 08:51:57 +01:00
Nick Craig-Wood 3ad8fb8634 Make DeleteFile and DeleteFiles return errors 2016-06-28 08:51:57 +01:00
Nick Craig-Wood ab43005422 Make NewObject return an error 
* make it return an error
  * make a canonical error fs.ErrorNotFound
  * make a test for it
  * remove logs/debugs of error
 2016-06-28 08:51:57 +01:00
Nick Craig-Wood b1f131964e Rename NewFsObject to NewObject 2016-06-28 08:51:57 +01:00
Nick Craig-Wood 1a87b69376 Get rid of LimitedFs - FIXME needs docs on copying single files 
If remote:path points to a file make NewFs return a sentinel error
fs.ErrorIsFile and an Fs which points to the parent.

Use this to remove the LimitedFs and just add this file to the
--files-from list.

This means that server side operations can be used also.

Fixes #518
Fixes #545
 2016-06-28 08:51:43 +01:00
Nick Craig-Wood 5a3b109e25 Fix issues identified by go vet -shadow - fixes #530 2016-06-21 21:17:52 +01:00
Nick Craig-Wood a67c7461ee s3: skip SetModTime for objects > 5GB - fixes #534 2016-06-19 17:26:44 +01:00
Nick Craig-Wood ab0947ee37 Fix typo in changelog 2016-06-18 16:58:37 +01:00
Nick Craig-Wood bd0227450e Version v1.30 2016-06-18 16:41:46 +01:00
Nick Craig-Wood f438f1e9ef Fix stats print 2016-06-18 16:41:46 +01:00
Nick Craig-Wood 3f7b2c1ade Add Justin R. Wilson to contributors 2016-06-18 14:31:17 +01:00
Nick Craig-Wood d3dd672640 Document recursion requirements for Fses 2016-06-18 14:12:47 +01:00
Nick Craig-Wood 2a46be8cf3 b2: implement large file uploading - fixes #456 2016-06-18 13:38:05 +01:00
Nick Craig-Wood 1b4370bde1 Rework retry logic when copying objects 
* Fix off by one retry logic - fixes #406
  * Retry any retriable errors
  * Restructure code
 2016-06-18 10:55:58 +01:00