f-denkena/rclone
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
292 Commits 1 Branch 0 Tags
59ba8f28c8d93537118dd6ba1f1dd8121a823351
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Nick Craig-Wood 59ba8f28c8 Implement move command - fixes #35 
* Define Mover interface to move a single object
  * Define DirMover interface to move a directory
  * Implement DirMove operation
  * Add `rclone move` command
  * Tests for Dir Move

To Do
  * Implement Move, DirMover in local, drive, dropbox
  * unit test for Mover
  * unit test for DirMover
 2015-08-28 08:49:16 +01:00
Nick Craig-Wood d298b578ab s3: Fix after upstream API changes in aws-sdk-go/aws - fixes #114 2015-08-28 08:47:41 +01:00
Nick Craig-Wood fabbc035c4 Make a current version download with a fixed URL for scripting - fixes #106 2015-08-27 20:11:11 +01:00
Nick Craig-Wood 6530b07cde FAQ entry about copying the config file 2015-08-27 19:46:28 +01:00
Nick Craig-Wood f8b7eaec93 s3: Document cross region bucket limitations - fixes #105 2015-08-25 20:15:50 +01:00
Nick Craig-Wood 5c226e91c0 Ignore rmdir in limited fs rather than throwing error - fixes #112 2015-08-25 19:16:25 +01:00
Nick Craig-Wood 8e3d45d2dc dropbox: increase chunk size to improve upload speeds - fixes #103 
Chunks aren't buffered in memory, so chose 128M as the default size as
producing the maximum throughput.  This takes the throughput from 78
kBytes/s to 4MBytes/s a 50x improvement!
 2015-08-25 19:01:37 +01:00
Nick Craig-Wood a96b522958 Implement server side copies if possible - fixes #99 
Add optional fs.Copier interface

Implemented for
  * swift
  * s3
  * drive
  * dropbox
  * google cloud storage
 2015-08-23 21:18:38 +01:00
Nick Craig-Wood fedf81c2b7 Add Name() to Fs interface to return name as passed to NewFs 2015-08-23 13:36:38 +01:00
Nick Craig-Wood 0c6f816a49 Implement --retries flag - fixes #109 2015-08-20 21:07:00 +01:00
Nick Craig-Wood dfe771fb0c Correct log messages for remotes which don't support modtime/md5sum 2015-08-20 20:48:58 +01:00
Nick Craig-Wood bc19e2d84b dropbox: Issue an error message when trying to upload bad file name - fixes #108 2015-08-20 20:46:35 +01:00
Nick Craig-Wood 8c4d91cff7 Add privacy policy to the website 2015-08-19 22:10:04 +01:00
Nick Craig-Wood 2fcc18779b Version v1.18 2015-08-17 17:59:37 +01:00
Nick Craig-Wood 96cc3e5a0b Add FAQ to menu and documentation fixes 2015-08-17 17:26:37 +01:00
Nick Craig-Wood 1d9e76bb0f dropbox: remove datastore - Fixes #55 #84 
This means that dropbox no longer stores MD5SUMs and modified times.

Fixup the tests so that blank MD5SUMs are ignored, and that if
Precision is set to a fs.ModTimeNotSupported, ModTimes can be ignored too.

This opens the door for other FSs which don't support metadata easily.
 2015-08-17 17:26:36 +01:00
Nick Craig-Wood 337110b7a0 s3: remove verbose debug about invalid md5sums for multipart upload 2015-08-16 18:14:22 +01:00
Nick Craig-Wood 83733205f6 Fix favicon - fixes #79 2015-08-16 15:51:39 +01:00
Nick Craig-Wood d8306938a1 drive: Add --drive-use-trash flag so rclone trashes instead of deletes - fixes #82 2015-08-16 15:51:39 +01:00
Nick Craig-Wood 88ea8b305d drive: Add "Forbidden to download" message for files with no downloadURL - fixes #95 2015-08-16 15:51:39 +01:00
Nick Craig-Wood e2f4d7b5e3 FAQ entry about using rclone from multiple places - fixes #78 2015-08-16 15:51:39 +01:00
Nick Craig-Wood 8140869767 s3: Update documentation after transition to new SDK - also fixes #47 2015-08-16 15:51:38 +01:00
Nick Craig-Wood 6a8de87116 s3: make v2 signatures work for ceph 2015-08-16 15:51:38 +01:00
Nick Craig-Wood 0da6f24221 s3: use official github.com/aws/aws-sdk-go including multipart upload - fixes #101 2015-08-16 15:51:04 +01:00
Nick Craig-Wood 771e60bd07 Show errors when reading the config file 2015-08-15 17:15:02 +01:00
Nick Craig-Wood 40b3c4883f testing: add -remote flag to allow unit tests to run on other remotes 
Use this on the individual fs unit tests, eg

    cd s3
    go run -v -remote mytestS3:
 2015-08-15 15:40:18 +01:00
Nick Craig-Wood e320f4a988 Add Shimon Doodkin to contributors 2015-08-10 11:25:21 +01:00
Nick Craig-Wood 472f065ce7 Make sizes in Dropbox 64 bit - fixes #80 2015-08-10 11:17:54 +01:00
Nick Craig-Wood 6c6d7eb770 Drop support for Go 1.1 2015-08-10 11:17:54 +01:00
Nick Craig-Wood c646ada3c3 docs: add FAQ and update docs 2015-08-10 11:17:43 +01:00
Nick Craig-Wood f55359b3ac Fix created directories not obeying umask 2015-07-29 10:21:12 +01:00
Nick Craig-Wood 9d9a17547a Version v1.17 2015-06-14 15:36:16 +01:00
Nick Craig-Wood c6dc88766b Add Leonid Shalupov to contributors 2015-06-14 15:32:16 +01:00
Nick Craig-Wood c663e24669 Note how to update rclone dependencies in the install documentation 2015-06-14 15:04:39 +01:00
Nick Craig-Wood 5948764e9e Link to changelog 2015-06-14 15:04:29 +01:00
Nick Craig-Wood 539ad44757 Version v1.16 2015-06-09 18:00:33 +01:00
Nick Craig-Wood 74994a2ec1 Fix uploading big files which was causing timeouts or panics 
The symtom was one of these two on upload of files only
  * panic: d.nx != 0 in crypto/md5.(*digest).checkSum
  * read tcp: i/o timeout

It turned out to be a combination of two upstream bugs

  * https://github.com/mreiferson/go-httpclient/commit/5a2187309eac92103d0329aaae48e06837f260fb
  * https://groups.google.com/forum/#!topic/golang-dev/0Nl6k5Sj6UU

This commit contains a work-around for the second problem, I've fixed
the first and had the change accepted upstream.
 2015-06-09 17:32:45 +01:00
Nick Craig-Wood 97dced6a0b Don't check md5sum after download with --size-only - fixes #75 2015-06-09 13:18:40 +01:00
Nick Craig-Wood e04acb09ce Version v1.15 2015-06-06 15:45:00 +01:00
Nick Craig-Wood 87ed7fc932 Document rclone's limitations with directories - fixes #69 2015-06-06 14:33:08 +01:00
Nick Craig-Wood 90744301d3 Fix package docs so they appear in godoc correctly 2015-06-06 14:24:30 +01:00
Nick Craig-Wood bf4879f57f Expand docs and remove duplicated information 2015-06-06 14:17:14 +01:00
Nick Craig-Wood e22b445cff Implement --size-only flag to sync on size not checksum & modtime - fixes #75 2015-06-06 08:49:01 +01:00
Nick Craig-Wood 5ab7970e18 dropbox: update docs about case insensitivity - see #53 2015-06-06 08:44:09 +01:00
Nick Craig-Wood e984eeedc4 Add Alex Couper to contributors 2015-06-05 20:12:56 +01:00
Nick Craig-Wood 968b5a0984 Update notes 2015-06-05 20:00:36 +01:00
Nick Craig-Wood d9fcc32f70 Version v1.14 2015-05-21 20:13:40 +01:00
Nick Craig-Wood 870a9fc3b2 local: fix encoding of non utf-8 file names - fixes #66 2015-05-21 18:40:16 +01:00
Nick Craig-Wood 8e3703abeb drive: docs about rate limiting 2015-05-21 18:39:46 +01:00
Nick Craig-Wood ba81277bbe google cloud storage: Fix compile after API change in "google.golang.org/api/storage/v1" 
The breaking change is: google-api-go-generator: remove underscores from identifiers.
 2015-05-19 08:18:26 +01:00
Nick Craig-Wood 88293a4b8a Version v1.13 2015-05-10 12:39:06 +01:00
Nick Craig-Wood 981104519e Revise documentation (especially sync) - fixes #39 2015-05-10 12:17:24 +01:00
Nick Craig-Wood 1d254a3674 Implement --timeout and --conntimeout - fixes #54 
NB dropbox still to do
 2015-05-10 11:29:55 +01:00
Nick Craig-Wood f88d171afd s3: ignore etags from multipart uploads which aren't md5sums - fixes #56 2015-05-10 11:29:55 +01:00
Nick Craig-Wood ba2091725e Version v1.12 2015-03-15 15:55:38 +00:00
Nick Craig-Wood 7c120b8bc5 drive: add --drive-chunk-size and --drive-upload-cutoff parameters 2015-03-15 15:27:55 +00:00
Nick Craig-Wood 5cc5429f99 drive: switch to insert from update when a failed copy deletes the upload 2015-03-15 15:27:55 +00:00
Nick Craig-Wood 09d71239b6 Make file size render more neatly and prevent from being < 0 2015-03-15 15:27:55 +00:00
Nick Craig-Wood c643e4585e core: Log duplicate files if they are detected 2015-03-15 15:27:55 +00:00
Nick Craig-Wood 873db29391 Log all objects more informatively 2015-03-15 15:27:55 +00:00
Nick Craig-Wood 81a933ae38 drive: Use chunked upload for files - fixes #33 2015-03-15 15:27:55 +00:00
Nick Craig-Wood ecb3c7bcef drive, googlecloudstorage: remove SeekWrapper after googleapi fix 2015-03-04 20:47:59 +00:00
Nick Craig-Wood 80000b904c Version v1.11 2015-03-04 17:59:31 +00:00
Nick Craig-Wood c47c9cd440 swift: add region parameter - fixes #38 2015-03-04 17:09:53 +00:00
Nick Craig-Wood b4a0941d4c In remote paths, change native directory separators to / - fixes #37 2015-03-02 17:04:34 +00:00
Nick Craig-Wood c03d6a1ec3 drive: fix crash on failed to update remote mtime - fixes #36 2015-03-02 09:25:33 +00:00
Nick Craig-Wood 46d39ebaf7 Factor Mime Type guessing into fs.MimeType() 2015-03-02 09:21:15 +00:00
Nick Craig-Wood fe68737268 Fix niggles found by go vet 2015-02-28 15:35:54 +00:00
Nick Craig-Wood 2360bf907a Add synchronization to list output to stop corruptions - fixes #29 2015-02-28 15:30:40 +00:00
Nick Craig-Wood aa093e991e Ensure all stats/log messages to go stderr - fixes #30 2015-02-28 14:39:00 +00:00
Nick Craig-Wood a5974999eb Update docs - fixes #32 2015-02-28 14:15:47 +00:00
Nick Craig-Wood 24a6ff54c2 Add --log-file flag to log everything (including panics) to file 2015-02-28 08:10:20 +00:00
Nick Craig-Wood e89ea3360e Make it possible to disable stats printing with --stats=0 2015-02-27 15:22:26 +00:00
Nick Craig-Wood 85f8552c4d Tidy logging 2015-02-27 15:22:05 +00:00
Nick Craig-Wood a287e3ced7 Implement --bwlimit to limit data transfer bandwidth 2015-02-27 15:03:47 +00:00
Nick Craig-Wood 8e4d8d13b8 drive: rename internal api 2015-02-20 09:51:07 +00:00
Nick Craig-Wood cf208ad21b Version v1.10 2015-02-12 18:00:20 +00:00
Nick Craig-Wood 0faed16899 s3: list an unlimited number of items - fixes #22 2015-02-10 17:58:29 +00:00
Nick Craig-Wood 8d1c0ad07c Fix config loop - fixes #25 2015-02-10 16:48:04 +00:00
Nick Craig-Wood 165e89c266 Version v1.09 2015-02-07 22:44:23 +00:00
Nick Craig-Wood b4e19cfd62 windows: make tests work properly 2015-02-07 22:32:51 +00:00
Nick Craig-Wood 20ad96f3cd windows: Stop drive letters (eg C:) getting mixed up with remotes (eg drive:) 
This was done by stopping the user configuring single letter remotes
and making sure we don't treat single letter remotes as a remote name,
but as a drive letter.
 2015-02-07 22:32:51 +00:00
Nick Craig-Wood d64a37772f local: Fix directory separators on Windows - fixes #24 2015-02-07 22:32:51 +00:00
Nick Craig-Wood 5fb6f94579 drive: fix rate limit exceeded errors - fixes #20 
This is done by pacing the requests to drive and backing them off
using an exponential increase.  Put and Modify operations can now be
retried also.
 2015-02-07 22:32:51 +00:00
Nick Craig-Wood 20535348db Update docs to remove obsolete bug - fixes #21 2015-02-07 22:32:51 +00:00
Nick Craig-Wood 3d83a265c5 Update notes 2015-02-05 22:44:02 +00:00
Nick Craig-Wood 18a8a61cc5 Release v1.08 2015-02-04 22:31:56 +00:00
Nick Craig-Wood 1758621a51 drive: fix subdirectory listing to not list entire drive - fixes #23 
This was causing inexplicably slow transfers on subdirectories of
drives with lots of files.
 2015-02-04 22:22:03 +00:00
Nick Craig-Wood 5710247bf6 drive: Fix SetModTime 2015-01-04 23:19:59 +00:00
Nick Craig-Wood 78b03929b7 Fix ModTime test 2015-01-04 16:57:55 +00:00
Nick Craig-Wood 492362ec7d Catch nil from List() in tests 2015-01-04 16:23:22 +00:00
Nick Craig-Wood 51b24a1dc6 dropbox: adapt code to recent library changes 2014-12-23 13:55:22 +00:00
Nick Craig-Wood cfdb48c864 Version v1.07 2014-12-23 11:26:32 +00:00
Nick Craig-Wood 14567952b3 google cloud storage: Fix memory leak - fixes #17 
This was the same problem as issue #5 (which affected google drive)
 2014-12-23 11:03:34 +00:00
Nick Craig-Wood 2b052671e2 swift: Add docs for tentant 2014-12-12 20:38:35 +00:00
Nick Craig-Wood 439a126af6 Version v1.06 2014-12-12 20:13:03 +00:00
Nick Craig-Wood 0fb35f081a Use new location of Google API package - fixes #16 2014-12-12 20:02:08 +00:00
Nick Craig-Wood 9ba25c7219 Test with go 1.4 too 2014-12-12 19:27:14 +00:00
Nick Craig-Wood af9c447146 Fix "Couldn't find home directory" on OSX - fixes #15 2014-12-12 19:18:23 +00:00
Nick Craig-Wood ee6b39aa6c Add tenant parameter for swift - fixes #13 2014-12-12 15:26:08 +00:00