f-denkena/rclone
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
10,313 Commits 1 Branch 0 Tags
c99f09060c7d3ce3a3fa10248b5abd53fe576d72
Commit Graph
49 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Klaus PostandNick Craig-Wood 5ca7f1fe87 encoder/filename: Wrap scsu package 2021-02-12 11:39:39 +00:00
Klaus PostandNick Craig-Wood f14220ef1e encoder/filename: Add 2 more tables and tests. 2021-02-12 11:39:39 +00:00
Klaus PostandNick Craig-Wood 424aaac2e1 encoder/filename: Add SCSU as tables 
Instead of only adding SCSU, add it as an existing table.

Allow direct SCSU and add a, perhaps, reasonable table as well.

Add byte interfaces that doesn't base64 encode the URL as well with `EncodeBytes` and `DecodeBytes`.

Fuzz tested and decode tests added.
 2021-02-12 11:39:39 +00:00
Klaus PostandGitHub cb7534dcdf lib: Add file name compression 
Allows to compress short arbitrary strings and returns a string using base64 url encoding.

Generator for tables included and a few samples has been added. Add more to init.go

Tested with fuzzing for crash resistance and symmetry, see fuzz.go
 2020-08-13 16:14:11 +01:00
Klaus Post e0aa4bb492 Fix incomplete local hashes. 
Fixes #533
 2016-06-19 16:51:49 +02:00
Klaus PostandNick Craig-Wood 6c0c9abd57 Use "password:" instead of "password>" prompt 
Fixes #410
 2016-04-21 19:39:46 +01:00
Klaus PostandNick Craig-Wood 7f2e9d9a6b Require go v1.5 for compilation 
Google cloud package requires go v1.5 to compile, so we need to require the same for rclone.

Fixes #408
 2016-04-04 17:34:39 +01:00
Klaus PostandNick Craig-Wood 02c11dd4a7 Don't de-reference swift connection 
The connection object contains a mutex, so it is good practice not to dereference it to a value.

Reported by Go tip "go vet".
 2016-03-23 17:09:05 +00:00
Klaus PostandNick Craig-Wood 40dc575aa4 Update Travis CI 
- Only use golint if version is > Go 1.4
- Add Go 1.6 and tip as test targets.
 2016-03-23 17:07:26 +00:00
Klaus PostandNick Craig-Wood f8101771c9 Disable keepalive to keep server from serving stale results. 
Fixes issue #402

Bonus fix: Fix "multiple header writes" warning when no code is received.
 2016-03-23 16:57:56 +00:00
Klaus PostandNick Craig-Wood 8f4d6973fb Fix missing "quit" option when there are no remotes. 2016-03-23 16:57:56 +00:00
Klaus Post e57c4406f3 Add mutex to "warned" map. 
Fixes #385
 2016-03-10 15:51:56 +01:00
klauspostandNick Craig-Wood 758c7f2d84 Avoid b2 temporary file. 
If source can provide SHA1 hash we don't copy input to a temporary file.

Fixes #358
 2016-02-19 18:07:15 +00:00
klauspostandNick Craig-Wood ef06371c93 Create separate interface for object information. 
Take out read-only information about a Fs in a separate struct to limit access.

See discussion at #282.
 2016-02-19 13:31:09 +00:00
klauspost 84b00b362f Change back to original goconfig package. 
Add documentation for `--ask-password`.
 2016-02-17 11:45:05 +01:00
klauspost bfd7601cf9 Add configuration file encryption 
See #317 for details.

Use `rclone config` to add/change/remove password.

Tests that loads the default configuration will now fail with a better error message, and add a switch that makes it possible to disable password prompts and fail instead.

Make it possible to use the "RCLONE_CONFIG_PASS" environment variable as password for configuration.
 2016-02-16 16:32:05 +01:00
klauspostandNick Craig-Wood 14069fd8e6 Implement --delete-before, --delete-during, --delete-after - fixes #252. 2016-01-26 16:57:09 +00:00
klauspostandNick Craig-Wood ce04a073ef Update templates to changes in the latest hugo version 
Fixes #295
 2016-01-16 14:11:52 +00:00
klauspostandNick Craig-Wood 7ae40cb352 Update information on revised hash functionality. 2016-01-16 10:17:11 +00:00
klauspost 78c3a5ccfa Add support for multiple hash types. 
Add support for multiple hash types with negotiation of common hash types for comparison.

Manually rebased version of #277 (see discussion there)
 2016-01-11 13:39:33 +01:00
klauspostandNick Craig-Wood 25f22ec561 Add "--ignore-existing" flag. 
Add option to completely ignore existing files and not consider them for transfer.

Fixes #274
 2016-01-08 08:20:04 +00:00
klauspostandNick Craig-Wood bcbd30bb8a Add easier headless configuration. 
This will allow setting up a remote with copy&paste of values to a headless machine. It will allow copy+pasting a token into the configuration.

This requires rclone to be on a machine with a proper browser. Custom client id and secrets are supported.

To test token generation, use `rclone auth "fs type"`.
 2016-01-07 20:31:23 +00:00
klauspostandNick Craig-Wood 4ce2a84df0 Document workaround for ACD maximum file size. 
Document workaround for ACD maximum file size and display a warning in verbose mode before upload starts.

Fixes #215.
 2016-01-05 17:12:16 +00:00
klauspostandNick Craig-Wood 3c31d711b3 Add local file system option to disable UNC on Windows. 
This will add an option to disable UNC conversion on Windows to deal with buggy file system implementations like EncFS.

Fixes #261
 2016-01-05 17:08:11 +00:00
klauspost b85ddc4e4f Extend CI tests to include formatting checks. 
CI tests now tests 'go vet', 'go fmt' (via goimports) and golint.

Adds Travis experimental OSX support.
 2015-11-03 13:50:29 +01:00
klauspostandNick Craig-Wood e4a9e27a55 Add proxy information to FAQ. 
Fixes #160.
 2015-11-02 20:19:50 +00:00
Klaus PostandNick Craig-Wood 22645eea2e Add AppVeyor Windows CI to tests 
AppVeyor is free, and functions pretty much like Travis, only on Windows.
 2015-11-02 20:11:06 +00:00
klauspostandNick Craig-Wood 345c98ed62 Update to AWS SDK 0.10.0 
Tested with S3 and Dreamhost

Here is a link to the release notes:

http://aws.amazon.com/releasenotes/5476699172355228
 2015-11-02 19:52:11 +00:00
klauspost b872ff0237 Add option to disable server certificate verification. 
The option name mirrors the 'wget' option (also `--no-check-certificate`). The cURL equivalent is called `--insecure`, which is a bit unclear.

Put in the "developers" section in documentation with proper warnings.

Fixes #168
 2015-10-29 16:42:25 +01:00
klauspostandNick Craig-Wood 6a3580c556 Show status of master branch, so it doesn't show the status of the last pushed branch, 2015-10-19 18:56:03 +01:00
klauspostandNick Craig-Wood 16c9fba5de Fix tests failing on Windows. 
* ":" is kept when part of a drive.
  * Create tests.
  * Fix test framework.
 2015-10-19 17:36:15 +01:00
Klaus PostandNick Craig-Wood 6344c3051c Add async readahead buffer 
This adds an async read buffer of 4x4MB when copying files >10MB.

This fixes #164 and reduces the number of IO operations for copy/move.
 2015-10-12 08:30:27 +01:00
klauspost ab9f521cbd Allow '&' and disallow ':' in Windows filenames. 
Fixes #161
 2015-10-05 11:04:25 +02:00
klauspostandNick Craig-Wood 3234c28f7c Minor adjustments to Unc path conversion function. 2015-09-17 18:42:10 +01:00
klauspostandNick Craig-Wood 6fbd9cf24b Make Google Drive Directory reads concurrent for increased speed. 
If you use the -v flag then this will show the progress of directory
listing.  Fixes #143.
 2015-09-17 18:42:10 +01:00
klauspostandNick Craig-Wood bc5b63ffef Move ACD Connection-limiter into Pacer. 2015-09-17 18:42:10 +01:00
klauspost 788ef76f1c Show more of the filename and align output. 
Print more of the file name, and make the output aligned, so it is nicer on frequent updates.
 2015-09-17 15:53:45 +02:00
klauspostandNick Craig-Wood 0a5870208e Display individual transfer progress 
Improve progress printing by displaying individual file progress, as well
as a moving average speed with ETA. Example output:

2015/09/15 16:38:21
Transferred:    183599104 Bytes (4646.49 kByte/s)
Errors:                 0
Checks:                 1
Transferred:            0
Elapsed time:       38.5s
Transferring:
 * 01_06_14.mp3: 33% done. avg: 1280.5, cur: 1288.8 kByte/s. ETA: 1m12s
 * 01_12_15.mp3: 33% done. avg: 1002.2, cur:  943.4 kByte/s. ETA: 1m17s
 * 01_13_14.mp3: 48% done. avg: 1456.8, cur: 1425.2 kByte/s. ETA: 39s
 * 01_19_15.mp3: 28% done. avg: 1226.9, cur: 1114.4 kByte/s. ETA: 1m37s
 2015-09-16 19:42:46 +01:00
klauspost 3219334c3e Add "-race" test to travis. 2015-09-16 12:26:28 +02:00
klauspost 79fd662676 Protect concurrent read/writes to pacer. 
Protects all variables in the pacer from concurrent modifications. It is now safe to modify pacer settings while it is running.

I decided to not go for an RWMutex, since all accesses are very short, so the overhead of an RWMutex isn't worth it.

Fixes #138.
 2015-09-16 12:25:55 +02:00
klauspostandNick Craig-Wood f50f353b5d local: always use UNC paths on Windows - fixes #124, fixes #130, fixes #90 
* Convert all paths to UNC paths on Windows.
  * Update local filesystem to always use UNC paths.
  * Change tests, so they can work with Windows character replacements.
  * Remove "/" suffix on paths.
  * Always use path/filepath
 2015-09-14 19:58:03 +01:00
Klaus PostandNick Craig-Wood 00afe6cc96 ACD: Shorten minimum and maximum sleep. 2015-09-14 18:22:21 +01:00
Klaus PostandNick Craig-Wood dd48e62b7e Limit concurrent directory listings. 
Never run more than "fs.Config.Checkers" directory listings at once.
 2015-09-14 18:11:59 +01:00
Klaus PostandNick Craig-Wood a1a780e847 Read folders in separate goroutines. 
As proposed in the FIXME, read folders in parallel.
This appears to fix "Next token is expired" on very big directories.
The only downside is that this doesn't abort at once if an error is found.
I added some logging, so there is some output for "-v".
 2015-09-14 18:11:59 +01:00
Klaus PostandNick Craig-Wood fa87077211 Retry if a timeout occurs. 
This happens more rarely than 500, but can still be encountered.
 2015-09-14 18:02:35 +01:00
Klaus PostandNick Craig-Wood 6ac7145d2d Retry when we get a "500 Internal server error". 
It seems like this happens randomly, and retrying works fine for that.
 2015-09-14 18:02:35 +01:00
Klaus Post 3ecbf2af25 Fix missing link text. 2015-09-12 11:22:17 +02:00
klauspostandNick Craig-Wood faee50b238 List multiple matched commands. 
* When multiple commands match, list them all.
* Change example to an existing command.
 2015-09-11 19:44:50 +01:00
klauspostandNick Craig-Wood 807d4a3c00 Improve OAUTH2 usability - fixes #131 
Shorten the URL to be used by the user and automatically use the
returned code by the server. The browser is opened on
`http://(bindaddress)/auth`, and redirected to the actual URL. When
the code is returned it is automatically inserted, instead of
requiring a copy+paste.

This is also a workaround for the "open" package, which escapes "&"
wrongly on Windows, so the opened URL's are invalid.
 2015-09-11 19:44:50 +01:00