f-denkena/rclone
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
4,829 Commits 1 Branch 0 Tags
275bf456d3f3a80680ce3bd53d8e10d55d3feac7
Commit Graph
4829 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Nick Craig-Wood 275bf456d3 Add Josh Soref to contributors 2020-10-15 12:57:48 +01:00
Nick Craig-Wood 7dfa871095 Add Dan Hipschman to contributors 2020-10-15 12:57:48 +01:00
Nick Craig-Wood 70cc88de22 Add Ameer Dawood to contributors 2020-10-15 12:57:48 +01:00
Nick Craig-Wood 4bc0f46955 Add Dov Murik to contributors 2020-10-15 12:57:48 +01:00
Anagh Kumar Baranwal 5b09599a23 drive: Added flag --drive-stop-on-download-limit to stop transfers when the download limit is exceeded 
Signed-off-by: Anagh Kumar Baranwal <6824881+darthShadow@users.noreply.github.com>
 2020-10-14 15:51:01 +01:00
David f4dd8e3fe8 dedupe: minor clarification in docs 
dedupe will not delete equal files if they are located in another folder.
 2020-10-14 15:31:47 +01:00
Josh Soref d0888edc0a Spelling fixes 
Fix spelling of: above, already, anonymous, associated,
authentication, bandwidth, because, between, blocks, calculate,
candidates, cautious, changelog, cleaner, clipboard, command,
completely, concurrently, considered, constructs, corrupt, current,
daemon, dependencies, deprecated, directory, dispatcher, download,
eligible, ellipsis, encrypter, endpoint, entrieslist, essentially,
existing writers, existing, expires, filesystem, flushing, frequently,
hierarchy, however, implementation, implements, inaccurate,
individually, insensitive, longer, maximum, metadata, modified,
multipart, namedirfirst, nextcloud, obscured, opened, optional,
owncloud, pacific, passphrase, password, permanently, persimmon,
positive, potato, protocol, quota, receiving, recommends, referring,
requires, revisited, satisfied, satisfies, satisfy, semver,
serialized, session, storage, strategies, stringlist, successful,
supported, surprise, temporarily, temporary, transactions, unneeded,
update, uploads, wrapped

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-14 15:21:31 +01:00
Nick Craig-Wood 51a230d7fd fs: Implement UnWrapObjectInfo for getting original object out of src objects 2020-10-14 15:20:06 +01:00
Anagh Kumar Baranwal fc5b14b620 s3: Added --s3-disable-http2 to disable http/2 
Fixes #4673

Signed-off-by: Anagh Kumar Baranwal <6824881+darthShadow@users.noreply.github.com>
 2020-10-13 17:11:22 +01:00
Stephen Harris bbddadbd04 sftp: remember entered password in AskPass mode 
As reported in

  https://github.com/rclone/rclone/issues/4660#issuecomment-705502792

After switching to a password callback function, if the ssh connection
aborts and needs to be reconnected then the user is-reprompted for their
password.  Instead we now remember the password they entered and just give
that back.  We do lose the ability for them to correct mistakes, but that's
the situation from before switching to callbacks.  We keep the benefits
of not asking for passwords until the SSH connection succeeds (right
known_hosts entry, for example).

This required a small refactor of how `f := &Fs{}` was built, so we can
store the saved password in the Fs object
 2020-10-13 16:53:11 +01:00
Nick Craig-Wood 7428e47ebc local: fix sizes and syncing with --links option on Windows - fixes #4581 
Before this change rclone returned the size from the Stat call of the
link. On Windows this reads as 0 always, however on unix it reads as
the length of the text in the link. This caused errors like this when
syncing:

    Failed to copy: corrupted on transfer: sizes differ 0 vs 13

This change causes Windows platforms to read the link and use that as
the size of the link instead of 0 which fixes the problem.
 2020-10-13 16:29:56 +01:00
Nick Craig-Wood 72083c65ad cmd: make backend env vars show in help as the defaults for backend flags 
Before this change

    RCLONE_DRIVE_CHUNK_SIZE=111M rclone help flags | grep drive-chunk-size

Would show the default value, not the setting of RCLONE_DRIVE_CHUNK_SIZE
as the non backend flags do.

This change makes it work as expected by setting the default of the
option to the environment variable.

Fixes #4659
 2020-10-13 15:43:58 +01:00
Dan Hipschman 70f92fd6b3 crypt: small simplification, no functionality change 2020-10-12 17:20:39 +01:00
Nick Craig-Wood a86cedbc24 vfs: Fix --no-modtime to not attempt to set modtimes (as documented) 
See: https://forum.rclone.org/t/rclone-mount-with-azure-blob-archive-tier/19414
 2020-10-09 17:01:16 +01:00
Nick Craig-Wood 0906f8dd3b onedrive: fix disk usage for sharepoint 
Some onedrive sharepoints appear to return all 0s for quota

    "quota":{"deleted":0,"remaining":0,"total":0,"used":0}

This commit detects this and returns unknown for all quota parts.

See: https://forum.rclone.org/t/zero-size-volume-when-mounting-onedrive-sharepoint/19597
 2020-10-09 14:11:56 +01:00
buengese 664213cedb jottacloud: remove clientSecret from config when upgrading to token based authentication - #4645 2020-10-08 11:51:17 +02:00
Ameer Dawood 75a7226174 mount: docs: correction of repeated word 2020-10-07 14:25:31 +01:00
Stephen Harris 9e925becb6 sftp: defer asking for user passwords until the SSH connection succeeds 
Issue: 4660
    https://github.com/rclone/rclone/issues/4660

Unexpected side effect: a wrong password allows for the user to retry!
 2020-10-07 12:01:17 +01:00
Anagh Kumar Baranwal e3a5bb9b48 s3: Add missing regions for AWS 
Signed-off-by: Anagh Kumar Baranwal <6824881+darthShadow@users.noreply.github.com>
 2020-10-06 16:54:42 +01:00
Dov Murik b7eeb0e260 docs: Box: explain about the backslash-like unicode character 
Add the full name of the backslash-lookalike unicode character.
 2020-10-06 16:47:49 +01:00
Nick Craig-Wood 84d64ddabc Add LaSombra to contributors 2020-10-06 16:42:28 +01:00
Nick Craig-Wood 6c9f92aee6 Add Hekmon to contributors 2020-10-06 16:42:28 +01:00
Nick Craig-Wood 893297760b Add gyutw to contributors 2020-10-06 16:42:28 +01:00
Leo Luan c5c56cda02 vfs: Add a missed update of used cache space 
The missed update can cause incorrect before-cleaning cache stats
and a pre-mature condition broadcast in purgeOld before the cache
space use is reduced below the quota.
 2020-10-06 16:35:23 +01:00
Leo Luan 2295123cad vfs: Add exponential backoff during ENOSPC retries 
Add an exponentially increasing delay during retries up ENOSPC error
to avoid exhausting the 10 retries too soon when the cache space
recovery from item resets is not available from the file system yet
or consumed by other large cache writes.
 2020-10-06 16:35:23 +01:00
Leo Luan ff0280c0cb vfs: Fix missed concurrency control between some item operations and reset 
Item reset is invoked by cache cleaner for synchronous recovery
from ENOSPC errors. The reset operation removes the cache file and
closes/reopens the downloaders.  Although most parts of reset and
other item operations are done with the item mutex held, the mutex
is released during fd.WriteAt and downloaders calls. We used preAccess
and postAccess calls to serialize Reset, ReadAt, and Open, but missed
some other item operations. The patch adds preAccess/postAccess
calls in Sync, Truncate, Close, WriteAt, and rename.
 2020-10-06 16:35:23 +01:00
Leo Luan 64d736a57b vfs: Fix a race condition in retryFailedResets 
A failed item reset is saved in the errItems for retryFailedResets
to process.  If the item gets closed before the retry, the item may
have been removed from the c.item array. Previous code did not
account for this condition. This patch adds the check for the
exitence of the retry items in retryFailedResets.
 2020-10-06 16:35:23 +01:00
Leo Luan 5f1d5a1897 vfs: Fix a deadlock vulnerability in downloaders.Close 
The downloaders.Close() call acquires the downloaders' mutex before
calling the wait group wait and the main downloaders thread has a
periodical (5 seconds interval) call to kick its waiters and the
waiter dispatch function tries to get the mutex. So a deadlock can
occur if the Close() call starts, gets the mutex, while the main
downloader thread already got the timer's tick and proceeded to
call kickWaiters. The deadlock happens when the Close call gets
the mutex between the timer's kick and the main downloader thread
gets the mutex first. So it's a pretty short period of time and
it probably explains why the problem has not surfaced, maybe
something like tens of nanoseconds out of 5 seconds (~10^^-8).
It took 5 days of continued stressing the Close calls for the
deadlock to appear.
 2020-10-06 16:35:23 +01:00
LaSombra aac2406e19 cmd: add --progress-terminal-title to print ETA to terminal title 
Adds a flag, --progress-terminal-title, that when used with --progress,
will print the string `ETA: %s` to the terminal title.

This also adds WriteTerminalTitle to lib/terminal
 2020-10-06 16:34:26 +01:00
Stephen Harris 6dc28ef50a sftp: Allow user to optionally check server hosts key to add security 
Based on Issue 4087
  https://github.com/rclone/rclone/issues/4087

Current behaviour is insecure.  If the user specifies this value then we
switch to validating the server hostkey and so can detect server changes
or MITM-type attacks.
 2020-10-06 16:27:42 +01:00
Hekmon 66def93373 mount cmd: update systemd status with cache stats 2020-10-06 16:21:30 +01:00
Hekmon c58023a9ba enhance systemd integration 
* log level identification
* manual activation with flag
* automatic systemd launch detection
 2020-10-06 16:21:30 +01:00
buengese 3edc9ff0b0 jottacloud: remove DirMove workaround as it's not required anymore - also fixes #4655 2020-10-05 20:13:05 +02:00
edwardxml 8e8ae1edc7 crypt: update docs 
Mostly tense, clarity and point of view proposed changes.

There is still some duplication and benefits that would accrue from further examples.
 2020-10-05 17:19:00 +01:00
Nick Craig-Wood 20b00db390 operations: fix spurious "--checksum is in use but the source and destination have no hashes in common" 
Before this change rclone would emit the message

    --checksum is in use but the source and destination have no hashes in common; falling back to --size-only

When the source or destination hash was missing as well as when the
source and destination had no hashes in common.

This first case is very confusing for users when the source and
destination do have a hash in common.

This change fixes that and makes sure the error message is not emitted
on missing hashes even when there is a hash in common.

See: https://forum.rclone.org/t/source-and-destination-have-no-hashes-in-common-for-unencrypted-drive-to-local-sync/19531
 2020-10-05 16:07:05 +01:00
Nick Craig-Wood db4bbf9521 operations: fix use of --suffix without --backup-dir 
As part of the original work adding this feature it was overlooked
that this didn't actually work for full rclone copy/sync.

This commit fixes the problem and adds a test to make sure it stays
working.

See: https://forum.rclone.org/t/suffix-not-working-on-folder-upload-via-ssh-sftp/19526
 2020-10-04 16:40:20 +01:00
Nick Craig-Wood 2b7994e739 operations: add logs when need to upload files to set mod times #1505 2020-09-29 17:04:29 +01:00
gyutw e7fbdac8e0 fichier: increase maximum file size from 100GB to 300GB - fixes #4634 2020-09-28 20:27:17 +02:00
Nick Craig-Wood 41ec712aa9 ftp,sftp: fix docs for usernames 
- factor env.CurrentUser out of backend/sftp
- Use env.CurrentUser in ftp and sftp
- fix docs to have correct username
 2020-09-27 11:44:05 +01:00
Stephen Harris 17acae2b00 sftp: allow cert based auth via optional pubkey 
Discussion at
  https://forum.rclone.org/t/ssh-certificate-based-authentication-does-not-work/19222

Basically we allow the user to specify their own public key cert rather
than letting the SSH client extract the pubkey from the private key.
This allows certificate based authentication to work.
 2020-09-27 11:10:13 +01:00
Nick Craig-Wood 57261c7e97 mount: docs: remove incorrect statement about --vfs-cache-mode full 
See: https://forum.rclone.org/t/is-this-documentation-correct/19376
 2020-09-27 11:04:59 +01:00
Ivan Andreev d8239e0194 mailru: remove deprecated protocol quirks 2020-09-26 15:38:32 +03:00
Ivan Andreev 004c3796de chunker: disable ListR to fix missing files on GDrive (workaround #3972) 2020-09-26 15:19:16 +03:00
Ivan Andreev 18c7549770 mailru: fix invalid timestamp on corrupted files (fixes #4229) 2020-09-26 15:12:30 +03:00
Nick Craig-Wood e5190f14ce drive: implement "rclone backend copyid" command for copying files by ID 
This allows files to be copied by ID from google drive. These can be
copied to any rclone remote and if the remote is a google drive then
server side copy will be attempted.

Fixes #3625
 2020-09-25 17:53:51 +01:00
Nick Craig-Wood 433b73a5a8 accounting: stabilize display order of transfers on Windows 
Before this change we sorted transfers in the stats list solely on
time started. However if --check-first was in use then lots of
transfers could be started in the same millisecond. Because Windows
time resolution is only 1mS this caused the entries to sort equal and
bounce around in the list.

This change fixes the sort so that if the time is equal it uses the
name which should stabilize the order.

Fixes #4599
 2020-09-24 19:10:37 +01:00
Nick Craig-Wood ab88a3341f Add Russell Cattelan to contributors 2020-09-24 19:10:37 +01:00
Nick Craig-Wood 181da3ce9b Add Christopher Stewart to contributors 2020-09-24 19:10:37 +01:00
Russell Cattelan b14a58c9b8 cmd/mount2: fix the swapped UID / GID values 2020-09-23 23:06:33 +01:00
buengese 60cc2cba1f sftp: always convert the checksum to lower case - fixes #4518 2020-09-22 03:15:09 +02:00