f-denkena/rclone
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5,637 Commits 1 Branch 0 Tags
dd97fbc55f013699cd3364429e719f5d95b9fab7
Commit Graph
822 Commits
Author SHA1 Message Date
lostheli c8cfa43ccc Add a download flag to hashsum and related commands to force rclone to download and hash files locally 
This commit modifies the operations.hashSum function by adding an alternate code path. This code path is triggered by passing downloadFlag = True. When activated, rclone will download files from the remote and hash them locally. downloadFlag = False preserves the existing behavior of using the remote to retrieve the hash.

This commit modifies HashLister to support the new hashSum method as well as consolidating the roles of HashLister, HashListerBase64, Md5sum, and Sha1sum.  The printing of hashes from the function defined in HashLister has been revised to work with --progress.  There are light changes to operations.syncFprintf and cmd.startProgress for this.

The unit test operations_test.TestHashSums is modified to support this change and test the download functionality.

The command functions hashsum, md5sum, sha1sum, and dbhashsum are modified to support this change.  A download flag has been added and an output-file flag has been added.  The output-file flag writes hashes to a file instead of stdout to avoid the need to redirect stdout.
 2020-12-27 15:40:44 +00:00
Nick Craig-Wood 73bb9322f5 rc: prefer actual listener address if using ":port" or "addr:0" only 
Before this change rclone would turn `localhost:8888` into
`127.0.0.1:8888` which apparently does not work with some browsers.

See: https://github.com/rclone/rclone-webui-react/issues/117
 2020-12-26 16:28:54 +00:00
Nick Craig-Wood e45716cac2 mount: add "." and ".." to directories to match cmount and expectations 
See: https://forum.rclone.org/t/empty-directorys-size-for-a-mounted-crypt-remote/21077
 2020-12-17 12:14:22 +00:00
Nick Craig-Wood c98dd8755c log: fix enabling systemd logging when using --log-file 
This also moves all the systemd logging decisions to fs/log
 2020-12-17 11:55:27 +00:00
Nick Craig-Wood 86014cebd7 dedupe: add --dedupe-mode list to just list dupes, changing nothing 2020-12-02 16:52:12 +00:00
Nick Craig-Wood 507f861c67 dedupe: add --by-hash to dedupe on hash not file name - fixes #1674 2020-12-02 16:52:12 +00:00
Nick Craig-Wood 8bf4697dc2 serve http/webdav: redirect requests to the base url without the / 
When using `--baseurl` before this patch, if a request was made to the
base URL without a trailing / then rclone would return a 404 error.

Unfortunately GVFS / Nautilus makes the request without the /
regardless of what the user put in.

This patch redirects the request to the base URL with a /. So if the
user was using `--baseurl rclone` then a request to
http://localhost/rclone would be redirected with a 308 response to
http://localhost/rclone/

Fixes #4814
 2020-12-01 18:00:07 +00:00
Nick Craig-Wood f7d9b15707 cmount: don't call host.Umount if a signal has been received 
Before this change cgofuse and libatexit would race to see who could
unmount the file system with unpredicatable results. On Linux it could
report an error or not, depending.

This change checks to see if umount is beng called from a signal and
if so leaves the unmounting to cgofuse/libfuse.

See #4804
 2020-11-29 17:44:00 +00:00
Nick Craig-Wood 1cfce703b2 mountlib: make sure we don't call umount more than once #4804 
Before this change when using CTRL-C with rclone cmount the
mount would be unmounted twice.
 2020-11-29 17:44:00 +00:00
Nick Craig-Wood c22d04aa30 filter: deglobalise to put filter config into the context #4685 2020-11-27 17:28:42 +00:00
Nick Craig-Wood 354b4f19ec rc: add context to flag Reload function #4685 2020-11-27 17:28:42 +00:00
edwardxml 19a8b66cee docs: update rclone about docs 
Create a full loop of documentation for rclone about, backends overview
and individual backend pages.

Discussion:
https://github.com/rclone/rclone/pull/4774 relates

Previously pull was requested, in part, under ref
https://github.com/rclone/rclone/pull/4801

Notes:
Introduce a tentative draft see-link format the end of sections to try
rather than lots of in-para links.

Update about.go incl link to list of backends not supporting about.

In list of backends not supporting about, include link to about command
reference.

I appreciate there may be decisions to make going forward about whether
command links should be code formatted, and using proper pretty url
links, but I have fudged that for now.

Update backend pages that do not support about with wording used
previously for ftp - it is in passive voice but I can live with it. (my
own wording and fault). The note is applied to a limitations section. If
one does not already exist it is created (even if there are other
limitations with their own sections)
 2020-11-27 14:08:52 +00:00
Anagh Kumar Baranwal 07dee18d6b cmount: Add optional brew tag to throw an error when using mount in the 
binaries installed via Homebrew - Fixes #4775

Signed-off-by: Anagh Kumar Baranwal <6824881+darthShadow@users.noreply.github.com>
 2020-11-26 22:56:59 +00:00
Nick Craig-Wood 9d574c0d63 fshttp: read config from ctx not passed in ConfigInfo #4685 2020-11-26 16:40:12 +00:00
Nick Craig-Wood 2e21c58e6a fs: deglobalise the config #4685 
This is done by making fs.Config private and attaching it to the
context instead.

The Config should be obtained with fs.GetConfig and fs.AddConfig
should be used to get a new mutable config that can be changed.
 2020-11-26 16:40:12 +00:00
albertony 79474b2e4c docs: update documentation of commands delete,purge,rmdir,rmdirs 2020-11-23 17:10:08 +00:00
Deepak Sah 5b9115d87a serve ftp: add options to enable TLS - fixes #3640 2020-11-23 16:07:51 +00:00
Nick Craig-Wood f0905499e3 random: seed math/rand in one place with crypto strong seed #4783 
This shouldn't be read as encouraging the use of math/rand instead of
crypto/rand in security sensitive contexts, rather as a safer default
if that does happen by accident.
 2020-11-18 17:48:44 +00:00
Nick Craig-Wood ceeac84cfe serve restic: implement object cache 
This caches all the objects returned from the List call. This makes
opening them much quicker so speeds up prune and restores. It also
uses fewer transactions. It can be disabled with
`--cache-objects=false`.

This was discovered when using the B2 backend when the budget was
being blown on list object calls which can avoided with a bit of
caching.

For typical 1 million file backup for a latop or server this will only
use a small amount more memory.
 2020-11-12 17:58:46 +00:00
albertony 2708a7569e mount: docs: make note about mounting as network drive less confusing 2020-11-11 20:37:57 +00:00
Nick Craig-Wood e204f89685 servetest: add -sub-run flag for running a subset of the backend tests 
Use like this (eg in cmd/serve/sftp)

    go test -v -run TestSftp/Normal -sub-run "TestIntegration/FsMkdir/FsPutFiles/FsDirMove"
 2020-11-10 16:47:25 +00:00
Nick Craig-Wood f7efce594b config: add context.Context #3257 #4685 
This add config to the Config callback in the backends and the related
config functions.
 2020-11-09 18:05:54 +00:00
Nick Craig-Wood 1fb6ad700f accounting: add context.Context #3257 #4685 2020-11-09 18:05:54 +00:00
Nick Craig-Wood d846210978 fs: Add context to NewFs #3257 #4685 
This adds a context.Context parameter to NewFs and related calls.

This is necessary as part of reading config from the context -
backends need to be able to read the global config.
 2020-11-09 18:05:54 +00:00
Adam Plánský f89ff3872d ncdu: add toggle option for average size in directory - key 'a' 
Add toggle option to show average size in directory. This toggle
function is for ncdu and is binded to key 'a'.
 2020-10-30 15:33:54 +00:00
Nick Craig-Wood bfcd4113c3 mount: implement mknod to make NFS file creation work - fixes #2115 
It turns out that NFS calls mknod in FUSE even though we have create
defined. This was causing EIO errors when creating files.

This patch fixes it by implementing mknod. The way it is implemented
means that to write to an NFS file system you'll need --vfs-cache-mode
writes.
 2020-10-29 15:12:36 +00:00
Nick Craig-Wood 0e7fc7613f mount: make mount be cmount under macOS #4393 
This also adds an alias to the mount command so it responds as `rclone
cmount` as well as `rclone mount`.
 2020-10-29 13:34:39 +00:00
Nick Craig-Wood 8ac2f52b6e mount: disable bazil/fuse based mount on macOS #4393 
The library is no longer supported on macOS.
 2020-10-29 13:34:39 +00:00
Josh Soref 8bc9b2b883 docs: grammar: examples are examples 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Josh Soref a15f50254a docs: grammar: if, then 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Josh Soref 3e1cb8302a docs: spelling: etc. 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Josh Soref e4a87f772f docs: spelling: e.g. 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Josh Soref bbe7eb35f1 docs: spelling: server-side 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Adam Plánský e00bf3d723 ncdu: add sort by average size in directory 
Add keyboard shortcut 'A' which sort by average size in directory.

If files/folders have same avgSize sort by actual size

Fixes: #4699
 2020-10-27 13:28:38 +00:00
Nick Craig-Wood 00a5086ff2 Remove accidentally committed binary and fix formatting 
These were both committed in

b7253fc1c1 mount: docs: add note that allow-root and allow-other is not relevant on windows
 2020-10-26 19:07:42 +00:00
albertony b7253fc1c1 mount: docs: add note that allow-root and allow-other is not relevant on windows 2020-10-26 16:21:43 +00:00
Ingo 5164c3d2d0 genautocomplete: add support to output to stdout 2020-10-22 17:28:33 +01:00
David f4dd8e3fe8 dedupe: minor clarification in docs 
dedupe will not delete equal files if they are located in another folder.
 2020-10-14 15:31:47 +01:00
Josh Soref d0888edc0a Spelling fixes 
Fix spelling of: above, already, anonymous, associated,
authentication, bandwidth, because, between, blocks, calculate,
candidates, cautious, changelog, cleaner, clipboard, command,
completely, concurrently, considered, constructs, corrupt, current,
daemon, dependencies, deprecated, directory, dispatcher, download,
eligible, ellipsis, encrypter, endpoint, entrieslist, essentially,
existing writers, existing, expires, filesystem, flushing, frequently,
hierarchy, however, implementation, implements, inaccurate,
individually, insensitive, longer, maximum, metadata, modified,
multipart, namedirfirst, nextcloud, obscured, opened, optional,
owncloud, pacific, passphrase, password, permanently, persimmon,
positive, potato, protocol, quota, receiving, recommends, referring,
requires, revisited, satisfied, satisfies, satisfy, semver,
serialized, session, storage, strategies, stringlist, successful,
supported, surprise, temporarily, temporary, transactions, unneeded,
update, uploads, wrapped

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-14 15:21:31 +01:00
Nick Craig-Wood 72083c65ad cmd: make backend env vars show in help as the defaults for backend flags 
Before this change

    RCLONE_DRIVE_CHUNK_SIZE=111M rclone help flags | grep drive-chunk-size

Would show the default value, not the setting of RCLONE_DRIVE_CHUNK_SIZE
as the non backend flags do.

This change makes it work as expected by setting the default of the
option to the environment variable.

Fixes #4659
 2020-10-13 15:43:58 +01:00
Ameer Dawood 75a7226174 mount: docs: correction of repeated word 2020-10-07 14:25:31 +01:00
LaSombra aac2406e19 cmd: add --progress-terminal-title to print ETA to terminal title 
Adds a flag, --progress-terminal-title, that when used with --progress,
will print the string `ETA: %s` to the terminal title.

This also adds WriteTerminalTitle to lib/terminal
 2020-10-06 16:34:26 +01:00
Hekmon c58023a9ba enhance systemd integration 
* log level identification
* manual activation with flag
* automatic systemd launch detection
 2020-10-06 16:21:30 +01:00
Nick Craig-Wood 57261c7e97 mount: docs: remove incorrect statement about --vfs-cache-mode full 
See: https://forum.rclone.org/t/is-this-documentation-correct/19376
 2020-09-27 11:04:59 +01:00
Russell Cattelan b14a58c9b8 cmd/mount2: fix the swapped UID / GID values 2020-09-23 23:06:33 +01:00
Evan Harris e92294b482 docs: Updated mount command to reflect that it requires Go 1.13 or newer 2020-09-08 16:40:43 +01:00
edwardxml c3d1474eb9 docs: Add full stops for consistency in rclone --help 
closes #4560 closes #4561 closes #4562 closes #4563 closes #4564
 2020-09-08 16:26:09 +01:00
Nick Craig-Wood 8140f67092 check: fix docs 
See: https://forum.rclone.org/t/possible-issue-with-documention/18926
 2020-09-07 12:10:52 +01:00
Nick Craig-Wood 4a001b8a02 check: add back missing --download flag - fixes #4565 
This was accidentally removed when refactoring check and cryptcheck in

8b6f2bbb4b check,cryptcheck: add reporting of filenames for same/missing/changed #3264
 2020-09-05 09:29:35 +01:00
Nick Craig-Wood 70c8566cb8 fs: Pin created backends until parents are finalized 
This attempts to solve the backend lifecycle problem by

- Pinning backends mentioned on the command line into the cache
  indefinitely

- Unpinning backends when the containing structure (VFS, wrapping
  backend) is destroyed

See: https://forum.rclone.org/t/rclone-rc-backend-command-not-working-as-expected/18834
 2020-09-01 18:21:03 +01:00