Commit Graph
100 Commits
Author SHA1 Message Date
albertony 4829527dac jottacloud: refactor timestamp handling
Jottacloud have several different apis and endpoints using a mix of different
timestamp formats. In existing code the list operation (after the recent liststream
implementation) uses format from golang's time.RFC3339 constant. Uploads (using the
allocate api) uses format from a hard coded constant with value identical to golang's
time.RFC3339. And then we have the classic JFS time format, which is similar to RFC3339
but not identical, using a different constant. Also the naming is a bit confusing,
since the term api is used both as a generic term and also as a reference to the
newer format used in the api subdomain where the allocate endpoint is located.
This commit refactors these things a bit.
2022-04-27 13:41:39 +02:00
albertony cc8dde402f jottacloud: refactor SetModTime function
Now using the utility function for deduplication that was newly implemented to
fix an issue with server-side copy. This function uses the original, and generic,
"jfs" api (and its "cphash" feature), instead of the newer "allocate" api dedicated
for uploads. Both apis support similar deduplication functionaly that we rely on for
the SetModTime operation. One advantage of using the jfs variant is that the allocate
api is specialized for uploads, an initial request performs modtime-only changes and
deduplication if possible but if not possible it creates an incomplete file revision
and returns a special url to be used with a following request to upload missing content.
In the SetModTime function we only sent the first request, using metadata from existing
remote file but different timestamps, which lead to a modtime-only change. If, for some
reason, this should fail it would leave the incomplete revision behind. Probably not
a problem, but the jfs implementation used with this commit is simpler and
a more "standalone" request which either succeeds or fails without expecting additional
requests.
2022-04-27 12:06:36 +02:00
albertony 2b67ad17aa jottacloud: fix issue with server-side copy when destination is in trash
A strange feature (probably bug) in the api used by the server-side copy implementation
in Jottacloud backend is that if the destination file is in trash, the copy request
succeeds but the destination will still be in trash! When this situation occurs in
rclone, the copy command will fail with "Failed to copy: object not found" because
rclone verifies that the file info in the response from the copy request is valid,
and since it is marked as deleted it is treated as invalid.

This commit works around this problem by looking for this situation in the response
from the copy operation, and send an additional request to a built-in deduplication
endpoint that will restore the file from trash.

Fixes #6112
2022-04-27 12:06:36 +02:00
albertony 6da3522499 jottacloud: minor cleanup of upload response
The UploadResponse type included several properties that are no longer returned
by Jottacloud, and the backend implementation did not use them anyway.
2022-04-27 08:40:34 +02:00
albertony 97606bbdef ncdu: refactor accumulated attributes into a struct 2022-04-26 21:12:52 +02:00
albertony a15885dd74 docs/ncdu: document flag prefixes 2022-04-26 21:12:52 +02:00
albertony 87c201c92a ncdu: fix issue where dir size is summed when file sizes are -1
Some backends may not provide size for all objects, and instead
return -1. Existing version included these in directory sums,
with strange results. With this commit rclone ncdu will consider
negative sizes as zero, but add a new prefix flag '~' with a
description that indicates the shown size is inaccurate.

Fixes #6084
2022-04-26 21:12:52 +02:00
albertony d77736c21a docs/size: extend documentation of size command 2022-04-26 19:37:15 +02:00
albertony 86bd5f6922 size: warn about inaccurate results when objects with unknown size 2022-04-26 19:37:15 +02:00
albertony fe271a4e35 docs/drive: mention that google docs count as empty files in directory totals 2022-04-26 19:34:37 +02:00
albertony b1d43f8d41 jottacloud: fix scope in token request
The existing code in rclone set the value "offline_access+openid",
when encoded in body it will become "offline_access%2Bopenid". I think
this is wrong. Probably an artifact of "double urlencoding" mixup -
either in rclone or in the jottacloud cli tool version it was sniffed
from? It does work, though. The token received will have scopes "email
offline_access" in it, and the same is true if I change to only
sending "offline_access" as scope.

If a proper space delimited list of "offline_access openid"
is used in the request, the response also includes openid scope:
"openid email offline_access". I think this is more correct and this
patch implements this.

See: #6107
2022-04-22 12:52:00 +01:00
albertony f065a267f6 docs: fix some links to command pages 2022-04-07 15:50:41 +02:00
albertony c26dc69e1b docs/jottacloud: add note that mime types are not available with --fast-list 2022-02-02 13:12:50 +01:00
albertony b0de0b4609 docs: include all commands in online help top menu drop-down 2022-02-01 20:40:50 +01:00
albertony f54641511a librclone: add support for mount commands
Fixes #5661
2022-02-01 19:29:36 +01:00
albertony 07f53aebdc touch: fix issue where directory is created instead of file
Detected on ftp, sftp and Dropbox backends.

Fixes #5952
2022-01-28 20:29:12 +01:00
albertony bd6d36b3f6 docs: improve standard list of properties for options 2022-01-28 19:43:51 +01:00
albertony 164774d7e1 Add Shmz Ozggrn to contributors 2022-01-27 09:43:42 +01:00
albertony a667e03fc9 http: improved recognition of url pointing to a single file - fixes #5929 2022-01-26 11:41:01 +01:00
albertony 1045344943 http: status string already includes the status code 2022-01-26 11:41:01 +01:00
albertony 5e469db420 docs/http: fix list layout in --http-no-head help
Existing help text ended with a list, but then auto-generated list items
Config, Env Var, Type and Default would be included in the same list.
2022-01-26 11:41:01 +01:00
albertony 946e84d194 http: use string contains instead of index 2022-01-26 11:41:01 +01:00
albertony 162aba60eb http: error strings should not be capitalized 2022-01-26 11:41:01 +01:00
albertony d8a874c32b Make http tests line ending agnostic 2022-01-26 11:41:01 +01:00
albertony 9c451d9ac6 Fix linting errors 2022-01-26 00:02:17 +01:00
albertony 8f3f24672c docs/serve: move help for template option into separate section 2022-01-25 18:19:21 +01:00
albertony 2523dd6220 version: report correct friendly-name for windows 10/11 versions after 2004
Until Windows 10 version 2004 (May 2020) this can be found from registry entry
ReleaseID, after that we must use entry DisplayVersion (ReleaseId is stuck at 2009).
Source: https://ss64.com/nt/ver.html
2022-01-24 21:27:42 +01:00
albertony c504d97017 config: fix display of config choices with empty help text 2022-01-18 20:17:57 +01:00
albertony b783f09fc6 config: show default and example values in correct input syntax instead of quoted and escaped golang string syntax
See #5551
2022-01-16 14:57:38 +01:00
albertony a301478a13 config: improved punctuation in initial config prompt 2022-01-16 14:57:38 +01:00
albertony 63b450a2a5 config: minor improvement of help text for encoding option
See #5551
2022-01-16 14:57:38 +01:00
albertony 843b77aaaa docs/ftp: improved default value description of port and username options
See #5551
2022-01-16 14:57:38 +01:00
albertony 3641727edb config: fix issue where required password options had to be re-entered when editing existing remote
See #5551
2022-01-16 14:57:38 +01:00
albertony 38e2f835ed config: fix handling of default, exclusive and required properties of multiple-choice options
Previously an empty input (just pressing enter) was only allowed for multiple-choice
options that did not have the Exclusive property set. With this change the existing
Required property is introduced into the multiple choice handling, so that one can have
Exclusive and Required options where only a value from the list is allowed, and one can
have Exclusive but not Required options where an empty value is accepted but any
non-empty value must still be matching an item from the list.

Fixes #5549

See #5551
2022-01-16 14:57:38 +01:00
albertony bd4bbed592 config: remove explicit setting of required property to true for options with a default value
See #5551
2022-01-16 14:57:38 +01:00
albertony 994b501188 config: remove explicit setting of required property to its default value false
See #5551
2022-01-16 14:57:38 +01:00
albertony dfa9381814 docs/jottacloud: correct reference to temp-dir 2022-01-16 14:34:15 +01:00
albertony 2a85feda4b docs/jottacloud: add note about upload only being supported on jotta device 2022-01-16 14:34:15 +01:00
albertony ad46af9168 docs/librclone: note that adding -ldflags -s to the build command will reduce size of library file 2022-01-16 14:32:01 +01:00
albertony 2fed02211c docs/librclone: document use from C/C++ on Windows 2022-01-16 14:11:56 +01:00
albertony 237daa8aaf dedupe: add quit as a choice in interactive mode
Fixes #5881
2022-01-14 19:57:48 +01:00
albertony 8aeca6c033 docs: align menu items when icons have different sizes 2022-01-14 17:39:27 +00:00
albertony fd82876086 librclone: allow empty string or null input instead of empty json object 2022-01-14 17:37:13 +00:00
albertony deef659aef Add Bumsu Hyeon to contributors 2022-01-13 13:25:20 +01:00
albertony 06bdf7c64c Add Lu Wang to contributors 2022-01-12 21:33:35 +01:00
albertony 871cc2f62d docs: fix links to rc sections 2022-01-12 19:51:26 +01:00
albertony b55575e622 docs: fix typo 2022-01-03 18:46:40 +01:00
albertony 328f0e7135 docs: fix links to rc debug commands 2021-12-30 21:52:34 +01:00
albertony a52814eed9 docs: fix links to rc data types section 2021-12-30 20:46:39 +01:00
albertony 071a9e882d docs: capitalization of flag usage strings 2021-12-30 14:07:24 +01:00
albertony 4e2ca3330c tree: remove obsolete --human replaced by global --human-readable - fixes #5868 2021-12-21 20:17:00 +01:00
albertony c4cb167d4a Add rsapkf and Will Holtz to contributors 2021-11-21 19:26:05 +01:00
albertony 00aafc957e sftp: add rclone to list of supported md5sum/sha1sum commands to look for
See #5781
2021-11-11 15:16:45 +01:00
albertony 29abbd2032 hashsum: support creating hash from data received on stdin
See #5781
2021-11-11 15:16:45 +01:00
albertony 25366268fe Add Atílio Antônio to contributors 2021-11-04 12:55:49 +01:00
albertony 39e2af7974 config: allow dot in remote names (#5606) 2021-11-01 20:50:06 +01:00
albertony d9c5be32e7 Add thomae to contributors 2021-11-01 15:10:14 +01:00
albertony 05321f4aef docs/sftp: more detailed explanation of pubkey file and certificate 2021-11-01 13:14:17 +00:00
albertony c9d7248d85 docs/jottacloud: update description of whitelabel services 2021-11-01 12:57:51 +00:00
albertony da8f9be84b docs: describe the --human-readable option in more detail 2021-11-01 12:55:52 +00:00
albertony 91cdaffcc1 docs: add faq section explaining why rclone changes fullwidth characters in file names 2021-11-01 12:46:23 +00:00
albertony b4944f4520 docs/librclone: document that strings are utf8 encoded 2021-11-01 12:39:00 +00:00
albertony 286b152e7b librclone: free strings in python example 2021-11-01 12:36:52 +00:00
albertony a8059b8a90 docs/mount: add note that to execute programs one must set custom filesystem permissions (#5771) 2021-10-31 00:48:56 +02:00
albertony a238877ad8 docs: note that destination is always a directory (#5766) 2021-10-30 00:30:00 +02:00
albertony 4f05ece39e test: fix touchdir test on backends without modtime support 2021-10-22 13:37:34 +02:00
albertony 9c8c0a58b5 touch: fix recursive touch due to recently introduced error ErrorIsDir 2021-10-22 13:37:34 +02:00
albertony a70c20fe6b touch: improve error message from recursive touch 2021-10-22 13:37:34 +02:00
albertony 3a03f2778c test: ignore integration test TestCopyFileMaxTransfer on Google Drive
The test fails because it expects a copy with MaxTransfer and CutoffModeHard should
return fatal error, because this is thrown from accounting (ErrorMaxTransferLimitReachedFatal),
but in case of Google Drive the external google API catches and replaces it with a
non-fatal error:

pw.CloseWithError(fmt.Errorf("googleapi: Copy failed: %v", err))

(https://github.com/googleapis/google-api-go-client/blob/7290f25351cc90fdeb492127c98cc3fb023793d4/internal/gensupport/media.go#L140)
2021-10-21 12:42:25 +01:00
albertony f589dbc077 docs: don't show fictional example values for options as multiple choice items
See #5538
2021-10-20 22:56:19 +02:00
albertony 8efead1ee6 docs: update guide for contributing documentation
See #5538
2021-10-20 22:56:19 +02:00
albertony 9a17b32b5d docs: automatically remove ending punctuation from first line of backend option help string when used for flag usage
See #5538
2021-10-20 22:56:19 +02:00
albertony 8b65c55711 cmd/config: improve option prompt
See #5538
2021-10-20 22:56:19 +02:00
albertony e2f47ecdeb docs: punctuation cleanup
See #5538
2021-10-20 22:56:19 +02:00
albertony b868561951 jottacloud: return direct download link from rclone link command
If the shared path is a directory, then the download will be a zip archive.

Fixes #5391

See #5399
2021-10-20 19:54:29 +02:00
albertony 78db3dba0e jottacloud: add support for UserInfo feature
See #5399
2021-10-20 19:54:29 +02:00
albertony 41876dd669 touch: add support for touching files in directory, with options for recursive, filtering and dry-run/interactive
Fixes #5301
2021-10-20 19:24:57 +02:00
albertony 16949fde09 Do not override mime types from os defaults
https://forum.rclone.org/t/rclone-serve-http-save-as/26672
2021-10-18 13:28:22 +01:00
albertony 29b8c71522 build: force utf8 when updating backend docs from python script (#5721) 2021-10-15 18:51:57 +02:00
albertony c2597a4fa3 docs: cleanup header levels in backend docs (#5698) 2021-10-14 15:40:18 +02:00
albertony 7d66bfbb7c docs: toc styling (#5695) 2021-10-13 15:04:11 +02:00
albertony 85074f8f88 librclone: add RcloneFreeString function
See PR #5703

Based on initial work by Weng Haoyu (@wengh) in PR #5362
2021-10-11 19:10:07 +02:00
albertony c7329d2ece docs: add section in install documentation about portable install
See #5591
2021-10-11 15:08:35 +02:00
albertony f3e71f129c config: convert --cache-dir value to an absolute path 2021-10-11 15:08:35 +02:00
albertony 0ffdca42d5 docs: document --cache-dir flag 2021-10-11 15:08:35 +02:00
albertony dbb6f94d95 config: make temporary directory user-configurable
See #5591
2021-10-11 15:08:35 +02:00
albertony 352f9bcd47 config: add paths command to show configured paths
See #5591
2021-10-11 15:08:35 +02:00
albertony 894a5a1a83 serve sftp: fix generation of server keys on windows 2021-10-11 10:43:16 +01:00
albertony ada6a92c8b serve sftp: generate an Ed25519 server key as well as ECDSA and RSA 2021-10-11 10:43:16 +01:00
albertony a5b8fcc127 docs: align dropdown items when icons have different sizes 2021-10-09 18:49:05 +03:00
albertony fbc7f2e61b lib/file: improve error message when attempting to create dir on nonexistent drive on windows
This replaces built-in os.MkdirAll with a patched version that stops the recursion
when reaching the volume part of the path. The original version would continue recursion,
and for extended length paths end up with \\? as the top-level directory, and the error
message would then be something like:
mkdir \\?: The filename, directory name, or volume label syntax is incorrect.
2021-10-01 23:18:39 +02:00
albertony 26b6c83e49 docs: extend documentation on valid remote names 2021-10-01 15:18:04 +02:00
albertony 59c74ea1b8 config: support hyphen in remote name from environment variable 2021-10-01 15:18:04 +02:00
albertony b389b84685 jottacloud: refactor all file state checks into common functions 2021-09-30 19:34:48 +02:00
albertony b0f06d9920 jottacloud: improved error handling with SetModTime and corrupt files in general 2021-09-30 19:34:48 +02:00
albertony 159229527d jottacloud: implement SetModTime to support modtime-only changes - #5627 2021-09-30 19:34:48 +02:00
albertony b5a27b1c75 docs: cleanup header levels 2021-09-30 17:54:57 +02:00
albertony db7db952c1 Add Jonta to contributors 2021-09-30 15:38:24 +02:00
albertony f1f923a986 Change byte unit format from MiByte to MiB 2021-08-31 09:57:27 +02:00
albertony 8500d95579 test: consider global option for printing human-readable sizes and avoid unsigned integer overflow 2021-08-31 09:57:27 +02:00