Commit Graph
100 Commits
Author SHA1 Message Date
albertonyandNick Craig-Wood 67f5f04a77 build: fix lint option max-issues-per-linter 2022-09-14 12:11:54 +01:00
albertony 35349657cd docs/sftp: document use of chunk_size option in sftp remote paired with serve sftp
Related to 0008cb4934
2022-08-31 00:04:04 +02:00
albertony 0008cb4934 docs: document that serve sftp uses chunk size 32 KiB
See #6404
2022-08-30 10:57:25 +02:00
albertony 2ea5b4f0b8 Add YFdyh000 to contributors 2022-08-30 10:26:03 +02:00
albertony 555def2da7 build: add package comments to silence revive linter 2022-08-28 13:43:51 +02:00
albertony 02b7613104 docs/jottacloud: improve description of the standard authentication 2022-08-28 10:31:39 +02:00
albertony b342c6cf9c docs/ftp: improve documentation of anonymous ftp - fixes #5650 2022-08-28 10:22:29 +02:00
albertony 8a6857c295 Add Simon Bos to contributors 2022-08-28 10:19:39 +02:00
albertony 21fd13f10d Add Ryan Morey to contributors 2022-08-28 10:18:33 +02:00
albertony 5cc7797f9e Add anonion to contributors 2022-08-28 10:18:14 +02:00
albertony 8bf2d6b6c8 Add João Henrique Franco to contributors 2022-08-28 10:16:37 +02:00
albertony 6b158f33a3 serve sftp: document legacy code for checksum detection
See #6351
2022-08-06 20:46:38 +02:00
albertonyandNick Craig-Wood 72227a0151 jottacloud: do not store username in config when using standard auth
Previously, with standard auth, the username would be stored in config - but only after
entering the non-standard device/mountpoint sequence during config (a feature introduced
with #5926). Regardless of that, rclone always requests the username from the api at
startup (NewFS).

In #6270 (commit 9dbed02329) this was changed to always
store username in config (consistency), and then also use it to avoid the repeated
customer info request in NewFs (performance). But, as reported in #6309, it did not work
with legacy auth, where user enters username manually, if user entered an email address
instead of the internal username required for api requests. This change was therefore
recently reverted.

The current commit takes another step back to not store the username in config during
the non-standard device/mountpoint config sequence (consistentcy). The username will
now only be stored in config when using legacy auth, where it is an input parameter.
2022-07-25 18:23:09 +01:00
albertonyandNick Craig-Wood f0396070eb sftp: fix issue with WS_FTP by working around failing RealPath 2022-07-20 18:07:50 +01:00
albertony 8c19b355a5 docs: fix links to mount command from install docs 2022-07-13 12:33:54 +02:00
albertony 5c6a958ad8 go mod tidy: github.com/pkg/xattr should be direct 2022-07-04 11:24:59 +02:00
albertony ad8c94e982 staticcheck: redundant return statement 2022-07-04 11:24:59 +02:00
albertony e5bf6a813c staticcheck: google api New is deprecated: please use NewService instead 2022-07-04 11:24:59 +02:00
albertony f18095b004 staticcheck: ignore deprecations that are not relevant 2022-07-04 11:24:59 +02:00
albertony c70e890966 staticcheck: TLS config NameToCertificate is deprecated, should instead let library select the first compatible chain from Certificates 2022-07-04 11:24:59 +02:00
albertony 986bb17656 staticcheck: awserr.BatchError is deprecated: Replaced with BatchedErrors 2022-07-04 11:24:59 +02:00
albertony 92a43c5f7b staticcheck: should use a simple channel send/receive instead of select with a single case 2022-07-04 11:24:59 +02:00
albertony 9612ca6110 staticcheck: ignore unused if platform dependent 2022-07-04 11:24:59 +02:00
albertony 1f9560e873 selfupdate: replace deprecated x/crypto/openpgp package with ProtonMail/go-crypto 2022-07-04 11:24:59 +02:00
albertony c9d67c86fb staticcheck: ignore suggestion to use context.TODO instead of nil when testing nil Context 2022-07-04 11:24:59 +02:00
albertony 3e9c5eca3b yandex: handle api error on server-side move 2022-07-04 11:24:59 +02:00
albertony a1fd60ec2b staticcheck: empty branch 2022-07-04 11:24:59 +02:00
albertony 7b8c974dec staticcheck: ineffective break statement 2022-07-04 11:24:59 +02:00
albertony 5b579cea47 staticcheck: use golang.org/x/text/cases instead of deprecated strings.Title
strings.Title has been deprecated since Go 1.18 and an alternative has been
available since Go 1.0. The rule Title uses for word boundaries does not handle
Unicode punctuation properly. Use golang.org/x/text/cases instead.
2022-07-04 11:24:59 +02:00
albertony 7822df565e staticcheck: unused func 2022-07-04 11:24:59 +02:00
albertony 3435bf7f34 staticcheck: no value of type int64 is greater than math.MaxInt64 2022-07-04 11:24:59 +02:00
albertony 0772cae314 staticcheck: use result of type assertion to simplify cases 2022-07-04 11:24:59 +02:00
albertony 9dbed02329 jottacloud: always store username in config and use it to avoid initial api request
Existing version did save username in config, but only when entering the custom
device/mountpoint sequence in config. Regardless of that, it did always look up the
username at startup with an api request.

This commit improves it so that the username will always be stored in config,
and when using standard authentication it picks it from the login token instead of
requesting it from the remote api, and also in fs constructor it picks it from config
instead of requesting it from remote api (again).
2022-07-03 12:56:25 +02:00
albertony 73e3bb09d7 http: fix missing response when using custom auth handler 2022-07-02 15:29:50 +02:00
albertony fdd2f8e6d2 Error strings should not be capitalized
Reported by staticcheck 2022.1.2 (v0.3.2)

See: staticcheck.io
2022-06-23 23:26:02 +02:00
albertony 53f831f40a docs: add missing code section formatting to commands and flags 2022-06-21 23:43:00 +02:00
albertony 9f81b4df4f docs/lsjson: fix code block indentation 2022-06-21 23:43:00 +02:00
albertony bf54c909c9 docs: improved capitalization 2022-06-21 23:43:00 +02:00
albertony dbf1234edf docs: skip "Connection" suffix from FTP, SSH/SFTP and HTTP backend names 2022-06-21 23:43:00 +02:00
albertony 70d9d75801 docs: improve serve command descriptions 2022-06-21 23:43:00 +02:00
albertony bc70a95fca docs: consistent capitalization of WebDAV, DLNA, HTTP 2022-06-21 23:43:00 +02:00
albertony ee87e919c5 docs/ncdu: note that refresh screen shortcut will fix screen corruption 2022-06-21 23:43:00 +02:00
albertony 4f0eae366f docs/ncdu: fix inconsistency in key map help text
Ctrl+L was listed as ^L, while Ctrl+c was listed as c-C. Changed the latter to ^c.
2022-06-21 23:43:00 +02:00
albertony de5ccaab8e docs: cross link doc pages for related commands 2022-06-21 23:43:00 +02:00
albertony 4b7dc35cf4 Fix sync docs incorrect merge
This copies the changes from an autogenerated section in the following commit:
https://github.com/rclone/rclone/commit/f2a15a174f8cdd21950b123f102ec9f9105098e1
2022-06-21 23:43:00 +02:00
albertony ec117593f1 Fix lint issues reported by staticcheck
Used staticcheck 2022.1.2 (v0.3.2)

See: staticcheck.io
2022-06-13 21:13:50 +02:00
albertonyandNick Craig-Wood afa30abd33 mount: remove legacy OS X remnants 2022-06-13 18:06:38 +01:00
albertonyandNick Craig-Wood 70d1d8d760 mount: replace deprecated fuse.ENOSYS with syscall.ENOSYS 2022-06-13 17:59:44 +01:00
albertonyandNick Craig-Wood 5006ede266 mailru: use variable type int instead of time.Duration for keeping number of seconds
Fixes problem reported by staticcheck lint tool (v0.3.2):
Poorly chosen name for variable of type time.Duration (ST1011).
2022-06-13 17:42:24 +01:00
albertony 5db9a2f831 sftp: use vendor-specific VFS statistics extension for about if available
See #5763
2022-06-08 21:14:53 +02:00
albertony b4091f282a sftp: add support for about and hashsum on windows server
Windows shells like cmd and powershell needs to use different quoting/escaping
of strings and paths than the unix shell, and also absolute paths must be fixed
by removing leading slash that the POSIX formatted paths have
(e.g. /C:/Users does not work in shell, it must be converted to C:/Users).

Tries to autodetect shell type (cmd, powershell, unix) on first use.

Implemented default builtin powershell functions for hashsum and about when remote
shell is powershell.

See #5763

Fixes #5758
2022-06-08 21:14:53 +02:00
albertonyandGitHub 218bf2183d docs: add missing backends from listing of optional features (Akamai, Koofr and Sia) 2022-06-08 20:47:58 +02:00
albertony acd7ad9190 config: ajust section headers to improve readability
See #6211
2022-06-08 19:24:38 +02:00
albertony b246584a02 config: more readable listing of remote options
Differentiate output of 'config show remote' command from listing options as part
of interactive config process for consistency: 'config show remote' consistent with
'config show', while listing in interactive config consistent with other output.

See #6211
2022-06-08 19:24:38 +02:00
albertony 61a75bfe07 config: add empty line between sections to improve readability
See #6211
2022-06-08 19:24:38 +02:00
albertony e3d44612c1 jottacloud: error strings should not be capitalized 2022-06-08 17:56:37 +02:00
albertony b2388f1294 jottacloud: refactor endpoint paths 2022-06-08 17:56:37 +02:00
albertony a571c1fb46 jottacloud: refactor naming of different api urls 2022-06-08 17:56:37 +02:00
albertony 01340acad2 jottacloud: add support for upload to custom device and mountpoint
See #5926
2022-06-08 17:56:37 +02:00
albertony 700ca23a71 config: add utility function for backend config with list and custom input 2022-06-08 17:56:37 +02:00
albertony f4f0e444bf filter: allow multiple --exclude-if-present flags - fixes #6219 2022-06-08 17:11:52 +02:00
albertonyandGitHub 20aaeba547 docs: clarify backend support for setting modtime only (#5638) 2022-06-08 16:29:35 +02:00
albertony 8979337313 Add Eric Wolf to contributors 2022-06-01 20:02:25 +02:00
albertony 7361c98b2d Add Mr-Kanister to contributors 2022-05-27 10:34:58 +02:00
albertonyandNick Craig-Wood 47b1a0d6fa docs: improve guide for installing from source - fixes #5809 2022-05-17 12:11:20 +01:00
albertonyandNick Craig-Wood ce168ecac2 Configurable version suffix independent of version number 2022-05-17 12:10:01 +01:00
albertony b929a56f46 Add Christian Galo to contributors 2022-05-16 13:01:55 +02:00
albertony 0e77072dcc vfs: error strings should not be capitalized 2022-05-16 12:43:43 +02:00
albertony 2437eb3cce vfs: fix incorrect detection of root in parent directory utility function
When using filepath.Dir, a difference to path.Dir is that it returns os PathSeparator
instead of slash when the path consists entirely of separators.

Also fixed casing of the function name, use OS in all caps instead of Os
as recommended here: https://github.com/golang/go/wiki/CodeReviewComments#initialisms
2022-05-16 12:43:43 +02:00
albertonyandNick Craig-Wood 8697f0bd26 about: improved error message 2022-05-13 12:08:10 +01:00
albertonyandNick Craig-Wood a9f18f8093 Set proper exit code for errors that are not low-level retried (e.g. size/timestamp changing)
Fixes #5785
2022-05-13 12:02:55 +01:00
albertonyandNick Craig-Wood a9d3283d97 jottacloud: fix listing output of remote with special characters
This fixes the failing integration test: TestIntegration/FsMkdir/FsPutFiles/FsIsFile
2022-05-12 20:41:07 +01:00
albertony 4829527dac jottacloud: refactor timestamp handling
Jottacloud have several different apis and endpoints using a mix of different
timestamp formats. In existing code the list operation (after the recent liststream
implementation) uses format from golang's time.RFC3339 constant. Uploads (using the
allocate api) uses format from a hard coded constant with value identical to golang's
time.RFC3339. And then we have the classic JFS time format, which is similar to RFC3339
but not identical, using a different constant. Also the naming is a bit confusing,
since the term api is used both as a generic term and also as a reference to the
newer format used in the api subdomain where the allocate endpoint is located.
This commit refactors these things a bit.
2022-04-27 13:41:39 +02:00
albertony cc8dde402f jottacloud: refactor SetModTime function
Now using the utility function for deduplication that was newly implemented to
fix an issue with server-side copy. This function uses the original, and generic,
"jfs" api (and its "cphash" feature), instead of the newer "allocate" api dedicated
for uploads. Both apis support similar deduplication functionaly that we rely on for
the SetModTime operation. One advantage of using the jfs variant is that the allocate
api is specialized for uploads, an initial request performs modtime-only changes and
deduplication if possible but if not possible it creates an incomplete file revision
and returns a special url to be used with a following request to upload missing content.
In the SetModTime function we only sent the first request, using metadata from existing
remote file but different timestamps, which lead to a modtime-only change. If, for some
reason, this should fail it would leave the incomplete revision behind. Probably not
a problem, but the jfs implementation used with this commit is simpler and
a more "standalone" request which either succeeds or fails without expecting additional
requests.
2022-04-27 12:06:36 +02:00
albertony 2b67ad17aa jottacloud: fix issue with server-side copy when destination is in trash
A strange feature (probably bug) in the api used by the server-side copy implementation
in Jottacloud backend is that if the destination file is in trash, the copy request
succeeds but the destination will still be in trash! When this situation occurs in
rclone, the copy command will fail with "Failed to copy: object not found" because
rclone verifies that the file info in the response from the copy request is valid,
and since it is marked as deleted it is treated as invalid.

This commit works around this problem by looking for this situation in the response
from the copy operation, and send an additional request to a built-in deduplication
endpoint that will restore the file from trash.

Fixes #6112
2022-04-27 12:06:36 +02:00
albertony 6da3522499 jottacloud: minor cleanup of upload response
The UploadResponse type included several properties that are no longer returned
by Jottacloud, and the backend implementation did not use them anyway.
2022-04-27 08:40:34 +02:00
albertony 97606bbdef ncdu: refactor accumulated attributes into a struct 2022-04-26 21:12:52 +02:00
albertony a15885dd74 docs/ncdu: document flag prefixes 2022-04-26 21:12:52 +02:00
albertony 87c201c92a ncdu: fix issue where dir size is summed when file sizes are -1
Some backends may not provide size for all objects, and instead
return -1. Existing version included these in directory sums,
with strange results. With this commit rclone ncdu will consider
negative sizes as zero, but add a new prefix flag '~' with a
description that indicates the shown size is inaccurate.

Fixes #6084
2022-04-26 21:12:52 +02:00
albertony d77736c21a docs/size: extend documentation of size command 2022-04-26 19:37:15 +02:00
albertony 86bd5f6922 size: warn about inaccurate results when objects with unknown size 2022-04-26 19:37:15 +02:00
albertony fe271a4e35 docs/drive: mention that google docs count as empty files in directory totals 2022-04-26 19:34:37 +02:00
albertonyandGitHub b1d43f8d41 jottacloud: fix scope in token request
The existing code in rclone set the value "offline_access+openid",
when encoded in body it will become "offline_access%2Bopenid". I think
this is wrong. Probably an artifact of "double urlencoding" mixup -
either in rclone or in the jottacloud cli tool version it was sniffed
from? It does work, though. The token received will have scopes "email
offline_access" in it, and the same is true if I change to only
sending "offline_access" as scope.

If a proper space delimited list of "offline_access openid"
is used in the request, the response also includes openid scope:
"openid email offline_access". I think this is more correct and this
patch implements this.

See: #6107
2022-04-22 12:52:00 +01:00
albertony f065a267f6 docs: fix some links to command pages 2022-04-07 15:50:41 +02:00
albertony c26dc69e1b docs/jottacloud: add note that mime types are not available with --fast-list 2022-02-02 13:12:50 +01:00
albertony b0de0b4609 docs: include all commands in online help top menu drop-down 2022-02-01 20:40:50 +01:00
albertony f54641511a librclone: add support for mount commands
Fixes #5661
2022-02-01 19:29:36 +01:00
albertony 07f53aebdc touch: fix issue where directory is created instead of file
Detected on ftp, sftp and Dropbox backends.

Fixes #5952
2022-01-28 20:29:12 +01:00
albertony bd6d36b3f6 docs: improve standard list of properties for options 2022-01-28 19:43:51 +01:00
albertonyandGitHub 164774d7e1 Add Shmz Ozggrn to contributors 2022-01-27 09:43:42 +01:00
albertony a667e03fc9 http: improved recognition of url pointing to a single file - fixes #5929 2022-01-26 11:41:01 +01:00
albertony 1045344943 http: status string already includes the status code 2022-01-26 11:41:01 +01:00
albertony 5e469db420 docs/http: fix list layout in --http-no-head help
Existing help text ended with a list, but then auto-generated list items
Config, Env Var, Type and Default would be included in the same list.
2022-01-26 11:41:01 +01:00
albertony 946e84d194 http: use string contains instead of index 2022-01-26 11:41:01 +01:00
albertony 162aba60eb http: error strings should not be capitalized 2022-01-26 11:41:01 +01:00
albertony d8a874c32b Make http tests line ending agnostic 2022-01-26 11:41:01 +01:00
albertony 9c451d9ac6 Fix linting errors 2022-01-26 00:02:17 +01:00
albertony 8f3f24672c docs/serve: move help for template option into separate section 2022-01-25 18:19:21 +01:00
albertony 2523dd6220 version: report correct friendly-name for windows 10/11 versions after 2004
Until Windows 10 version 2004 (May 2020) this can be found from registry entry
ReleaseID, after that we must use entry DisplayVersion (ReleaseId is stuck at 2009).
Source: https://ss64.com/nt/ver.html
2022-01-24 21:27:42 +01:00
albertony c504d97017 config: fix display of config choices with empty help text 2022-01-18 20:17:57 +01:00