f-denkena/rclone
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
9,961 Commits 1 Branch 0 Tags
8ac8d1821c87517bd8be5dd5c67976d22aa3e4db
Commit Graph
414 Commits
Author SHA1 Message Date
Nolan Woods e489a101f6 lib/http: add default 404 handler 2021-05-02 11:30:02 +01:00
Nolan Woods 002d323c94 lib/http: Move HTTP object serialization logic to lib/http 2021-04-28 22:54:15 +01:00
Nolan Woods 4ad62ec016 lib/http: Add authentication middleware with basic auth implementation 2021-04-28 22:54:15 +01:00
Nolan Woods 95ee14bb2c feat: Add lib/http 
lib/http provides an abstraction for a central http server that services can bind routes to
 2021-04-28 22:54:15 +01:00
Cnly 386acaa110 oauthutil: fix #5265 old authorize result not recognised 2021-04-23 01:20:52 +08:00
Alex Chen 31a8211afa oauthutil: raise fatal error if token expired without refresh token (#5252) 2021-04-18 12:04:13 +08:00
Nick Craig-Wood 60dcafe04d test makefiles: add --seed flag and make data generated repeatable #5214 2021-04-12 18:14:01 +01:00
albertony f2d3264054 config: prevent use of windows reserved names in config file name 2021-04-12 18:17:19 +02:00
Dominik Mydlil c163e6b250 b2: factor version handling into lib/version 
Standardizes the filename version tagging so that it can be used by any
backend.
 2021-04-12 15:59:18 +01:00
Nick Craig-Wood f52ae75a51 rclone authorize: Send and receive extra config options to fix oauth 
Before this change any backends which required extra config in the
oauth phase (like the `region` for zoho) didn't work with `rclone
authorize`.

This change serializes the extra config and passes it to `rclone
authorize` and returns new config items to be set from rclone
authorize.

`rclone authorize` will still accept its previous configuration
parameters for use with old rclones.

Fixes #5178
 2021-04-08 12:34:15 +01:00
Nick Craig-Wood b9fd02039b authorize: refactor to use new config interfaces #5178 2021-04-08 12:34:15 +01:00
Ivan Andreev 3859fe2f52 cmd/version: print os/version, kernel and bitness (#5204) 
Related to #5121

Note: OpenBSD is stub yet. This will be fixed after upstream PR gets resolved
https://github.com/shirou/gopsutil/pull/993
 2021-04-05 21:53:09 +03:00
Ivan Andreev 9eab258ffb build: add build tag noselfupdate 
Allow downstream packaging to build rclone without selfupdate command:
$ go build -tags noselfupdate

Fixes #5187
 2021-04-04 11:22:09 +03:00
Ivan Andreev 93353c431b selfupdate: dont detect FUSE if build is static 
Before this patch selfupdate detected ANY build with cmount tag as a build
having libFUSE capabilities. However, only dynamic builds really have it.
The official linux builds are static and have the cmount tag as of the time
of this writing. This results in inability to update official linux binaries.
This patch fixes that. The build can be fixed independently.
 2021-04-03 21:54:15 +03:00
Nick Craig-Wood c0c74003f2 fs/cache: add --fs-cache-expire-duration to control the fs cache 
This commit makes the previously statically configured fs cache configurable.

It introduces two parameters `--fs-cache-expire-duration` and
`--fs-cache-expire-interval` to control the caching of the items.

It also adds new interfaces to lib/cache to set these.
 2021-03-30 12:46:47 +01:00
Nick Craig-Wood fc57648b75 lib/rest: fix multipart uploads stopping on context cancel 
Before this change when the context was cancelled (due to
--max-duration for example) this could deadlock when uploading
multipart uploads.

This change fixes the problem by introducing another go routine to
monitor the context and close the pipe with an error when the context
errors.
 2021-03-29 19:09:47 +01:00
Nick Craig-Wood 8c5c91e68f lib/readers: add NewContextReader to error on context errors 2021-03-29 19:09:47 +01:00
Nick Craig-Wood cb46092883 lib/atexit: unregister interrupt handler once it has fired so users can interrupt again 2021-03-23 08:03:00 +00:00
Nick Craig-Wood a2cd5d8fa3 lib/atexit: fix deadlock calling Finalise while Run is running 2021-03-23 08:03:00 +00:00
Ivan Andreev ef5c212f9b version: show build tags and type of executable 
This patch modifies the output of `rclone version`.
The `os/arch` line is split into `os/type` and `os/arch`.
The `go version` line is now tagged as `go/version` for consistency.

Additionally the `go/linking` line tells whether the rclone
was linked as a static or dynamic executable.
The new `go/tags` line shows a space separated list of build tags.

The info about linking and build tags is also added to the output
of the `core/version` RC endpoint.
 2021-03-22 22:55:24 +03:00
Nick Craig-Wood f4750928ee lib/cache: add Delete and DeletePrefix methods #4811 2021-03-15 19:22:07 +00:00
Nick Craig-Wood 5e038a5e1e lib/file: retry preallocate on EINTR 
Before this change, sometimes preallocate failed with EINTR which
rclone ignored.

Retrying the syscall is the correct thing to do and seems to make
preallocate 100% reliable.
 2021-03-15 19:22:07 +00:00
Nick Craig-Wood b9bf91c510 lib/file: don't run preallocate concurrently 
This seems to cause file systems to get the amount of free space
wrong.
 2021-03-15 19:22:06 +00:00
Nick Craig-Wood 40b58d59ad lib/file: make pre-allocate detect disk full errors and return them 2021-03-15 19:22:06 +00:00
Nick Craig-Wood 8d847a4e94 lib/atexit: fix occasional failure to unmount with CTRL-C #4957 
Before this change CTRL-C could come in to exit rclone which would
start the atexit actions running. The Fuse unmount then signals rclone
to exit which wasn't waiting for the already running atexit actions to
complete.

This change makes sure that if the atexit actions are started they
should be completed.
 2021-03-15 19:22:06 +00:00
Nick Craig-Wood 46078d391f config: make config file reads reload the config file if needed #4996 
Before this change the config file needed to be explicitly reloaded.
This coupled the config file implementation with the backends
needlessly.

This change stats the config file to see if it needs to be reloaded on
every config file operation.

This allows us to remove calls to

- config.SaveConfig
- config.GetFresh

Which now makes the the only needed interface to the config file be
that provided by configmap.Map when rclone is not being configured.

This also adds tests for configfile
 2021-03-14 16:03:35 +00:00
Nick Craig-Wood 37e630178e dropbox: add scopes to oauth request and optionally "members.read" 
This change adds the scopes rclone wants during the oauth request.
Previously rclone left these blank to get a default set.

This allows rclone to add the "members.read" scope which is necessary
for "impersonate" to work, but only when it is in use as it require
authorisation from a Team Admin.

See: https://forum.rclone.org/t/dropbox-no-members-read/22223/3
 2021-02-13 12:35:24 +00:00
Klaus Post 5ca7f1fe87 encoder/filename: Wrap scsu package 2021-02-12 11:39:39 +00:00
Klaus Post f14220ef1e encoder/filename: Add 2 more tables and tests. 2021-02-12 11:39:39 +00:00
Klaus Post 424aaac2e1 encoder/filename: Add SCSU as tables 
Instead of only adding SCSU, add it as an existing table.

Allow direct SCSU and add a, perhaps, reasonable table as well.

Add byte interfaces that doesn't base64 encode the URL as well with `EncodeBytes` and `DecodeBytes`.

Fuzz tested and decode tests added.
 2021-02-12 11:39:39 +00:00
Nick Craig-Wood bcac8fdc83 Use http.NewRequestWithContext where possible after go1.13 minimum version 2021-02-03 17:41:27 +00:00
albertony 7a496752f3 fs: add support for flag --no-console on windows to hide the console window 2021-01-27 18:44:35 +00:00
albertony e92cb9e8f8 mount: more user friendly mounting as network drive on windows 
Add --network-mode option to activate mounting as network drive without having to set volume prefix.
Add support for automatic drive letter assignment (not specific to network drive mounting).
Allow full network share unc path in --volname, which will also implicitely activate network drive mounting.
Allow full network share unc path as mountpoint, which will also implicitely activate network drive mounting, and the specified path will be used as volume prefix and the remote will be mounted on an automatically assigned drive letter instead.
 2020-12-28 13:59:34 +00:00
Brad Ackerman 08b9ede217 azureblob: add support for managed identities 
Fixes #3213
 2020-12-28 13:23:35 +00:00
Nick Craig-Wood 83406bc473 atexit: add Signalled() function - set if called from a signal #4804 2020-11-29 17:44:00 +00:00
Nick Craig-Wood 9d574c0d63 fshttp: read config from ctx not passed in ConfigInfo #4685 2020-11-26 16:40:12 +00:00
Nick Craig-Wood 2e21c58e6a fs: deglobalise the config #4685 
This is done by making fs.Config private and attaching it to the
context instead.

The Config should be obtained with fs.GetConfig and fs.AddConfig
should be used to get a new mutable config that can be changed.
 2020-11-26 16:40:12 +00:00
Nick Craig-Wood f0905499e3 random: seed math/rand in one place with crypto strong seed #4783 
This shouldn't be read as encouraging the use of math/rand instead of
crypto/rand in security sensitive contexts, rather as a safer default
if that does happen by accident.
 2020-11-18 17:48:44 +00:00
Nick Craig-Wood 7985df3768 random: fix incorrect use of math/rand instead of crypto/rand CVE-2020-28924 
For implications see the linked issue.

Fixes #4783
 2020-11-18 12:03:01 +00:00
Nick Craig-Wood f7efce594b config: add context.Context #3257 #4685 
This add config to the Config callback in the backends and the related
config functions.
 2020-11-09 18:05:54 +00:00
Josh Soref 3e1cb8302a docs: spelling: etc. 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Josh Soref e4a87f772f docs: spelling: e.g. 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Josh Soref d0888edc0a Spelling fixes 
Fix spelling of: above, already, anonymous, associated,
authentication, bandwidth, because, between, blocks, calculate,
candidates, cautious, changelog, cleaner, clipboard, command,
completely, concurrently, considered, constructs, corrupt, current,
daemon, dependencies, deprecated, directory, dispatcher, download,
eligible, ellipsis, encrypter, endpoint, entrieslist, essentially,
existing writers, existing, expires, filesystem, flushing, frequently,
hierarchy, however, implementation, implements, inaccurate,
individually, insensitive, longer, maximum, metadata, modified,
multipart, namedirfirst, nextcloud, obscured, opened, optional,
owncloud, pacific, passphrase, password, permanently, persimmon,
positive, potato, protocol, quota, receiving, recommends, referring,
requires, revisited, satisfied, satisfies, satisfy, semver,
serialized, session, storage, strategies, stringlist, successful,
supported, surprise, temporarily, temporary, transactions, unneeded,
update, uploads, wrapped

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-14 15:21:31 +01:00
LaSombra aac2406e19 cmd: add --progress-terminal-title to print ETA to terminal title 
Adds a flag, --progress-terminal-title, that when used with --progress,
will print the string `ETA: %s` to the terminal title.

This also adds WriteTerminalTitle to lib/terminal
 2020-10-06 16:34:26 +01:00
Nick Craig-Wood 41ec712aa9 ftp,sftp: fix docs for usernames 
- factor env.CurrentUser out of backend/sftp
- Use env.CurrentUser in ftp and sftp
- fix docs to have correct username
 2020-09-27 11:44:05 +01:00
Klaus Post cb7534dcdf lib: Add file name compression 
Allows to compress short arbitrary strings and returns a string using base64 url encoding.

Generator for tables included and a few samples has been added. Add more to init.go

Tested with fuzzing for crash resistance and symmetry, see fuzz.go
 2020-08-13 16:14:11 +01:00
Nick Craig-Wood 3a14b1d5a9 build: make rclone build with wasm 
Needed to drop
- azureblob backend
- cache backend
- qingstor backend
- cachestats command
- ncdu command
 2020-08-10 17:32:21 +01:00
Tim Gallant 30eb094f28 oauthutil: adds SharedOptions for OAuth backends 
1. adds SharedOptions data structure to oauthutil
2. adds config.ConfigToken option to oauthutil.SharedOptions
3. updates the backends that have oauth functionality

Fixes #2849
 2020-08-07 16:32:01 +01:00
Nick Craig-Wood 2cc381b91d build: disable lib/plugin under gccgo to make rclone build with gccgo 2020-07-28 09:56:31 +01:00
Nick Craig-Wood d84527a730 lib/ranges: ranges libary for dealing with partially downloaded files 2020-06-30 12:01:36 +01:00