Commit Graph
20 Commits
Author SHA1 Message Date
Aleksandar JankovicandNick Craig-Wood 38e70f1797 rc/jobs: add listener for finished jobs
Add jobs.OnFinish method to register listener that will trigger when job
is finished.

Includes fix for stopping listeners.
2021-01-21 13:43:41 +00:00
Aleksandar JankovicandNick Craig-Wood 708b967f15 backend/s3: fix multipart abort context
S3 couldn't abort multi-part upload when context is canceled
because canceled context prevents abort request from being sent.
2020-02-25 12:11:32 +01:00
Aleksandar JankovicandNick Craig-Wood b9fb313f71 fs/accounting: add option to delete stats
Removed PruneAllTransfers because it had no use. startedTransfers are
set to nil in ResetCounters.
2020-01-03 17:44:05 +00:00
Aleksandar JankovicandNick Craig-Wood 0e64df4b4c fs/accounting: consistency cleanup 2020-01-03 17:44:05 +00:00
Aleksandar JankovicandNick Craig-Wood 4b20afa94a backend/s3: fix ExpiryWindow value
ExpiryWindow accepts duration but it was set to value 3.
This changes it to 3 * time.Minute since default is 5 min.
2019-11-05 13:55:55 +00:00
Aleksandar JankovicandNick Craig-Wood 6b55b8b133 s3: add option for multipart failiure behaviour
This is needed for resuming uploads across different sessions.
2019-10-02 16:49:16 +01:00
Aleksandar JankovicandNick Craig-Wood 934440a9df accounting: fix total duration calculation
Fixes: #3498
2019-09-12 10:29:40 +01:00
Aleksandar JankovicandNick Craig-Wood 6a3e301303 accounting: add call to clear stats
- Make calls more consistent by changing path to kebab case.
- Add stacktrace information to job panics
2019-08-02 16:56:19 +01:00
Aleksandar JankovicandNick Craig-Wood 4ba6532915 accounting: make stats response consistent
core/stats can return two different schemas in 'transferring' field.
One is object with fields the other is just plain string.
This is confusing, unnecessary and makes defining response schema
more difficult. It also returns `lastError` as value which can be
rendered differently depending on source of error.

This change standardizes 'transferring' filed to always return
object but with reduced fields if they are not available.
Former string item is converted to {name:remote_name} object.

'lastError' is forced to be a string as in some cases it can be encoded
as an object.
2019-07-28 14:48:19 +01:00
Aleksandar JankovicandNick Craig-Wood ff235e4e56 docs: update documentation for stats 2019-07-28 14:48:19 +01:00
Aleksandar JankovicandNick Craig-Wood 68e641f6cf accounting: add limits and listing to stats groups 2019-07-28 14:48:19 +01:00
Aleksandar JankovicandNick Craig-Wood 53a1a0e3ef accounting: add reference to completed transfers
Add core/transferred call that lists completed transfers and their
status.
2019-07-28 14:48:19 +01:00
Aleksandar JankovicandNick Craig-Wood 8243ff8bc8 accounting: isolate stats to groups
Introduce stats groups that will isolate accounting for logically
different transferring operations. That way multiple accounting
operations can be done in parallel without interfering with each other
stats.

Using groups is optional. There is dedicated global stats that will be
used by default if no group is specified. This is operating mode for CLI
usage which is just fire and forget operation.

For running rclone as rc http server each request will create it's own
group. Also there is an option to specify your own group.
2019-07-28 14:48:19 +01:00
Aleksandar JankovicandNick Craig-Wood be0464f5f1 accounting: change stats interface
This is done to make clear ownership over accounting object and prepare
for removing global stats object.

Stats elapsed time calculation has been altered to account for actual
transfer time instead of stats creation time.
2019-07-28 14:48:19 +01:00
Aleksandar JankovicandNick Craig-Wood cc0800a72e march: return errors when listing dirs
Partially fixes #3172
2019-07-01 15:32:46 +01:00
Aleksandar JankovicandNick Craig-Wood 7b2b396d37 context: fix errgroup interaction with context
Fixes #3307
2019-07-01 11:51:51 +01:00
Aleksandar JankovicandNick Craig-Wood 5935cb0a29 jobs: add ability to stop async jobs
Depends on #3257
2019-06-19 14:17:41 +01:00
Aleksandar JankovicandNick Craig-Wood f78cd1e043 Add context propagation to rclone
- Change rclone/fs interfaces to accept context.Context
- Update interface implementations to use context.Context
- Change top level usage to propagate context to lover level functions

Context propagation is needed for stopping transfers and passing other
request-scoped values.
2019-06-19 11:59:46 +01:00
Aleksandar JankovicandNick Craig-Wood a2c317b46e chore: update .gitignore 2019-06-19 11:59:46 +01:00
Aleksandar JankovicandNick Craig-Wood 93207ead9c rc/jobs: make job expiry timeouts configurable 2019-06-15 09:55:32 +01:00