f-denkena/rclone
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Version v1.42

Browse Source
This commit is contained in:
Nick Craig-Wood
2018-06-16 18:21:09 +01:00
parent c47a4c9703
commit a9adb43896
66 changed files with 2793 additions and 337 deletions
Download Patch File Download Diff File Expand all files Collapse all files
MANUAL.html
+381 -78
View File
@@ -12,7 +12,7 @@
<div id="header">
<h1 class="title">rclone(1) User Manual</h1>
<h2 class="author">Nick Craig-Wood</h2>
<h3 class="date">Apr 28, 2018</h3>
<h3 class="date">Jun 16, 2018</h3>
</div>
<h1 id="rclone">Rclone</h1>
<p><a href="https://rclone.org/"><img src="https://rclone.org/img/rclone-120x120.png" alt="Logo" /></a></p>
@@ -37,8 +37,9 @@
<li>Microsoft Azure Blob Storage</li>
<li>Microsoft OneDrive</li>
<li>Minio</li>
<li>Nextloud</li>
<li>Nextcloud</li>
<li>OVH</li>
<li>OpenDrive</li>
<li>Openstack Swift</li>
<li>Oracle Cloud Storage</li>
<li>ownCloud</li>
@@ -156,6 +157,7 @@ sudo mv rclone /usr/local/bin/</code></pre>
<li><a href="https://rclone.org/azureblob/">Microsoft Azure Blob Storage</a></li>
<li><a href="https://rclone.org/onedrive/">Microsoft OneDrive</a></li>
<li><a href="https://rclone.org/swift/">Openstack Swift / Rackspace Cloudfiles / Memset Memstore</a></li>
<li><a href="https://rclone.org/opendrive/">OpenDrive</a></li>
<li><a href="https://rclone.org/pcloud/">Pcloud</a></li>
<li><a href="https://rclone.org/qingstor/">QingStor</a></li>
<li><a href="https://rclone.org/sftp/">SFTP</a></li>
@@ -266,10 +268,12 @@ rclone --dry-run --min-size 100M delete remote:path</code></pre>
<p>Checks the files in the source and destination match. It compares sizes and hashes (MD5 or SHA1) and logs a report of files which don't match. It doesn't alter the source or destination.</p>
<p>If you supply the --size-only flag, it will only compare the sizes not the hashes as well. Use this for a quick check.</p>
<p>If you supply the --download flag, it will download the data from both remotes and check them against each other on the fly. This can be useful for remotes that don't support hashes or if you really want to check all the data.</p>
<p>If you supply the --one-way flag, it will only check that files in source match the files in destination, not the other way around. Meaning extra files in destination that are not in the source will not trigger an error.</p>
<pre><code>rclone check source:path dest:path [flags]</code></pre>
<h3 id="options-8">Options</h3>
<pre><code> --download Check by downloading rather than with hash.
-h, --help help for check</code></pre>
-h, --help help for check
--one-way Check one way only, source files must exist on remote</code></pre>
<h2 id="rclone-ls">rclone ls</h2>
<p>List the objects in the path with size and path.</p>
<h3 id="synopsis-9">Synopsis</h3>
@@ -625,9 +629,11 @@ if src is directory
<p>You can use it like this also, but that will involve downloading all the files in remote:path.</p>
<pre><code>rclone cryptcheck remote:path encryptedremote:path</code></pre>
<p>After it has run it will log the status of the encryptedremote:.</p>
<p>If you supply the --one-way flag, it will only check that files in source match the files in destination, not the other way around. Meaning extra files in destination that are not in the source will not trigger an error.</p>
<pre><code>rclone cryptcheck remote:path cryptedremote:path [flags]</code></pre>
<h3 id="options-32">Options</h3>
<pre><code> -h, --help help for cryptcheck</code></pre>
<pre><code> -h, --help help for cryptcheck
--one-way Check one way only, source files must exist on destination</code></pre>
<h2 id="rclone-cryptdecode">rclone cryptdecode</h2>
<p>Cryptdecode returns unencrypted file names.</p>
<h3 id="synopsis-33">Synopsis</h3>
@@ -648,15 +654,22 @@ rclone cryptdecode --reverse encryptedremote: filename1 filename2</code></pre>
<pre><code>rclone dbhashsum remote:path [flags]</code></pre>
<h3 id="options-34">Options</h3>
<pre><code> -h, --help help for dbhashsum</code></pre>
<h2 id="rclone-deletefile">rclone deletefile</h2>
<p>Remove a single file path from remote.</p>
<h3 id="synopsis-35">Synopsis</h3>
<p>Remove a single file path from remote. Unlike <code>delete</code> it cannot be used to remove a directory and it doesn't obey include/exclude filters - if the specified file exists, it will always be removed.</p>
<pre><code>rclone deletefile remote:path [flags]</code></pre>
<h3 id="options-35">Options</h3>
<pre><code> -h, --help help for deletefile</code></pre>
<h2 id="rclone-genautocomplete">rclone genautocomplete</h2>
<p>Output completion script for a given shell.</p>
<h3 id="synopsis-35">Synopsis</h3>
<h3 id="synopsis-36">Synopsis</h3>
<p>Generates a shell completion script for rclone. Run with --help to list the supported shells.</p>
<h3 id="options-35">Options</h3>
<h3 id="options-36">Options</h3>
<pre><code> -h, --help help for genautocomplete</code></pre>
<h2 id="rclone-genautocomplete-bash">rclone genautocomplete bash</h2>
<p>Output bash completion script for rclone.</p>
<h3 id="synopsis-36">Synopsis</h3>
<h3 id="synopsis-37">Synopsis</h3>
<p>Generates a bash shell autocompletion script for rclone.</p>
<p>This writes to /etc/bash_completion.d/rclone by default so will probably need to be run with sudo or as root, eg</p>
<pre><code>sudo rclone genautocomplete bash</code></pre>
@@ -664,11 +677,11 @@ rclone cryptdecode --reverse encryptedremote: filename1 filename2</code></pre>
<pre><code>. /etc/bash_completion</code></pre>
<p>If you supply a command line argument the script will be written there.</p>
<pre><code>rclone genautocomplete bash [output_file] [flags]</code></pre>
<h3 id="options-36">Options</h3>
<h3 id="options-37">Options</h3>
<pre><code> -h, --help help for bash</code></pre>
<h2 id="rclone-genautocomplete-zsh">rclone genautocomplete zsh</h2>
<p>Output zsh completion script for rclone.</p>
<h3 id="synopsis-37">Synopsis</h3>
<h3 id="synopsis-38">Synopsis</h3>
<p>Generates a zsh autocompletion script for rclone.</p>
<p>This writes to /usr/share/zsh/vendor-completions/_rclone by default so will probably need to be run with sudo or as root, eg</p>
<pre><code>sudo rclone genautocomplete zsh</code></pre>
@@ -676,18 +689,18 @@ rclone cryptdecode --reverse encryptedremote: filename1 filename2</code></pre>
<pre><code>autoload -U compinit &amp;&amp; compinit</code></pre>
<p>If you supply a command line argument the script will be written there.</p>
<pre><code>rclone genautocomplete zsh [output_file] [flags]</code></pre>
<h3 id="options-37">Options</h3>
<h3 id="options-38">Options</h3>
<pre><code> -h, --help help for zsh</code></pre>
<h2 id="rclone-gendocs">rclone gendocs</h2>
<p>Output markdown docs for rclone to the directory supplied.</p>
<h3 id="synopsis-38">Synopsis</h3>
<h3 id="synopsis-39">Synopsis</h3>
<p>This produces markdown docs for the rclone commands to the directory supplied. These are in a format suitable for hugo to render into the rclone.org website.</p>
<pre><code>rclone gendocs output_directory [flags]</code></pre>
<h3 id="options-38">Options</h3>
<h3 id="options-39">Options</h3>
<pre><code> -h, --help help for gendocs</code></pre>
<h2 id="rclone-hashsum">rclone hashsum</h2>
<p>Produces an hashsum file for all the objects in the path.</p>
<h3 id="synopsis-39">Synopsis</h3>
<h3 id="synopsis-40">Synopsis</h3>
<p>Produces a hash file for all the objects in the path using the hash named. The output is in the same format as the standard md5sum/sha1sum tool.</p>
<p>Run without a hash to see the list of supported hashes, eg</p>
<pre><code>$ rclone hashsum
@@ -699,30 +712,30 @@ Supported hashes are:
<p>Then</p>
<pre><code>$ rclone hashsum MD5 remote:path</code></pre>
<pre><code>rclone hashsum &lt;hash&gt; remote:path [flags]</code></pre>
<h3 id="options-39">Options</h3>
<h3 id="options-40">Options</h3>
<pre><code> -h, --help help for hashsum</code></pre>
<h2 id="rclone-link">rclone link</h2>
<p>Generate public link to file/folder.</p>
<h3 id="synopsis-40">Synopsis</h3>
<h3 id="synopsis-41">Synopsis</h3>
<p>rclone link will create or retrieve a public link to the given file or folder.</p>
<pre><code>rclone link remote:path/to/file
rclone link remote:path/to/folder/</code></pre>
<p>If successful, the last line of the output will contain the link. Exact capabilities depend on the remote, but the link will always be created with the least constraints – e.g. no expiry, no password protection, accessible without account.</p>
<pre><code>rclone link remote:path [flags]</code></pre>
<h3 id="options-40">Options</h3>
<h3 id="options-41">Options</h3>
<pre><code> -h, --help help for link</code></pre>
<h2 id="rclone-listremotes">rclone listremotes</h2>
<p>List all the remotes in the config file.</p>
<h3 id="synopsis-41">Synopsis</h3>
<h3 id="synopsis-42">Synopsis</h3>
<p>rclone listremotes lists all the available remotes from the config file.</p>
<p>When uses with the -l flag it lists the types too.</p>
<pre><code>rclone listremotes [flags]</code></pre>
<h3 id="options-41">Options</h3>
<h3 id="options-42">Options</h3>
<pre><code> -h, --help help for listremotes
-l, --long Show the type as well as names.</code></pre>
<h2 id="rclone-lsf">rclone lsf</h2>
<p>List directories and objects in remote:path formatted for parsing</p>
<h3 id="synopsis-42">Synopsis</h3>
<h3 id="synopsis-43">Synopsis</h3>
<p>List the contents of the source path (directories and objects) to standard output in a form which is easy to parse by scripts. By default this will just be the names of the objects and directories, one per line. The directories will have a / suffix.</p>
<p>Eg</p>
<pre><code>$ rclone lsf swift:bucket
@@ -735,7 +748,9 @@ fubuwic</code></pre>
<pre><code>p - path
s - size
t - modification time
h - hash</code></pre>
h - hash
i - ID of object if known
m - MimeType of object if known</code></pre>
<p>So if you wanted the path, size and modification time, you would use --format &quot;pst&quot;, or maybe --format &quot;tsp&quot; to put the path last.</p>
<p>Eg</p>
<pre><code>$ rclone lsf --format &quot;tsp&quot; swift:bucket
@@ -763,6 +778,16 @@ cd65ac234e6fea5925974a51cdd865cc canole
2016-06-25 18:55:43,94467,03b5341b4f234b9d984d03ad076bae91,diwogej7
2018-04-26 08:52:53,0,,ferejej3gux/
2016-06-25 18:55:40,37600,8fd37c3810dd660778137ac3a66cc06d,fubuwic</code></pre>
<p>You can output in CSV standard format. This will escape things in &quot; if they contain ,</p>
<p>Eg</p>
<pre><code>$ rclone lsf --csv --files-only --format ps remote:path
test.log,22355
test.sh,449
&quot;this file contains a comma, in the file name.txt&quot;,6</code></pre>
<p>Note that the --absolute parameter is useful for making lists of files to pass to an rclone copy with the --files-from flag.</p>
<p>For example to find all the files modified within one day and copy those only (without traversing the whole directory structure):</p>
<pre><code>rclone lsf --absolute --files-only --max-age 1d /path/to/local &gt; new_files
rclone copy --files-from new_files /path/to/local remote:path</code></pre>
<p>Any of the filtering options can be applied to this commmand.</p>
<p>There are several related list commands</p>
<ul>
@@ -777,8 +802,10 @@ cd65ac234e6fea5925974a51cdd865cc canole
<p>The other list commands <code>lsd</code>,<code>lsf</code>,<code>lsjson</code> do not recurse by default - use &quot;-R&quot; to make them recurse.</p>
<p>Listing a non existent directory will produce an error except for remotes which can't have empty directories (eg s3, swift, gcs, etc - the bucket based remotes).</p>
<pre><code>rclone lsf remote:path [flags]</code></pre>
<h3 id="options-42">Options</h3>
<pre><code> -d, --dir-slash Append a slash to directory names. (default true)
<h3 id="options-43">Options</h3>
<pre><code> --absolute Put a leading / in front of path names.
--csv Output in CSV format.
-d, --dir-slash Append a slash to directory names. (default true)
--dirs-only Only list directories.
--files-only Only list files.
-F, --format string Output format - see help for details (default &quot;p&quot;)
@@ -788,10 +815,10 @@ cd65ac234e6fea5925974a51cdd865cc canole
-s, --separator string Separator for the items in the format. (default &quot;;&quot;)</code></pre>
<h2 id="rclone-lsjson">rclone lsjson</h2>
<p>List directories and objects in the path in JSON format.</p>
<h3 id="synopsis-43">Synopsis</h3>
<h3 id="synopsis-44">Synopsis</h3>
<p>List directories and objects in the path in JSON format.</p>
<p>The output is an array of Items, where each Item looks like this</p>
<p>{ &quot;Hashes&quot; : { &quot;SHA-1&quot; : &quot;f572d396fae9206628714fb2ce00f72e94f2258f&quot;, &quot;MD5&quot; : &quot;b1946ac92492d2347c6235b4d2611184&quot;, &quot;DropboxHash&quot; : &quot;ecb65bb98f9d905b70458986c39fcbad7715e5f2fcc3b1f07767d7c83e2438cc&quot; }, &quot;IsDir&quot; : false, &quot;ModTime&quot; : &quot;2017-05-31T16:15:57.034468261+01:00&quot;, &quot;Name&quot; : &quot;file.txt&quot;, &quot;Encrypted&quot; : &quot;v0qpsdq8anpci8n929v3uu9338&quot;, &quot;Path&quot; : &quot;full/path/goes/here/file.txt&quot;, &quot;Size&quot; : 6 }</p>
<p>{ &quot;Hashes&quot; : { &quot;SHA-1&quot; : &quot;f572d396fae9206628714fb2ce00f72e94f2258f&quot;, &quot;MD5&quot; : &quot;b1946ac92492d2347c6235b4d2611184&quot;, &quot;DropboxHash&quot; : &quot;ecb65bb98f9d905b70458986c39fcbad7715e5f2fcc3b1f07767d7c83e2438cc&quot; }, &quot;ID&quot;: &quot;y2djkhiujf83u33&quot;, &quot;IsDir&quot; : false, &quot;MimeType&quot; : &quot;application/octet-stream&quot;, &quot;ModTime&quot; : &quot;2017-05-31T16:15:57.034468261+01:00&quot;, &quot;Name&quot; : &quot;file.txt&quot;, &quot;Encrypted&quot; : &quot;v0qpsdq8anpci8n929v3uu9338&quot;, &quot;Path&quot; : &quot;full/path/goes/here/file.txt&quot;, &quot;Size&quot; : 6 }</p>
<p>If --hash is not specified the Hashes property won't be emitted.</p>
<p>If --no-modtime is specified then ModTime will be blank.</p>
<p>If --encrypted is not specified the Encrypted won't be emitted.</p>
@@ -812,7 +839,7 @@ cd65ac234e6fea5925974a51cdd865cc canole
<p>The other list commands <code>lsd</code>,<code>lsf</code>,<code>lsjson</code> do not recurse by default - use &quot;-R&quot; to make them recurse.</p>
<p>Listing a non existent directory will produce an error except for remotes which can't have empty directories (eg s3, swift, gcs, etc - the bucket based remotes).</p>
<pre><code>rclone lsjson remote:path [flags]</code></pre>
<h3 id="options-43">Options</h3>
<h3 id="options-44">Options</h3>
<pre><code> -M, --encrypted Show the encrypted names.
--hash Include hashes in the output (may take longer).
-h, --help help for lsjson
@@ -820,7 +847,7 @@ cd65ac234e6fea5925974a51cdd865cc canole
-R, --recursive Recurse into the listing.</code></pre>
<h2 id="rclone-mount">rclone mount</h2>
<p>Mount the remote as a mountpoint. <strong>EXPERIMENTAL</strong></p>
<h3 id="synopsis-44">Synopsis</h3>
<h3 id="synopsis-45">Synopsis</h3>
<p>rclone mount allows Linux, FreeBSD, macOS and Windows to mount any of Rclone's cloud storage systems as a file system with FUSE.</p>
<p>This is <strong>EXPERIMENTAL</strong> - use with care.</p>
<p>First set up your remote using <code>rclone config</code>. Check it works with <code>rclone ls</code> etc.</p>
@@ -858,6 +885,11 @@ umount /path/to/local/mount</code></pre>
<p>Note that all the rclone filters can be used to select a subset of the files to be visible in the mount.</p>
<h3 id="systemd">systemd</h3>
<p>When running rclone mount as a systemd service, it is possible to use Type=notify. In this case the service will enter the started state after the mountpoint has been successfully set up. Units having the rclone mount service specified as a requirement will see all files and folders immediately in this mode.</p>
<h3 id="chunked-reading">chunked reading</h3>
<p>--vfs-read-chunk-size will enable reading the source objects in parts. This can reduce the used download quota for some remotes by requesting only chunks from the remote that are actually read at the cost of an increased number of requests.</p>
<p>When --vfs-read-chunk-size-limit is also specified and greater than --vfs-read-chunk-size, the chunk size for each open file will get doubled for each chunk read, until the specified value is reached. A value of -1 will disable the limit and the chunk size will grow indefinitely.</p>
<p>With --vfs-read-chunk-size 100M and --vfs-read-chunk-size-limit 0 the following parts will be downloaded: 0-100M, 100M-200M, 200M-300M, 300M-400M and so on. When --vfs-read-chunk-size-limit 500M is specified, the result would be 0-100M, 100M-300M, 300M-700M, 700M-1200M, 1200M-1700M and so on.</p>
<p>Chunked reading will only work with --vfs-cache-mode &lt; full, as the file will always be copied to the vfs cache before opening with --vfs-cache-mode full.</p>
<h3 id="directory-cache">Directory Cache</h3>
<p>Using the <code>--dir-cache-time</code> flag, you can set how long a directory should be considered up to date and not refreshed from the backend. Changes made locally in the mount may appear immediately or invalidate the cache. However, changes done on the remote will only be picked up once the cache expires.</p>
<p>Alternatively, you can send a <code>SIGHUP</code> signal to rclone for it to flush all directory caches, regardless of how old they are. Assuming only one rclone instance is running, you can reset the cache like this:</p>
@@ -910,7 +942,7 @@ umount /path/to/local/mount</code></pre>
<p>This mode should support all normal file system operations.</p>
<p>If an upload or download fails it will be retried up to --low-level-retries times.</p>
<pre><code>rclone mount remote:path /path/to/mountpoint [flags]</code></pre>
<h3 id="options-44">Options</h3>
<h3 id="options-45">Options</h3>
<pre><code> --allow-non-empty Allow mounting over a non-empty directory.
--allow-other Allow access to other users.
--allow-root Allow access to root user.
@@ -934,10 +966,13 @@ umount /path/to/local/mount</code></pre>
--vfs-cache-max-age duration Max age of objects in the cache. (default 1h0m0s)
--vfs-cache-mode string Cache mode off|minimal|writes|full (default &quot;off&quot;)
--vfs-cache-poll-interval duration Interval to poll the cache for stale objects. (default 1m0s)
--vfs-read-chunk-size int Read the source objects in chunks.
--vfs-read-chunk-size-limit int If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached. -1 is unlimited.
--volname string Set the volume name (not supported by all OSes).
--write-back-cache Makes kernel buffer writes before sending them to rclone. Without this, writethrough caching is used.</code></pre>
<h2 id="rclone-moveto">rclone moveto</h2>
<p>Move file or directory from source to dest.</p>
<h3 id="synopsis-45">Synopsis</h3>
<h3 id="synopsis-46">Synopsis</h3>
<p>If source:path is a file or directory then it moves it to a file or directory named dest:path.</p>
<p>This can be used to rename files or upload single files to other than their existing name. If the source is a directory then it acts exacty like the move command.</p>
<p>So</p>
@@ -952,11 +987,11 @@ if src is directory
<p>This doesn't transfer unchanged files, testing by size and modification time or MD5SUM. src will be deleted on successful transfer.</p>
<p><strong>Important</strong>: Since this can cause data loss, test first with the --dry-run flag.</p>
<pre><code>rclone moveto source:path dest:path [flags]</code></pre>
<h3 id="options-45">Options</h3>
<h3 id="options-46">Options</h3>
<pre><code> -h, --help help for moveto</code></pre>
<h2 id="rclone-ncdu">rclone ncdu</h2>
<p>Explore a remote with a text based user interface.</p>
<h3 id="synopsis-46">Synopsis</h3>
<h3 id="synopsis-47">Synopsis</h3>
<p>This displays a text based user interface allowing the navigation of a remote. It is most useful for answering the question - &quot;What is using all my disk space?&quot;.</p>
<script src="https://asciinema.org/a/157793.js" id="asciicast-157793" async></script>
<p>To make the user interface it first scans the entire remote given and builds an in memory representation. rclone ncdu can be used during this scanning phase and you will see it building up the directory structure as it goes along.</p>
@@ -972,61 +1007,61 @@ if src is directory
q/ESC/c-C to quit</code></pre>
<p>This an homage to the <a href="https://dev.yorhel.nl/ncdu">ncdu tool</a> but for rclone remotes. It is missing lots of features at the moment, most importantly deleting files, but is useful as it stands.</p>
<pre><code>rclone ncdu remote:path [flags]</code></pre>
<h3 id="options-46">Options</h3>
<h3 id="options-47">Options</h3>
<pre><code> -h, --help help for ncdu</code></pre>
<h2 id="rclone-obscure">rclone obscure</h2>
<p>Obscure password for use in the rclone.conf</p>
<h3 id="synopsis-47">Synopsis</h3>
<h3 id="synopsis-48">Synopsis</h3>
<p>Obscure password for use in the rclone.conf</p>
<pre><code>rclone obscure password [flags]</code></pre>
<h3 id="options-47">Options</h3>
<h3 id="options-48">Options</h3>
<pre><code> -h, --help help for obscure</code></pre>
<h2 id="rclone-rc">rclone rc</h2>
<p>Run a command against a running rclone.</p>
<h3 id="synopsis-48">Synopsis</h3>
<h3 id="synopsis-49">Synopsis</h3>
<p>This runs a command against a running rclone. By default it will use that specified in the --rc-addr command.</p>
<p>Arguments should be passed in as parameter=value.</p>
<p>The result will be returned as a JSON object by default.</p>
<p>Use &quot;rclone rc list&quot; to see a list of all possible commands.</p>
<p>Use &quot;rclone rc&quot; to see a list of all possible commands.</p>
<pre><code>rclone rc commands parameter [flags]</code></pre>
<h3 id="options-48">Options</h3>
<h3 id="options-49">Options</h3>
<pre><code> -h, --help help for rc
--no-output If set don&#39;t output the JSON result.
--url string URL to connect to rclone remote control. (default &quot;http://localhost:5572/&quot;)</code></pre>
<h2 id="rclone-rcat">rclone rcat</h2>
<p>Copies standard input to file on remote.</p>
<h3 id="synopsis-49">Synopsis</h3>
<h3 id="synopsis-50">Synopsis</h3>
<p>rclone rcat reads from standard input (stdin) and copies it to a single remote file.</p>
<pre><code>echo &quot;hello world&quot; | rclone rcat remote:path/to/file
ffmpeg - | rclone rcat --checksum remote:path/to/file</code></pre>
ffmpeg - | rclone rcat remote:path/to/file</code></pre>
<p>If the remote file already exists, it will be overwritten.</p>
<p>rcat will try to upload small files in a single request, which is usually more efficient than the streaming/chunked upload endpoints, which use multiple requests. Exact behaviour depends on the remote. What is considered a small file may be set through <code>--streaming-upload-cutoff</code>. Uploading only starts after the cutoff is reached or if the file ends before that. The data must fit into RAM. The cutoff needs to be small enough to adhere the limits of your remote, please see there. Generally speaking, setting this cutoff too high will decrease your performance.</p>
<p>Note that the upload can also not be retried because the data is not kept around until the upload succeeds. If you need to transfer a lot of data, you're better off caching locally and then <code>rclone move</code> it to the destination.</p>
<pre><code>rclone rcat remote:path [flags]</code></pre>
<h3 id="options-49">Options</h3>
<h3 id="options-50">Options</h3>
<pre><code> -h, --help help for rcat</code></pre>
<h2 id="rclone-rmdirs">rclone rmdirs</h2>
<p>Remove empty directories under the path.</p>
<h3 id="synopsis-50">Synopsis</h3>
<h3 id="synopsis-51">Synopsis</h3>
<p>This removes any empty directories (or directories that only contain empty directories) under the path that it finds, including the path if it has nothing in.</p>
<p>If you supply the --leave-root flag, it will not remove the root directory.</p>
<p>This is useful for tidying up remotes that rclone has left a lot of empty directories in.</p>
<pre><code>rclone rmdirs remote:path [flags]</code></pre>
<h3 id="options-50">Options</h3>
<h3 id="options-51">Options</h3>
<pre><code> -h, --help help for rmdirs
--leave-root Do not remove root directory if empty</code></pre>
<h2 id="rclone-serve">rclone serve</h2>
<p>Serve a remote over a protocol.</p>
<h3 id="synopsis-51">Synopsis</h3>
<h3 id="synopsis-52">Synopsis</h3>
<p>rclone serve is used to serve a remote over a given protocol. This command requires the use of a subcommand to specify the protocol, eg</p>
<pre><code>rclone serve http remote:</code></pre>
<p>Each subcommand has its own options which you can see in their help.</p>
<pre><code>rclone serve &lt;protocol&gt; [opts] &lt;remote&gt; [flags]</code></pre>
<h3 id="options-51">Options</h3>
<h3 id="options-52">Options</h3>
<pre><code> -h, --help help for serve</code></pre>
<h2 id="rclone-serve-http">rclone serve http</h2>
<p>Serve the remote over HTTP.</p>
<h3 id="synopsis-52">Synopsis</h3>
<h3 id="synopsis-53">Synopsis</h3>
<p>rclone serve http implements a basic web server to serve the remote over HTTP. This can be viewed in a web browser or you can make a remote of type http read from it.</p>
<p>You can use the filter flags (eg --include, --exclude) to control what is served.</p>
<p>The server will log errors. Use -v to see access logs.</p>
@@ -1101,7 +1136,7 @@ htpasswd -B htpasswd anotherUser</code></pre>
<p>This mode should support all normal file system operations.</p>
<p>If an upload or download fails it will be retried up to --low-level-retries times.</p>
<pre><code>rclone serve http remote:path [flags]</code></pre>
<h3 id="options-52">Options</h3>
<h3 id="options-53">Options</h3>
<pre><code> --addr string IPaddress:Port or :Port to bind server to. (default &quot;localhost:8080&quot;)
--cert string SSL PEM key (concatenation of certificate and CA certificate)
--client-ca string Client certificate authority to verify clients with
@@ -1125,10 +1160,12 @@ htpasswd -B htpasswd anotherUser</code></pre>
--user string User name for authentication.
--vfs-cache-max-age duration Max age of objects in the cache. (default 1h0m0s)
--vfs-cache-mode string Cache mode off|minimal|writes|full (default &quot;off&quot;)
--vfs-cache-poll-interval duration Interval to poll the cache for stale objects. (default 1m0s)</code></pre>
--vfs-cache-poll-interval duration Interval to poll the cache for stale objects. (default 1m0s)
--vfs-read-chunk-size int Read the source objects in chunks.
--vfs-read-chunk-size-limit int If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached. -1 is unlimited.</code></pre>
<h2 id="rclone-serve-restic">rclone serve restic</h2>
<p>Serve the remote for restic's REST API.</p>
<h3 id="synopsis-53">Synopsis</h3>
<h3 id="synopsis-54">Synopsis</h3>
<p>rclone serve restic implements restic's REST backend API over HTTP. This allows restic to use rclone as a data storage mechanism for cloud providers that restic does not support directly.</p>
<p><a href="https://restic.net/">Restic</a> is a command line program for doing backups.</p>
<p>The server will log errors. Use -v to see access logs.</p>
@@ -1185,7 +1222,7 @@ htpasswd -B htpasswd anotherUser</code></pre>
<p>By default this will serve over http. If you want you can serve over https. You will need to supply the --cert and --key flags. If you wish to do client side certificate validation then you will need to supply --client-ca also.</p>
<p>--cert should be a either a PEM encoded certificate or a concatenation of that with the CA certificate. --key should be the PEM encoded private key and --client-ca should be the PEM encoded client certificate authority certificate.</p>
<pre><code>rclone serve restic remote:path [flags]</code></pre>
<h3 id="options-53">Options</h3>
<h3 id="options-54">Options</h3>
<pre><code> --addr string IPaddress:Port or :Port to bind server to. (default &quot;localhost:8080&quot;)
--append-only disallow deletion of repository data
--cert string SSL PEM key (concatenation of certificate and CA certificate)
@@ -1202,7 +1239,7 @@ htpasswd -B htpasswd anotherUser</code></pre>
--user string User name for authentication.</code></pre>
<h2 id="rclone-serve-webdav">rclone serve webdav</h2>
<p>Serve remote:path over webdav.</p>
<h3 id="synopsis-54">Synopsis</h3>
<h3 id="synopsis-55">Synopsis</h3>
<p>rclone serve webdav implements a basic webdav server to serve the remote over HTTP via the webdav protocol. This can be viewed with a webdav client or you can make a remote of type webdav to read and write it.</p>
<p>NB at the moment each directory listing reads the start of each file which is undesirable: see https://github.com/golang/go/issues/22577</p>
<h3 id="server-options-2">Server options</h3>
@@ -1275,7 +1312,7 @@ htpasswd -B htpasswd anotherUser</code></pre>
<p>This mode should support all normal file system operations.</p>
<p>If an upload or download fails it will be retried up to --low-level-retries times.</p>
<pre><code>rclone serve webdav remote:path [flags]</code></pre>
<h3 id="options-54">Options</h3>
<h3 id="options-55">Options</h3>
<pre><code> --addr string IPaddress:Port or :Port to bind server to. (default &quot;localhost:8080&quot;)
--cert string SSL PEM key (concatenation of certificate and CA certificate)
--client-ca string Client certificate authority to verify clients with
@@ -1299,19 +1336,21 @@ htpasswd -B htpasswd anotherUser</code></pre>
--user string User name for authentication.
--vfs-cache-max-age duration Max age of objects in the cache. (default 1h0m0s)
--vfs-cache-mode string Cache mode off|minimal|writes|full (default &quot;off&quot;)
--vfs-cache-poll-interval duration Interval to poll the cache for stale objects. (default 1m0s)</code></pre>
--vfs-cache-poll-interval duration Interval to poll the cache for stale objects. (default 1m0s)
--vfs-read-chunk-size int Read the source objects in chunks.
--vfs-read-chunk-size-limit int If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached. -1 is unlimited.</code></pre>
<h2 id="rclone-touch">rclone touch</h2>
<p>Create new file or change file modification time.</p>
<h3 id="synopsis-55">Synopsis</h3>
<h3 id="synopsis-56">Synopsis</h3>
<p>Create new file or change file modification time.</p>
<pre><code>rclone touch remote:path [flags]</code></pre>
<h3 id="options-55">Options</h3>
<h3 id="options-56">Options</h3>
<pre><code> -h, --help help for touch
-C, --no-create Do not create the file if it does not exist.
-t, --timestamp string Change the modification times to the specified time instead of the current time of day. The argument is of the form &#39;YYMMDD&#39; (ex. 17.10.30) or &#39;YYYY-MM-DDTHH:MM:SS&#39; (ex. 2006-01-02T15:04:05)</code></pre>
<h2 id="rclone-tree">rclone tree</h2>
<p>List the contents of the remote in a tree like fashion.</p>
<h3 id="synopsis-56">Synopsis</h3>
<h3 id="synopsis-57">Synopsis</h3>
<p>rclone tree lists the contents of a remote in a similar way to the unix tree command.</p>
<p>For example</p>
<pre><code>$ rclone tree remote:path
@@ -1327,7 +1366,7 @@ htpasswd -B htpasswd anotherUser</code></pre>
<p>You can use any of the filtering options with the tree command (eg --include and --exclude). You can also use --fast-list.</p>
<p>The tree command has many options for controlling the listing which are compatible with the tree command. Note that not all of them have short options as they conflict with rclone's short options.</p>
<pre><code>rclone tree remote:path [flags]</code></pre>
<h3 id="options-56">Options</h3>
<h3 id="options-57">Options</h3>
<pre><code> -a, --all All files are listed (list . files too).
-C, --color Turn colorization on always.
-d, --dirs-only List directories only.
@@ -1391,7 +1430,7 @@ htpasswd -B htpasswd anotherUser</code></pre>
<p>This can be used when scripting to make aged backups efficiently, eg</p>
<pre><code>rclone sync remote:current-backup remote:previous-backup
rclone sync /path/to/files remote:current-backup</code></pre>
<h2 id="options-57">Options</h2>
<h2 id="options-58">Options</h2>
<p>Rclone has a number of options to control its behaviour.</p>
<p>Options which use TIME use the go time parser. A duration string is a possibly signed sequence of decimal numbers, each with optional fraction and a unit suffix, such as &quot;300ms&quot;, &quot;-1.5h&quot; or &quot;2h45m&quot;. Valid time units are &quot;ns&quot;, &quot;us&quot; (or &quot;µs&quot;), &quot;ms&quot;, &quot;s&quot;, &quot;m&quot;, &quot;h&quot;.</p>
<p>Options which use SIZE use kByte by default. However, a suffix of <code>b</code> for bytes, <code>k</code> for kBytes, <code>M</code> for MBytes, <code>G</code> for GBytes, <code>T</code> for TBytes and <code>P</code> for PBytes may be used. These are the binary units, eg 1, 2**10, 2**20, 2**30 respectively.</p>
@@ -1493,6 +1532,10 @@ rclone sync /path/to/files remote:current-backup</code></pre>
<p>For historical reasons the <code>lsd</code> command defaults to using a <code>--max-depth</code> of 1 - you can override this with the command line flag.</p>
<p>You can use this command to disable recursion (with <code>--max-depth 1</code>).</p>
<p>Note that if you use this with <code>sync</code> and <code>--delete-excluded</code> the files not recursed through are considered excluded and will be deleted on the destination. Test first with <code>--dry-run</code> if you are not sure what will happen.</p>
<h3 id="max-transfersize">--max-transfer=SIZE</h3>
<p>Rclone will stop transferring when it has reached the size specified. Defaults to off.</p>
<p>When the limit is reached all transfers will stop immediately.</p>
<p>Rclone will exit with exit code 8 if the transfer limit is reached.</p>
<h3 id="modify-windowtime">--modify-window=TIME</h3>
<p>When checking whether a file has been modified, this is the maximum allowed time difference that a file can have and still be considered equivalent.</p>
<p>The default is <code>1ns</code> unless this is overridden by a remote. For example OS X only stores modification times to the nearest second so if you are reading and writing to an OS X filing system this will be <code>1s</code> by default.</p>
@@ -1509,6 +1552,9 @@ rclone sync /path/to/files remote:current-backup</code></pre>
<p>Retry the entire sync if it fails this many times it fails (default 3).</p>
<p>Some remotes can be unreliable and a few retries help pick up the files which didn't get transferred because of errors.</p>
<p>Disable retries with <code>--retries 1</code>.</p>
<h3 id="retries-sleeptime">--retries-sleep=TIME</h3>
<p>This sets the interval between each retry specified by <code>--retries</code></p>
<p>The default is 0. Use 0 to disable.</p>
<h3 id="size-only">--size-only</h3>
<p>Normally rclone will look at modification time and size of files to see if they are equal. If you set this flag then rclone will check only the size.</p>
<p>This can be useful transferring files from Dropbox which have been modified by the desktop sync client which doesn't set checksums of modification times in the same way as rclone.</p>
@@ -1518,6 +1564,7 @@ rclone sync /path/to/files remote:current-backup</code></pre>
<p>The default is <code>1m</code>. Use 0 to disable.</p>
<p>If you set the stats interval then all commands can show stats. This can be useful when running other commands, <code>check</code> or <code>mount</code> for example.</p>
<p>Stats are logged at <code>INFO</code> level by default which means they won't show at default log level <code>NOTICE</code>. Use <code>--stats-log-level NOTICE</code> or <code>-v</code> to make them show. See the <a href="#logging">Logging section</a> for more info on log levels.</p>
<p>Note that on macOS you can send a SIGINFO (which is normally ctrl-T in the terminal) to make the stats print immediately.</p>
<h3 id="stats-file-name-length-integer">--stats-file-name-length integer</h3>
<p>By default, the <code>--stats</code> output will truncate file names and paths longer than 40 characters. This is equivalent to providing <code>--stats-file-name-length 40</code>. Use <code>--stats-file-name-length 0</code> to disable any truncation of file names printed by stats.</p>
<h3 id="stats-log-level-string">--stats-log-level string</h3>
@@ -1713,10 +1760,11 @@ export RCLONE_CONFIG_PASS</code></pre>
<li><code>5</code> - Temporary error (one that more retries might fix) (Retry errors)</li>
<li><code>6</code> - Less serious errors (like 461 errors from dropbox) (NoRetry errors)</li>
<li><code>7</code> - Fatal error (one that more retries won't fix, like account suspended) (Fatal errors)</li>
<li><code>8</code> - Transfer exceeded - limit set by --max-transfer reached</li>
</ul>
<h2 id="environment-variables">Environment Variables</h2>
<p>Rclone can be configured entirely using environment variables. These can be used to set defaults for options or config file entries.</p>
<h3 id="options-58">Options</h3>
<h3 id="options-59">Options</h3>
<p>Every option in rclone can have its default set by environment variable.</p>
<p>To find the name of the environment variable, first, take the long option name, strip the leading <code>--</code>, change <code>-</code> to <code>_</code>, make upper case and prepend <code>RCLONE_</code>.</p>
<p>For example, to always set <code>--stats 5s</code>, set the environment variable <code>RCLONE_STATS=5s</code>. If you set stats on the command line this will override the environment variable setting.</p>
@@ -2077,6 +2125,8 @@ rclone rc cache/expire remote=/ withData=true</code></pre>
<pre><code>rclone rc core/bwlimit rate=1M
rclone rc core/bwlimit rate=off</code></pre>
<p>The format of the parameter is exactly the same as passed to --bwlimit except only one bandwidth may be specified.</p>
<h3 id="coregc-runs-a-garbage-collection.">core/gc: Runs a garbage collection.</h3>
<p>This tells the go runtime to do a garbage collection run. It isn't necessary to call this normally, but it can be useful for debugging memory problems.</p>
<h3 id="corememstats-returns-the-memory-statistics">core/memstats: Returns the memory statistics</h3>
<p>This returns the memory statistics of the running program. What the values mean are explained in the go docs: https://golang.org/pkg/runtime/#MemStats</p>
<p>The most interesting values for most people are:</p>
@@ -2159,6 +2209,36 @@ $ echo $?
&quot;rutabaga&quot;: &quot;3&quot;,
&quot;sausage&quot;: 1
}</code></pre>
<h2 id="debugging-rclone-with-pprof">Debugging rclone with pprof</h2>
<p>If you use the <code>--rc</code> flag this will also enable the use of the go profiling tools on the same port.</p>
<p>To use these, first <a href="https://golang.org/doc/install">install go</a>.</p>
<p>Then (for example) to profile rclone's memory use you can run:</p>
<pre><code>go tool pprof -web http://localhost:5572/debug/pprof/heap</code></pre>
<p>This should open a page in your browser showing what is using what memory.</p>
<p>You can also use the <code>-text</code> flag to produce a textual summary</p>
<pre><code>$ go tool pprof -text http://localhost:5572/debug/pprof/heap
Showing nodes accounting for 1537.03kB, 100% of 1537.03kB total
flat flat% sum% cum cum%
1024.03kB 66.62% 66.62% 1024.03kB 66.62% github.com/ncw/rclone/vendor/golang.org/x/net/http2/hpack.addDecoderNode
513kB 33.38% 100% 513kB 33.38% net/http.newBufioWriterSize
0 0% 100% 1024.03kB 66.62% github.com/ncw/rclone/cmd/all.init
0 0% 100% 1024.03kB 66.62% github.com/ncw/rclone/cmd/serve.init
0 0% 100% 1024.03kB 66.62% github.com/ncw/rclone/cmd/serve/restic.init
0 0% 100% 1024.03kB 66.62% github.com/ncw/rclone/vendor/golang.org/x/net/http2.init
0 0% 100% 1024.03kB 66.62% github.com/ncw/rclone/vendor/golang.org/x/net/http2/hpack.init
0 0% 100% 1024.03kB 66.62% github.com/ncw/rclone/vendor/golang.org/x/net/http2/hpack.init.0
0 0% 100% 1024.03kB 66.62% main.init
0 0% 100% 513kB 33.38% net/http.(*conn).readRequest
0 0% 100% 513kB 33.38% net/http.(*conn).serve
0 0% 100% 1024.03kB 66.62% runtime.main</code></pre>
<p>Possible profiles to look at:</p>
<ul>
<li>Memory: <code>go tool pprof http://localhost:5572/debug/pprof/heap</code></li>
<li>30-second CPU profile: <code>go tool pprof http://localhost:5572/debug/pprof/profile</code></li>
<li>5-second execution trace: <code>wget http://localhost:5572/debug/pprof/trace?seconds=5</code></li>
</ul>
<p>See the <a href="https://golang.org/pkg/net/http/pprof/">net/http/pprof docs</a> for more info on how to use the profiling and for a general overview see <a href="https://blog.golang.org/profiling-go-programs">the Go team's blog post on profiling go programs</a>.</p>
<p>The profiling hook is <a href="https://stackoverflow.com/q/26545159/164234">zero overhead unless it is used</a>.</p>
<h1 id="overview-of-cloud-storage-systems">Overview of cloud storage systems</h1>
<p>Each cloud storage system is slightly different. Rclone attempts to provide a unified interface to them, but some underlying differences show through.</p>
<h2 id="features">Features</h2>
@@ -2280,6 +2360,14 @@ $ echo $?
<td align="center">R</td>
</tr>
<tr class="even">
<td>OpenDrive</td>
<td align="center">MD5</td>
<td align="center">Yes</td>
<td align="center">Yes</td>
<td align="center">No</td>
<td align="center">-</td>
</tr>
<tr class="odd">
<td>Openstack Swift</td>
<td align="center">MD5</td>
<td align="center">Yes</td>
@@ -2287,7 +2375,7 @@ $ echo $?
<td align="center">No</td>
<td align="center">R/W</td>
</tr>
<tr class="odd">
<tr class="even">
<td>pCloud</td>
<td align="center">MD5, SHA1</td>
<td align="center">Yes</td>
@@ -2295,7 +2383,7 @@ $ echo $?
<td align="center">No</td>
<td align="center">W</td>
</tr>
<tr class="even">
<tr class="odd">
<td>QingStor</td>
<td align="center">MD5</td>
<td align="center">No</td>
@@ -2303,7 +2391,7 @@ $ echo $?
<td align="center">No</td>
<td align="center">R/W</td>
</tr>
<tr class="odd">
<tr class="even">
<td>SFTP</td>
<td align="center">MD5, SHA1 ‡</td>
<td align="center">Yes</td>
@@ -2311,7 +2399,7 @@ $ echo $?
<td align="center">No</td>
<td align="center">-</td>
</tr>
<tr class="even">
<tr class="odd">
<td>WebDAV</td>
<td align="center">-</td>
<td align="center">Yes ††</td>
@@ -2319,7 +2407,7 @@ $ echo $?
<td align="center">No</td>
<td align="center">-</td>
</tr>
<tr class="odd">
<tr class="even">
<td>Yandex Disk</td>
<td align="center">MD5</td>
<td align="center">Yes</td>
@@ -2327,7 +2415,7 @@ $ echo $?
<td align="center">No</td>
<td align="center">R/W</td>
</tr>
<tr class="even">
<tr class="odd">
<td>The local filesystem</td>
<td align="center">All</td>
<td align="center">Yes</td>
@@ -2540,6 +2628,18 @@ $ echo $?
<td align="center">Yes</td>
</tr>
<tr class="even">
<td>OpenDrive</td>
<td align="center">Yes</td>
<td align="center">Yes</td>
<td align="center">Yes</td>
<td align="center">Yes</td>
<td align="center">No</td>
<td align="center">No</td>
<td align="center">No</td>
<td align="center">No</td>
<td align="center">No</td>
</tr>
<tr class="odd">
<td>Openstack Swift</td>
<td align="center">Yes †</td>
<td align="center">Yes</td>
@@ -2551,7 +2651,7 @@ $ echo $?
<td align="center">No <a href="https://github.com/ncw/rclone/issues/2178">#2178</a></td>
<td align="center">Yes</td>
</tr>
<tr class="odd">
<tr class="even">
<td>pCloud</td>
<td align="center">Yes</td>
<td align="center">Yes</td>
@@ -2563,7 +2663,7 @@ $ echo $?
<td align="center">No <a href="https://github.com/ncw/rclone/issues/2178">#2178</a></td>
<td align="center">Yes</td>
</tr>
<tr class="even">
<tr class="odd">
<td>QingStor</td>
<td align="center">No</td>
<td align="center">Yes</td>
@@ -2575,7 +2675,7 @@ $ echo $?
<td align="center">No <a href="https://github.com/ncw/rclone/issues/2178">#2178</a></td>
<td align="center">No</td>
</tr>
<tr class="odd">
<tr class="even">
<td>SFTP</td>
<td align="center">No</td>
<td align="center">No</td>
@@ -2587,7 +2687,7 @@ $ echo $?
<td align="center">No <a href="https://github.com/ncw/rclone/issues/2178">#2178</a></td>
<td align="center">No</td>
</tr>
<tr class="even">
<tr class="odd">
<td>WebDAV</td>
<td align="center">Yes</td>
<td align="center">Yes</td>
@@ -2599,7 +2699,7 @@ $ echo $?
<td align="center">No <a href="https://github.com/ncw/rclone/issues/2178">#2178</a></td>
<td align="center">No</td>
</tr>
<tr class="odd">
<tr class="even">
<td>Yandex Disk</td>
<td align="center">Yes</td>
<td align="center">No</td>
@@ -2611,7 +2711,7 @@ $ echo $?
<td align="center">No <a href="https://github.com/ncw/rclone/issues/2178">#2178</a></td>
<td align="center">No</td>
</tr>
<tr class="even">
<tr class="odd">
<td>The local filesystem</td>
<td align="center">Yes</td>
<td align="center">No</td>
@@ -3172,6 +3272,9 @@ y/e/d&gt; </code></pre>
<p>Any files larger than this will be uploaded in chunks of this size. The default is 5MB. The minimum is 5MB.</p>
<p>Note that 2 chunks of this size are buffered in memory per transfer.</p>
<p>If you are transferring large files over high speed links and you have enough memory, then increasing this will speed up the transfers.</p>
<h4 id="s3-upload-concurrency">--s3-upload-concurrency</h4>
<p>Number of chunks of the same file that are uploaded concurrently. Default is 2.</p>
<p>If you are uploading small amount of large file over high speed link and these uploads do not fully utilize your bandwidth, then increasing this may help to speed up the transfers.</p>
<h3 id="anonymous-access-to-public-buckets">Anonymous access to public buckets</h3>
<p>If you want to use rclone to access a public bucket, configure with a blank <code>access_key_id</code> and <code>secret_access_key</code>. Your config should end up looking like this:</p>
<pre><code>[anons3]
@@ -4394,6 +4497,12 @@ y/e/d&gt; y</code></pre>
<pre><code>rclone ls remote:</code></pre>
<p>To copy a local directory to a dropbox directory called backup</p>
<pre><code>rclone copy /home/source remote:backup</code></pre>
<h3 id="dropbox-for-business">Dropbox for business</h3>
<p>Rclone supports Dropbox for business and Team Folders.</p>
<p>When using Dropbox for business <code>remote:</code> and <code>remote:path/to/file</code> will refer to your personal folder.</p>
<p>If you wish to see Team Folders you must use a leading <code>/</code> in the path, so <code>rclone lsd remote:/</code> will refer to the root and show you all Team Folders and your User Folder.</p>
<p>You can then use team folders like this <code>remote:/TeamFolder</code> and <code>remote:/TeamFolder/path/to/file</code>.</p>
<p>A leading <code>/</code> for a Dropbox personal account will do nothing, but it will take an extra HTTP transaction so it should be avoided.</p>
<h3 id="modified-time-and-hashes-2">Modified time and Hashes</h3>
<p>Dropbox supports modified times, but the only way to set a modification time is to re-upload the file.</p>
<p>This means that if you uploaded your data with an older version of rclone which didn't support the v2 API and modified times, rclone will decide to upload all your old data to fix the modification times. If you don't want this to happen use <code>--size-only</code> or <code>--checksum</code> flag to stop it.</p>
@@ -4866,6 +4975,8 @@ y/e/d&gt; y</code></pre>
<p>To view your current quota you can use the <code>rclone about remote:</code> command which will display your usage limit (quota), the usage in Google Drive, the size of all files in the Trash and the space used by other Google services such as Gmail. This command does not take any path arguments.</p>
<h3 id="specific-options-7">Specific options</h3>
<p>Here are the command line options specific to this cloud storage system.</p>
<h4 id="drive-acknowledge-abuse">--drive-acknowledge-abuse</h4>
<p>If downloading a file returns the error <code>This file has been identified as malware or spam and cannot be downloaded</code> with the error code <code>cannotDownloadAbusiveFile</code> then supply this flag to rclone to indicate you acknowledge the risks of downloading the file and rclone will download it anyway.</p>
<h4 id="drive-auth-owner-only">--drive-auth-owner-only</h4>
<p>Only consider files owned by the authenticated user.</p>
<h4 id="drive-chunk-sizesize">--drive-chunk-size=SIZE</h4>
@@ -4995,6 +5106,9 @@ y/e/d&gt; y</code></pre>
</tr>
</tbody>
</table>
<h4 id="drive-alternate-export">--drive-alternate-export</h4>
<p>If this option is set this instructs rclone to use an alternate set of export URLs for drive documents. Users have reported that the official export URLs can't export large documents, whereas these unofficial ones can.</p>
<p>See rclone issue <a href="https://github.com/ncw/rclone/issues/2243">#2243</a> for background, <a href="https://issuetracker.google.com/issues/36761333">this google drive issue</a> and <a href="https://www.labnol.org/internet/direct-links-for-google-drive/28356/">this helpful post</a>.</p>
<h4 id="drive-impersonate-user">--drive-impersonate user</h4>
<p>When using a service account, this instructs rclone to impersonate the user passed in.</p>
<h4 id="drive-list-chunk-int">--drive-list-chunk int</h4>
@@ -5465,7 +5579,7 @@ y/e/d&gt; y</code></pre>
* Say p for a personal OneDrive account
b) Business
p) Personal
b/p&gt; </code></pre>
b/p&gt;</code></pre>
<p>After that rclone requires an authentication of your account. The application will first authenticate your account, then query the OneDrive resource URL and do a second (silent) authentication for this resource URL.</p>
<h3 id="modified-time-and-hashes-4">Modified time and hashes</h3>
<p>OneDrive allows modification times to be set on objects accurate to 1 second. These will be used to detect whether objects need syncing or not.</p>
@@ -5482,8 +5596,7 @@ b/p&gt; </code></pre>
<p>There are quite a few characters that can't be in OneDrive file names. These can't occur on Windows platforms, but on non-Windows platforms they are common. Rclone will map these names to and from an identical looking unicode equivalent. For example if a file has a <code>?</code> in it will be mapped to <code>？</code> instead.</p>
<p>The largest allowed file size is 10GiB (10,737,418,240 bytes).</p>
<h3 id="versioning-issue">Versioning issue</h3>
<p>Every change in OneDrive causes the service to create a new version. This counts against a users quota.<br />
For example changing the modification time of a file creates a second version, so the file is using twice the space.</p>
<p>Every change in OneDrive causes the service to create a new version. This counts against a users quota. For example changing the modification time of a file creates a second version, so the file is using twice the space.</p>
<p>The <code>copy</code> is the only rclone command affected by this as we copy the file and then afterwards set the modification time to match the source file.</p>
<p>User <a href="https://github.com/Weropol">Weropol</a> has found a method to disable versioning on OneDrive</p>
<ol style="list-style-type: decimal">
@@ -5492,12 +5605,91 @@ For example changing the modification time of a file creates a second version, s
<li>Once on the Site settings page, navigate to Site Administration &gt; Site libraries and lists.</li>
<li>Click Customize &quot;Documents&quot;.</li>
<li>Click General Settings &gt; Versioning Settings.</li>
<li>Under Document Version History select the option No versioning.<br />
Note: This will disable the creation of new file versions, but will not remove any previous versions. Your documents are safe.</li>
<li>Under Document Version History select the option No versioning. Note: This will disable the creation of new file versions, but will not remove any previous versions. Your documents are safe.</li>
<li>Apply the changes by clicking OK.</li>
<li>Use rclone to upload or modify files. (I also use the --no-update-modtime flag)</li>
<li>Restore the versioning settings after using rclone. (Optional)</li>
</ol>
<h3 id="troubleshooting">Troubleshooting</h3>
<pre><code>Error: access_denied
Code: AADSTS65005
Description: Using application &#39;rclone&#39; is currently not supported for your organization [YOUR_ORGANIZATION] because it is in an unmanaged state. An administrator needs to claim ownership of the company by DNS validation of [YOUR_ORGANIZATION] before the application rclone can be provisioned.</code></pre>
<p>This means that rclone can't use the OneDrive for Business API with your account. You can't do much about it, maybe write an email to your admins.</p>
<p>However, there are other ways to interact with your OneDrive account. Have a look at the webdav backend: https://rclone.org/webdav/#sharepoint</p>
<h2 id="opendrive">OpenDrive</h2>
<p>Paths are specified as <code>remote:path</code></p>
<p>Paths may be as deep as required, eg <code>remote:directory/subdirectory</code>.</p>
<p>Here is an example of how to make a remote called <code>remote</code>. First run:</p>
<pre><code> rclone config</code></pre>
<p>This will guide you through an interactive setup process:</p>
<pre><code>n) New remote
d) Delete remote
q) Quit config
e/n/d/q&gt; n
name&gt; remote
Type of storage to configure.
Choose a number from below, or type in your own value
1 / Amazon Drive
\ &quot;amazon cloud drive&quot;
2 / Amazon S3 (also Dreamhost, Ceph, Minio)
\ &quot;s3&quot;
3 / Backblaze B2
\ &quot;b2&quot;
4 / Dropbox
\ &quot;dropbox&quot;
5 / Encrypt/Decrypt a remote
\ &quot;crypt&quot;
6 / Google Cloud Storage (this is not Google Drive)
\ &quot;google cloud storage&quot;
7 / Google Drive
\ &quot;drive&quot;
8 / Hubic
\ &quot;hubic&quot;
9 / Local Disk
\ &quot;local&quot;
10 / OpenDrive
\ &quot;opendrive&quot;
11 / Microsoft OneDrive
\ &quot;onedrive&quot;
12 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
\ &quot;swift&quot;
13 / SSH/SFTP Connection
\ &quot;sftp&quot;
14 / Yandex Disk
\ &quot;yandex&quot;
Storage&gt; 10
Username
username&gt;
Password
y) Yes type in my own password
g) Generate random password
y/g&gt; y
Enter the password:
password:
Confirm the password:
password:
--------------------
[remote]
username =
password = *** ENCRYPTED ***
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d&gt; y</code></pre>
<p>List directories in top level of your OpenDrive</p>
<pre><code>rclone lsd remote:</code></pre>
<p>List all the files in your OpenDrive</p>
<pre><code>rclone ls remote:</code></pre>
<p>To copy a local directory to an OpenDrive directory called backup</p>
<pre><code>rclone copy /home/source remote:backup</code></pre>
<h3 id="modified-time-and-md5sums-1">Modified time and MD5SUMs</h3>
<p>OpenDrive allows modification times to be set on objects accurate to 1 second. These will be used to detect whether objects need syncing or not.</p>
<h3 id="deleting-files-4">Deleting files</h3>
<p>Any files you delete with rclone will end up in the trash. Amazon don't provide an API to permanently delete files, nor to empty the trash, so you will have to do that with one of Amazon's apps or via the OpenDrive website. As of November 17, 2016, files are automatically deleted by Amazon from the trash after 30 days.</p>
<h3 id="limitations-10">Limitations</h3>
<p>Note that OpenDrive is case insensitive so you can't have a file called &quot;Hello.doc&quot; and one called &quot;hello.doc&quot;.</p>
<p>There are quite a few characters that can't be in OpenDrive file names. These can't occur on Windows platforms, but on non-Windows platforms they are common. Rclone will map these names to and from an identical looking unicode equivalent. For example if a file has a <code>?</code> in it will be mapped to <code>？</code> instead.</p>
<h2 id="qingstor">QingStor</h2>
<p>Paths are specified as <code>remote:bucket</code> (or <code>remote:</code> for the <code>lsd</code> command.) You may put subdirectories in too, eg <code>remote:bucket/path/to/dir</code>.</p>
<p>Here is an example of making an QingStor configuration. First run</p>
@@ -5803,9 +5995,9 @@ rclone lsd myremote:</code></pre>
<h3 id="modified-time-8">Modified time</h3>
<p>The modified time is stored as metadata on the object as <code>X-Object-Meta-Mtime</code> as floating point since the epoch accurate to 1 ns.</p>
<p>This is a defacto standard (used in the official python-swiftclient amongst others) for storing the modification time for an object.</p>
<h3 id="limitations-10">Limitations</h3>
<h3 id="limitations-11">Limitations</h3>
<p>The Swift API doesn't return a correct MD5SUM for segmented files (Dynamic or Static Large Objects) so rclone won't check or use the MD5SUM for these.</p>
<h3 id="troubleshooting">Troubleshooting</h3>
<h3 id="troubleshooting-1">Troubleshooting</h3>
<h4 id="rclone-gives-failed-to-create-file-system-for-remote-bad-request">Rclone gives Failed to create file system for &quot;remote:&quot;: Bad Request</h4>
<p>Due to an oddity of the underlying swift library, it gives a &quot;Bad Request&quot; error rather than a more sensible error when the authentication fails for Swift.</p>
<p>So this most likely means your username / password is wrong. You can investigate further with the <code>--dump-bodies</code> flag.</p>
@@ -5903,7 +6095,7 @@ y/e/d&gt; y</code></pre>
<h3 id="modified-time-and-hashes-5">Modified time and hashes</h3>
<p>pCloud allows modification times to be set on objects accurate to 1 second. These will be used to detect whether objects need syncing or not. In order to set a Modification time pCloud requires the object be re-uploaded.</p>
<p>pCloud supports MD5 and SHA1 type hashes, so you can use the <code>--checksum</code> flag.</p>
<h3 id="deleting-files-4">Deleting files</h3>
<h3 id="deleting-files-5">Deleting files</h3>
<p>Deleted files will be moved to the trash. Your subscription level will determine how long items stay in the trash. <code>rclone cleanup</code> can be used to empty the trash.</p>
<h2 id="sftp">SFTP</h2>
<p>SFTP is the <a href="https://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol">Secure (or SSH) File Transfer Protocol</a>.</p>
@@ -6010,11 +6202,17 @@ y/e/d&gt; y</code></pre>
<p>Here are the command line options specific to this remote.</p>
<h4 id="sftp-ask-password">--sftp-ask-password</h4>
<p>Ask for the SFTP password if needed when no password has been configured.</p>
<h4 id="ssh-path-override">--ssh-path-override</h4>
<p>Override path used by SSH connection. Allows checksum calculation when SFTP and SSH paths are different. This issue affects among others Synology NAS boxes.</p>
<p>Shared folders can be found in directories representing volumes</p>
<pre><code>rclone sync /home/local/directory remote:/directory --ssh-path-override /volume2/directory</code></pre>
<p>Home directory can be found in a shared folder called <code>homes</code></p>
<pre><code>rclone sync /home/local/directory remote:/home/directory --ssh-path-override /volume1/homes/USER/directory</code></pre>
<h3 id="modified-time-9">Modified time</h3>
<p>Modified times are stored on the server to 1 second precision.</p>
<p>Modified times are used in syncing and are fully supported.</p>
<p>Some SFTP servers disable setting/modifying the file modification time after upload (for example, certain configurations of ProFTPd with mod_sftp). If you are using one of these servers, you can set the option <code>set_modtime = false</code> in your RClone backend configuration to disable this behaviour.</p>
<h3 id="limitations-11">Limitations</h3>
<h3 id="limitations-12">Limitations</h3>
<p>SFTP supports checksums if the same login has shell access and <code>md5sum</code> or <code>sha1sum</code> as well as <code>echo</code> are in the remote's PATH. This remote checksumming (file hashing) is recommended and enabled by default. Disabling the checksumming may be required if you are connecting to SFTP servers which are not under your control, and to which the execution of remote commands is prohibited. Set the configuration option <code>disable_hashcheck</code> to <code>true</code> to disable checksumming.</p>
<p>Note that some SFTP servers (eg Synology) the paths are different for SSH and SFTP so the hashes can't be calculated properly. For them using <code>disable_hashcheck</code> is a good idea.</p>
<p>The only ssh agent supported under Windows is Putty's pageant.</p>
@@ -6326,6 +6524,103 @@ nounc = true</code></pre>
<p>This flag disables warning messages on skipped symlinks or junction points, as you explicitly acknowledge that they should be skipped.</p>
<h2 id="changelog">Changelog</h2>
<ul>
<li>v1.42 - 2018-06-16
<ul>
<li>New backends</li>
<li>OpenDrive (Oliver Heyme, Jakub Karlicek, ncw)</li>
<li>New commands</li>
<li>deletefile command (Filip Bartodziej)</li>
<li>New Features</li>
<li>copy, move: Copy single files directly, don't use <code>--files-from</code> work-around
<ul>
<li>this makes them much more efficient</li>
</ul></li>
<li>Implement <code>--max-transfer</code> flag to quit transferring at a limit
<ul>
<li>make exit code 8 for <code>--max-transfer</code> exceeded</li>
</ul></li>
<li>copy: copy empty source directories to destination (Ishuah Kariuki)</li>
<li>check: Add <code>--one-way</code> flag (Kasper Byrdal Nielsen)</li>
<li>Add siginfo handler for macOS for ctrl-T stats (kubatasiemski)</li>
<li>rc
<ul>
<li>add core/gc to run a garbage collection on demand</li>
<li>enable go profiling by default on the <code>--rc</code> port</li>
<li>return error from remote on failure</li>
</ul></li>
<li>lsf
<ul>
<li>Add <code>--absolute</code> flag to add a leading / onto path names</li>
<li>Add <code>--csv</code> flag for compliant CSV output</li>
<li>Add 'm' format specifier to show the MimeType</li>
<li>Implement 'i' format for showing object ID</li>
</ul></li>
<li>lsjson
<ul>
<li>Add MimeType to the output</li>
<li>Add ID field to output to show Object ID</li>
</ul></li>
<li>Add <code>--retries-sleep</code> flag (Benjamin Joseph Dag)</li>
<li>Oauth tidy up web page and error handling (Henning Surmeier)</li>
<li>Bug Fixes</li>
<li>Password prompt output with <code>--log-file</code> fixed for unix (Filip Bartodziej)</li>
<li>Calculate ModifyWindow each time on the fly to fix various problems (Stefan Breunig)</li>
<li>Mount</li>
<li>Only print &quot;File.rename error&quot; if there actually is an error (Stefan Breunig)</li>
<li>Delay rename if file has open writers instead of failing outright (Stefan Breunig)</li>
<li>Ensure atexit gets run on interrupt</li>
<li>macOS enhancements
<ul>
<li>Make <code>--noappledouble</code> <code>--noapplexattr</code></li>
<li>Add <code>--volname</code> flag and remove special chars from it</li>
<li>Make Get/List/Set/Remove xattr return ENOSYS for efficiency</li>
<li>Make <code>--daemon</code> work for macOS without CGO</li>
</ul></li>
<li>VFS
<ul>
<li>Add <code>--vfs-read-chunk-size</code> and <code>--vfs-read-chunk-size-limit</code> (Fabian Möller)</li>
<li>Fix ChangeNotify for new or changed folders (Fabian Möller)</li>
</ul></li>
<li>Local</li>
<li>Fix symlink/junction point directory handling under Windows
<ul>
<li><strong>NB</strong> you will need to add <code>-L</code> to your command line to copy files with reparse points</li>
</ul></li>
<li>Cache</li>
<li>Add non cached dirs on notifications (Remus Bunduc)</li>
<li>Allow root to be expired from rc (Remus Bunduc)</li>
<li>Clean remaining empty folders from temp upload path (Remus Bunduc)</li>
<li>Cache lists using batch writes (Remus Bunduc)</li>
<li>Use secure websockets for HTTPS Plex addresses (John Clayton)</li>
<li>Reconnect plex websocket on failures (Remus Bunduc)</li>
<li>Fix panic when running without plex configs (Remus Bunduc)</li>
<li>Fix root folder caching (Remus Bunduc)</li>
<li>Crypt</li>
<li>Check the crypted hash of files when uploading for extra data security</li>
<li>Dropbox</li>
<li>Make Dropbox for business folders accessible using an initial <code>/</code> in the path</li>
<li>Google Cloud Storage</li>
<li>Low level retry all operations if necessary</li>
<li>Google Drive</li>
<li>Add <code>--drive-acknowledge-abuse</code> to download flagged files</li>
<li>Add <code>--drive-alternate-export</code> to fix large doc export</li>
<li>Don't attempt to choose Team Drives when using rclone config create</li>
<li>Fix change list polling with team drives</li>
<li>Fix ChangeNotify for folders (Fabian Möller)</li>
<li>Fix about (and df on a mount) for team drives</li>
<li>Onedrive</li>
<li>Errorhandler for onedrive for business requests (Henning Surmeier)</li>
<li>S3</li>
<li>Adjust upload concurrency with <code>--s3-upload-concurrency</code> (themylogin)</li>
<li>Fix <code>--s3-chunk-size</code> which was always using the minimum</li>
<li>SFTP</li>
<li>Add <code>--ssh-path-override</code> flag (Piotr Oleszczyk)</li>
<li>Fix slow downloads for long latency connections</li>
<li>Webdav</li>
<li>Add workarounds for biz.mail.ru</li>
<li>Ignore Reason-Phrase in status line to fix 4shared (Rodrigo)</li>
<li>Better error message generation</li>
</ul></li>
<li>v1.41 - 2018-04-28
<ul>
<li>New backends</li>
@@ -7881,6 +8176,14 @@ THE SOFTWARE.</code></pre>
<li>hensur <a href="mailto:me@hensur.de">me@hensur.de</a></li>
<li>Oliver Heyme <a href="mailto:de8olihe@lego.com">de8olihe@lego.com</a></li>
<li>Richard Yang <a href="mailto:richard@yenforyang.com">richard@yenforyang.com</a></li>
<li>Piotr Oleszczyk <a href="mailto:piotr.oleszczyk@gmail.com">piotr.oleszczyk@gmail.com</a></li>
<li>Rodrigo <a href="mailto:rodarima@gmail.com">rodarima@gmail.com</a></li>
<li>NoLooseEnds <a href="mailto:NoLooseEnds@users.noreply.github.com">NoLooseEnds@users.noreply.github.com</a></li>
<li>Jakub Karlicek <a href="mailto:jakub@karlicek.me">jakub@karlicek.me</a></li>
<li>John Clayton <a href="mailto:john@codemonkeylabs.com">john@codemonkeylabs.com</a></li>
<li>Kasper Byrdal Nielsen <a href="mailto:byrdal76@gmail.com">byrdal76@gmail.com</a></li>
<li>Benjamin Joseph Dag <a href="mailto:bjdag1234@users.noreply.github.com">bjdag1234@users.noreply.github.com</a></li>
<li>themylogin <a href="mailto:themylogin@gmail.com">themylogin@gmail.com</a></li>
</ul>
<h1 id="contact-the-rclone-project">Contact the rclone project</h1>
<h2 id="forum">Forum</h2>