Each upload allocated a fresh chunk-sized buffer (10 MiB by default)
regardless of the file size, so bulk transfers of many files churned
allocations and GC.
Buffer chunks with multipart.NewRW instead so chunk memory is reused
across uploads and is part of rclone's central memory management. The
pool.RW is seekable, so the existing rewind on retry carries over.
The body goes through lib/rest which already wraps it so the transport
can't close the pool buffer. The upload loop closes it after every
chunk, on error paths included. A source which ends before the
declared size is now reported as a short read before the chunk is sent
rather than as an incomplete write afterwards.
The FsPutRetry integration test covers the retry of a failed upload
request and checks the buffers are returned to the pool.
Nick Craig-Wood
6613d4ec63