YenForYangandNick Craig-Wood
269f90c1e4
I discovered that `rclone` always upload in chunks of 16MiB whenever uploading a file smaller than `--drive-upload-cutoff`. This is undesirable since the purpose of the flag `--drive-upload-cutoff` is to *prevent* chunking below a certain file size. I realized that it wasn't `rclone` forcing the 16MiB chunks. The `google-api-go-client` forces a chunk size default of [`googleapi.DefaultUploadChunkSize`](https://github.com/googleapis/google-api-go-client/blob/32bf29c2e17105d5f285adac4531846c57847f11/googleapi/googleapi.go#L55-L57) bytes for resumable type uploads. This means that all requests that use `*drive.Service` directly for upload without specifying a `googleapi.ChunkSize` will be forced to use a *`resumable`* `uploadType` (rather than `multipart`) for files less than `googleapi.DefaultUploadChunkSize`. This is also noted directly in the Drive API client documentation [here](https://pkg.go.dev/google.golang.org/api/drive/v3@v0.44.0#FilesUpdateCall.Media). This fixes the problem by passing `googleapi.ChunkSize(0)` to `Media()` method calls, which is the only way to disable chunking completely. This is mentioned in the API docs [here](https://pkg.go.dev/google.golang.org/api/googleapi@v0.44.0#ChunkSize). The other alternative would be to pass `googleapi.ChunkSize(f.opt.ChunkSize)` -- however, I'm *strongly* in favor of *not* doing this for performance reasons. By not explicitly passing a `googleapi.ChunkSize(0)`, we effectively allow [`PrepareUpload()`](https://pkg.go.dev/google.golang.org/api/internal/gensupport@v0.44.0#PrepareUpload) to create a [`NewMediaBuffer`](https://pkg.go.dev/google.golang.org/api/internal/gensupport@v0.44.0#NewMediaBuffer) that copies the original `io.Reader` passed to `Media()` in order to check that its size is less than `ChunkSize`, which will unnecessarily consume time and memory. `minChunkSize` is also changed to be `googleapi.MinUploadChunkSize`, as it is something specified we have no control over.