diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 5494faf72..72dc3945e 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -46,8 +46,9 @@ and https://github.com/rclone/rclone/blob/master/CONTRIBUTING.md#integration-tes - [ ] This change is trivial **OR** it has been discussed and agreed in the linked issue. - [ ] I have read the [contribution guidelines](https://github.com/rclone/rclone/blob/master/CONTRIBUTING.md#submitting-a-new-feature-or-bug-fix). +- [ ] **(If I used AI tools to help write this code)** I have read and understood the [AI-assisted contributions guidance](https://github.com/rclone/rclone/blob/master/CONTRIBUTING.md#ai-assisted-contributions), and I have tested and take ownership of this change myself. - [ ] I have added tests for all changes in this PR if appropriate. - [ ] I have added documentation for the changes if appropriate. - [ ] All commit messages are in [house style](https://github.com/rclone/rclone/blob/master/CONTRIBUTING.md#commit-messages). -- [ ] **(Backend changes only)** `test_all` passes for this backend and I can provide a test account for the integration tester - see [CONTRIBUTING.md](https://github.com/rclone/rclone/blob/master/CONTRIBUTING.md#integration-tests). +- [ ] **(Backend changes only)** `test_all` passes for this backend and if submitting a new backend can provide a test account for the integration tester - see [CONTRIBUTING.md](https://github.com/rclone/rclone/blob/master/CONTRIBUTING.md#integration-tests). - [ ] This Pull Request is ready for review. diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 000000000..4d674c7f3 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,134 @@ +# AGENTS.md + +This file provides guidance to AI coding agents (e.g. Claude Code, Codex, Cursor, Gemini CLI, and similar tools) when working with code in this repository. + +Rclone welcomes AI-assisted contributions, but the expectation is that you, the human submitter, understand every line you propose and have compiled and tested it against real rclone code - not just generated it. See the "AI-assisted contributions" section of [CONTRIBUTING.md](CONTRIBUTING.md) before opening a pull request. + +## Project Overview + +Rclone is a command-line program to sync files and directories to and from cloud storage providers. It's written in Go and supports 70+ backends (cloud storage systems). Think "rsync for cloud storage". + +## General Notes + +**We take backwards compatibility very seriously.** PRs should not change the observable behaviour of existing commands, flags or rc API without very good reason. Rclone does not try to preserve a stable Go API but try not to change it gratuitously. + +Rclone operates with a lot of different backends, so **compatibility is key**. It is the backend integration tests which guarantee that compatibility. Changes should consider both known and unknown backends and should take care not to break functionality of existing installations. + +The core parts of rclone under `fs` and `vfs` need to work with all backends and **backend specific hacks won't be merged**. Fixes likely need to go in the relevant backend or if new behaviour is really needed, a new Feature flag needs to be added. + +**Changes should be kept to the minimum.** Work hard to make the most elegant, smallest change you can. Do not refactor or re-order code unless necessary as this makes review more challenging. Re-use existing test scaffolding, existing or library routines (e.g. `lib`) where possible. + +Make sure added tests **actually test the code you have written** and test the intention behind the change. If you are fixing a problem, write the tests first to reproduce the problem before starting on the fix. + +## Build and Test Commands + +```bash +# Build rclone (simple) +go build + +# Build with version info (preferred) +make + +# Run all unit tests (no cloud credentials needed) +make quicktest +# or equivalently: +RCLONE_CONFIG="/notfound" go test ./... + +# Run tests for a specific package +cd backend/memory && go test -v +# or from root: +go test -v ./backend/memory/ + +# Run a single test +go test -v -run TestIntegration/FsCheckWrap ./backend/memory/ + +# Run tests with race detector +make racequicktest + +# Lint (requires golangci-lint) +golangci-lint run ./... + +# Run backend integration tests (requires configured TestRemote remote) +cd backend/drive && go test -v +# Run sync/operations integration tests against a remote +cd fs/sync && go test -v -remote TestDrive: +cd fs/operations && go test -v -remote TestDrive: + +# Run integration tests via test framework +go run ./fstest/test_all -backends drive +``` + +## Architecture + +### Entry Point and Plugin Registration + +`rclone.go` is the main entry point. It imports `backend/all` and `cmd/all` which use Go's `init()` pattern to register all backends and commands. Each backend calls `fs.Register()` with a `fs.RegInfo` struct during init. + +### Core Interfaces (`fs/`) + +The `fs` package defines the core abstractions: +- **`fs.Fs`** (`fs/types.go`): The filesystem interface every backend must implement (List, NewObject, Put, Mkdir, Rmdir). +- **`fs.Object`** (`fs/types.go`): Interface for a file/object (Open, Update, Remove, SetModTime). +- **`fs.Features`** (`fs/features.go`): Optional capabilities a backend can declare (Purge, Copy, Move, DirMove, etc.). Backends set function pointers for operations they support; nil means not supported. +- **`fs.RegInfo`** (`fs/registry.go`): Registration metadata for a backend including its name, config options, and NewFs constructor. + +### Backend Structure (`backend/`) + +Each backend is a single Go package (e.g., `backend/s3/`, `backend/drive/`). Key conventions: +- Main implementation in a single file (e.g., `s3.go`) - **do not** split into `fs.go`/`object.go`. +- API types go in a separate `api/types.go` file. +- Test file (e.g., `s3_test.go`) uses `fstests.Run()` from `fstest/fstests` for standardized integration tests. +- Register in `backend/all/all.go` via blank import. +- HTTP-based backends should use `lib/rest` for HTTP calls and `fs/fshttp` for the HTTP client. +- Use `lib/dircache` for directory-ID-based remotes, `lib/oauthutil` for OAuth, `lib/pacer` for rate limiting. + +### Command Structure (`cmd/`) + +Each command is a package under `cmd/` registered in `cmd/all/all.go` via blank import. Commands use cobra via `cmd.Main()`. + +### Key Subsystems + +- **`fs/operations/`**: Core file operations (Copy, Move, Delete, etc.) +- **`fs/sync/`**: Directory sync logic +- **`fs/march/`**: Parallel directory tree walker used by sync +- **`fs/filter/`**: Include/exclude filtering +- **`fs/accounting/`**: Transfer statistics and bandwidth limiting +- **`fs/config/`**: Configuration file management +- **`vfs/`**: Virtual filesystem layer (used by mount, serve) +- **`librclone/`**: C-compatible library interface for embedding rclone +- **`fstest/`**: Integration test framework; `fstest/fstests/` has the generic backend test suite + +## Commit Message Convention + +Prefix with the directory of the change, then a colon: `drive: add team drive support - fixes #885`. For cross-cutting changes use a broader prefix like `fs` or `operations`. + +Make the first line of your commit message a summary of the change that a user (not a developer) of rclone would like to read. So write `drive: fix server side copy of big files` instead of `drive: no longer set the MimeType in Move or Copy`. This is important because these lines go into the change log which is read by users. + +## Code Commenting Style + +Comments describe the code as it is now, for a future reader who has no knowledge of the change that introduced it. + +Every exported type, function, field, and constant has a **godoc comment** that starts with its name and is phrased as a present-tense statement of what it is or does (`// Mkdir makes the directory (container, bucket)`) + +**Document the contract** callers need - preconditions, what's returned, which sentinel errors are returned and when, and any "shouldn't return an error if it already exists" style caveats - rather than the implementation. + +**Keep comments terse**: a single line for most things, with extra paragraphs (separated by blank `//` lines) reserved for genuine subtlety. + +Inline comments inside function bodies should **explain why** - a non-obvious API quirk, a workaround, a gotcha, or an ordering constraint - and may cite an external reference (forum thread, vendor docs, RFC) when that's what makes the behaviour non-obvious; skip comments that merely restate what the code plainly does. + +Use `FIXME` and `TODO` for known shortcomings. + +**Do not write comments that narrate the change itself** or compare against the previous behaviour (no "now we also handle...", "changed to...", "previously this returned...", or references to bug/PR numbers in the code) - that context belongs in the commit message, not in source that will outlive the change. + +Code comments should only refer to **current** behaviour - and shouldn't describe behaviour that is trivially deducible from reading the code. + +## Linting Configuration + +Uses golangci-lint v2 with config in `.golangci.yml`. Enabled linters: errcheck, govet, ineffassign, staticcheck, unused, gocritic, misspell, revive, unconvert. The `goimports` formatter is also enabled. + +## Documentation + +- Backend option docs come from `Help:` fields in the Go source Options structs, not from markdown files. +- Command docs are in the command source code (e.g., `cmd/ls/ls.go`). +- Don't commit autogenerated doc changes from `make backenddocs` or `make commanddocs`. +- Website docs are in `docs/content/` as markdown, built with Hugo (`make serve` to preview). diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 000000000..3cd3ec0ed --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,6 @@ +# CLAUDE.md + +Guidance for this repository lives in AGENTS.md, shared with all AI coding +agents. Claude Code reads it via the import below. + +@AGENTS.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 69e9eed28..bbfc08ebf 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -113,6 +113,36 @@ GitHub. You may sometimes be asked to [base your changes on the latest master](#basing-your-changes-on-the-latest-master) or [squash your commits](#squashing-your-commits). +## AI-assisted contributions + +You are welcome to use AI coding assistants (Claude Code, Codex, Cursor, Gemini +CLI, and similar) to help write your contribution. Rclone has an +[AGENTS.md](AGENTS.md) file at the top of the repository describing the project's +conventions; point your tool at it so the code it produces matches rclone's +style. + +However, the same standard applies to every pull request whether or not a tool +was involved: **you are responsible for the code you submit.** Before you open a +pull request please make sure that: + +- You understand every line of the change and can explain why it is correct. If + a reviewer asks a question about it, you should be able to answer without + going back to the tool. +- You have actually built and run it. At a minimum `go build` and + `make quicktest` must pass, and for a backend change you should run the + [backend tests](#backend-testing) against a real remote where you can. +- The change is a genuine fix or feature that you have verified solves the + problem, not a plausible-looking guess. Unverified, AI-generated pull requests + that do not compile, do not pass the tests, invent APIs that do not exist, or + do not actually do what the description claims waste maintainer time and are + likely to be closed. +- You have trimmed the comments. AI tools tend to add verbose comments that + restate what the code plainly does or narrate the change being made; please + cut these down to match the [code commenting style](AGENTS.md#code-commenting-style). + +In short: an AI assistant is a tool to help *you* contribute, not a substitute +for understanding and testing your own work. + ## Using Git and GitHub ### Committing your changes