iclouddrive: add read only iCloud Photos support and SRP authentication

Add read-only iCloud Photos support to the existing iclouddrive
backend via `service = photos` config option.

Also includes auth improvements on top of #9209's SRP authentication.

**Photos features:**
- 3-level hierarchy: libraries (Personal + Shared Photo Library) →
  albums → photos/videos
- server-side smart albums (All Photos, Videos, Favorites,
  Screenshots, Live, Bursts, Panoramas, Slo-mo, Time-lapse, Portrait,
  Long Exposure, Animated, Hidden, Recently Deleted)
- User-created albums and nested album folders
- Live Photo `.MOV` companions as first-class entries
- Edited photo versions (`-edited` suffix) and RAW alternatives
- Duplicate filename dedup for camera counter wrap collisions
- Parallel cold listing for large albums
- Delta sync via CloudKit `changes/zone` - warm listings near-instant from disk cache
- Disk cache (libraries, albums, photos) with atomic writes for crash safety
- `ChangeNotify` support for FUSE mounts via `changes/zone` polling
- `ListR` support for `--fast-list` and recursive operations
- `--metadata` support - width, height, added-time, favorite, hidden
- Fresh download URLs per file - no stale URL failures on long copies
- FUSE mount documentation with recommended flags

**Auth improvements over #9209:**
- SMS 2FA fallback for users without trusted Apple devices
- Explicit push notification request - fixes iOS/macOS 26.4+ where 409
  no longer auto-pushes
- Thread safety for concurrent FUSE callers (mutexes on session and client state)
- Session endpoint caching - skips ~5s `/validate` round-trip on warm start
- `Disconnect` support - clears auth state + disk cache
- PCS cookie support for Advanced Data Protection accounts, including
  trusted-device approval for PCS cookies

Built on @coughlanio's Photos PoC (Closes #8734) and @mikegillan's SRP auth (#9209).

Fixes #7982
Co-authored-by: Chris Coughlan <chris@coughlan.io>
This commit is contained in:
Yakov Till
2026-04-27 16:55:31 +01:00
committed by GitHub
parent 6b67be9d48
commit d0c469c3c0
12 changed files with 6293 additions and 478 deletions
+83 -49
View File
@@ -1,4 +1,4 @@
// Package api provides functionality for interacting with the iCloud API.
// Package api provides functionality for interacting with the iCloud API
package api
import (
@@ -8,16 +8,19 @@ import (
"errors"
"fmt"
"net/http"
"os"
"path/filepath"
"strings"
"sync"
"github.com/rclone/rclone/fs"
"github.com/rclone/rclone/fs/config"
"github.com/rclone/rclone/fs/fshttp"
"github.com/rclone/rclone/lib/rest"
)
const (
baseEndpoint = "https://www.icloud.com"
homeEndpoint = "https://www.icloud.com"
setupEndpoint = "https://setup.icloud.com/setup/ws/1"
authEndpoint = "https://idmsa.apple.com/appleauth/auth"
)
@@ -28,26 +31,21 @@ type sessionSave func(*Session)
type Client struct {
appleID string
password string
remoteName string // rclone remote name, used for cache namespacing
srv *rest.Client
Session *Session
sessionSaveCallback sessionSave
drive *DriveService
mu sync.Mutex // protects drive and Authenticate
}
// New creates a new Client instance with the provided Apple ID, password, trust token, cookies, and session save callback.
//
// Parameters:
// - appleID: the Apple ID of the user.
// - password: the password of the user.
// - trustToken: the trust token for the session.
// - clientID: the client id for the session.
// - cookies: the cookies for the session.
// - sessionSaveCallback: the callback function to save the session.
func New(appleID, password, trustToken string, clientID string, cookies []*http.Cookie, sessionSaveCallback sessionSave) (*Client, error) {
// New creates a new iCloud API client and initializes its HTTP session
func New(appleID, password, trustToken string, clientID string, cookies []*http.Cookie, sessionSaveCallback sessionSave, remoteName string) (*Client, error) {
icloud := &Client{
appleID: appleID,
appleID: strings.ToLower(appleID), // Apple SRP requires lowercase in client-side proof
password: password,
remoteName: filepath.Base(remoteName),
srv: rest.NewClient(fshttp.NewClient(context.Background())),
Session: NewSession(),
sessionSaveCallback: sessionSaveCallback,
@@ -59,10 +57,12 @@ func New(appleID, password, trustToken string, clientID string, cookies []*http.
return icloud, nil
}
// DriveService returns the DriveService instance associated with the Client.
// DriveService returns the DriveService instance, creating it on first call
func (c *Client) DriveService() (*DriveService, error) {
var err error
c.mu.Lock()
defer c.mu.Unlock()
if c.drive == nil {
var err error
c.drive, err = NewDriveService(c)
if err != nil {
return nil, err
@@ -71,11 +71,7 @@ func (c *Client) DriveService() (*DriveService, error) {
return c.drive, nil
}
// Request makes a request and retries it if the session is invalid.
//
// This function is the main entry point for making requests to the iCloud
// API. If the initial request returns a 401 (Unauthorized), it will try to
// reauthenticate and retry the request.
// Request makes a request to the iCloud API, re-authenticating on 401/421
func (c *Client) Request(ctx context.Context, opts rest.Opts, request any, response any) (resp *http.Response, err error) {
resp, err = c.Session.Request(ctx, opts, request, response)
if err != nil && resp != nil {
@@ -89,55 +85,93 @@ func (c *Client) Request(ctx context.Context, opts rest.Opts, request any, respo
if c.Session.Requires2FA() {
return nil, errors.New("trust token expired, please reauth")
}
return c.RequestNoReAuth(ctx, opts, request, response)
return c.Session.Request(ctx, opts, request, response)
}
}
return resp, err
}
// RequestNoReAuth makes a request without re-authenticating.
//
// This function is useful when you have a session that is already
// authenticated, but you need to make a request without triggering
// a re-authentication.
func (c *Client) RequestNoReAuth(ctx context.Context, opts rest.Opts, request any, response any) (resp *http.Response, err error) {
// Make the request without re-authenticating
resp, err = c.Session.Request(ctx, opts, request, response)
return resp, err
}
// Authenticate authenticates the client with the iCloud API.
// Authenticate authenticates the client, reusing existing session if valid
func (c *Client) Authenticate(ctx context.Context) error {
c.mu.Lock()
defer c.mu.Unlock()
// Skip /validate round-trip when saved session has cookies + service endpoints
// Native client behavior: use cached session, reauth lazily on 401/421
if c.Session.Cookies != nil && len(c.Session.AccountInfo.Webservices) > 0 {
fs.Debugf(nil, "iclouddrive: reusing saved session")
return nil
}
// Try loading cached service endpoints to avoid /validate round-trip (~5s)
if c.Session.Cookies != nil && c.loadCachedWebservices() {
fs.Debugf(nil, "iclouddrive: reusing session with cached endpoints")
return nil
}
if c.Session.Cookies != nil {
if err := c.Session.ValidateSession(ctx); err == nil {
fs.Debugf("icloud", "Valid session, no need to reauth")
fs.Debugf(nil, "iclouddrive: valid session, no need to reauth")
c.saveCachedWebservices()
return nil
}
c.Session.Cookies = nil
}
fs.Debugf("icloud", "Authenticating as %s\n", c.appleID)
fs.Debugf(nil, "iclouddrive: authenticating")
err := c.Session.SignIn(ctx, c.appleID, c.password)
if err != nil {
return err
}
// If 2FA is required, don't try AuthWithToken yet — the caller
// must complete 2FA first, then call AuthWithToken.
// If 2FA is required, skip AuthWithToken - caller must complete 2FA first
if c.Session.Requires2FA() {
return nil
}
err = c.Session.AuthWithToken(ctx)
if err == nil && c.sessionSaveCallback != nil {
c.sessionSaveCallback(c.Session)
if err == nil {
c.saveCachedWebservices()
if c.sessionSaveCallback != nil {
c.sessionSaveCallback(c.Session)
}
}
return err
}
// SignIn signs in the client using the provided context and credentials.
func (c *Client) SignIn(ctx context.Context) error {
return c.Session.SignIn(ctx, c.appleID, c.password)
// loadCachedWebservices loads service endpoints from disk cache
func (c *Client) loadCachedWebservices() bool {
data, err := os.ReadFile(filepath.Join(config.GetCacheDir(), cacheSubdir, c.remoteName, "webservices.json"))
if err != nil {
return false
}
var ws map[string]*webService
if err := json.Unmarshal(data, &ws); err != nil {
return false
}
if len(ws) == 0 {
return false
}
c.Session.AccountInfo.Webservices = ws
return true
}
// saveCachedWebservices persists service endpoints to disk
func (c *Client) saveCachedWebservices() {
if len(c.Session.AccountInfo.Webservices) == 0 {
return
}
saveJSONCache(filepath.Join(config.GetCacheDir(), cacheSubdir, c.remoteName), "webservices.json", c.Session.AccountInfo.Webservices)
}
// CacheDir returns the disk cache directory for this remote
func (c *Client) CacheDir() string {
return filepath.Join(config.GetCacheDir(), cacheSubdir, c.remoteName)
}
// ClearCacheDir removes all disk cache files for a remote
func ClearCacheDir(remoteName string) {
dir := filepath.Join(config.GetCacheDir(), cacheSubdir, filepath.Base(remoteName))
if err := os.RemoveAll(dir); err != nil {
fs.Debugf(nil, "iclouddrive: failed to clear cache: %v", err)
}
}
// IntoReader marshals the provided values into a JSON encoded reader
@@ -155,19 +189,19 @@ type RequestError struct {
Text string
}
// Error satisfy the error interface.
// Error satisfies the error interface
func (e *RequestError) Error() string {
return fmt.Sprintf("%s: %s", e.Text, e.Status)
}
func newRequestError(Status string, Text string) *RequestError {
func newRequestError(status string, text string) *RequestError {
return &RequestError{
Status: strings.ToLower(Status),
Text: Text,
Status: strings.ToLower(status),
Text: text,
}
}
// newErr orf makes a new error from sprintf parameters.
func newRequestErrorf(Status string, Text string, Parameters ...any) *RequestError {
return newRequestError(strings.ToLower(Status), fmt.Sprintf(Text, Parameters...))
// newRequestErrorf makes a new error from sprintf parameters
func newRequestErrorf(status string, text string, params ...any) *RequestError {
return newRequestError(status, fmt.Sprintf(text, params...))
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+441 -227
View File
@@ -2,16 +2,21 @@ package api
import (
"context"
"crypto/sha256"
"encoding/base64"
"encoding/hex"
"encoding/json"
"fmt"
"io"
"maps"
"net/http"
"net/url"
"slices"
"strings"
"sync"
"time"
"github.com/google/uuid"
"github.com/oracle/oci-go-sdk/v65/common"
"github.com/rclone/rclone/fs"
"github.com/rclone/rclone/fs/fshttp"
"github.com/rclone/rclone/lib/rest"
@@ -32,6 +37,7 @@ type Session struct {
Cookies []*http.Cookie `json:"cookies"`
AccountInfo AccountInfo `json:"account_info"`
mu sync.Mutex `json:"-"` // protects session fields during concurrent Request calls
srv *rest.Client `json:"-"`
needs2FA bool `json:"-"` // set when SRP signin returns 409
}
@@ -45,20 +51,117 @@ type srpInitResponse struct {
C string `json:"c"`
}
// String returns the session as a string
// func (s *Session) String() string {
// jsession, _ := json.Marshal(s)
// return string(jsession)
// }
// Request makes a request
func (s *Session) Request(ctx context.Context, opts rest.Opts, request any, response any) (*http.Response, error) {
resp, err := s.srv.CallJSON(ctx, &opts, &request, &response)
if err != nil {
return resp, err
func cookieValueFingerprint(value string) string {
if value == "" {
return ""
}
sum := sha256.Sum256([]byte(value))
return hex.EncodeToString(sum[:6])
}
func authStateBodySummary(body []byte) string {
summary := []string{fmt.Sprintf("bytes=%d", len(body)), "hash=" + cookieValueFingerprint(string(body))}
var top map[string]json.RawMessage
if err := json.Unmarshal(body, &top); err != nil {
return strings.Join(summary, " ")
}
keys := make([]string, 0, len(top))
for key := range top {
keys = append(keys, key)
}
slices.Sort(keys)
summary = append(summary, fmt.Sprintf("keys=%v", keys))
if _, ok := top["phoneNumberVerification"]; ok {
summary = append(summary, "wrapped=true")
}
return strings.Join(summary, " ")
}
func cookieDebugSummary(c *http.Cookie) string {
parts := []string{c.Name}
if c.Value == "" {
parts = append(parts, "empty", "len=0")
} else {
parts = append(parts, "set", fmt.Sprintf("len=%d", len(c.Value)), "hash="+cookieValueFingerprint(c.Value))
}
if c.Path != "" {
parts = append(parts, "path="+c.Path)
}
if c.Domain != "" {
parts = append(parts, "domain="+c.Domain)
}
if c.MaxAge != 0 {
parts = append(parts, fmt.Sprintf("maxAge=%d", c.MaxAge))
}
if !c.Expires.IsZero() {
parts = append(parts, "expires")
}
return strings.Join(parts, ",")
}
func cookieDebugSummaries(cookies []*http.Cookie) []string {
if len(cookies) == 0 {
return nil
}
out := make([]string, 0, len(cookies))
for _, c := range cookies {
out = append(out, cookieDebugSummary(c))
}
return out
}
func cookieJarDebugSummaries(cookies []*http.Cookie) []string {
if len(cookies) == 0 {
return nil
}
out := make([]string, 0, len(cookies))
for _, c := range cookies {
summary := c.Name
if c.Value == "" {
summary += ",empty,len=0"
} else {
summary += fmt.Sprintf(",len=%d,hash=%s", len(c.Value), cookieValueFingerprint(c.Value))
}
out = append(out, summary)
}
return out
}
func (s *Session) mergeCookies(cookies []*http.Cookie) {
if len(cookies) == 0 {
return
}
existing := make(map[string]int, len(s.Cookies))
for i, c := range s.Cookies {
existing[c.Name] = i
}
for _, c := range cookies {
if c.Value == "" {
if idx, ok := existing[c.Name]; ok {
fs.Debugf(nil, "iclouddrive: deleting empty auth cookie: %s", cookieDebugSummary(c))
s.Cookies = append(s.Cookies[:idx], s.Cookies[idx+1:]...)
delete(existing, c.Name)
for j := idx; j < len(s.Cookies); j++ {
existing[s.Cookies[j].Name] = j
}
} else {
fs.Debugf(nil, "iclouddrive: ignoring empty auth cookie tombstone for missing cookie: %s", cookieDebugSummary(c))
}
continue
}
if idx, ok := existing[c.Name]; ok {
s.Cookies[idx] = c
} else {
s.Cookies = append(s.Cookies, c)
existing[c.Name] = len(s.Cookies) - 1
}
}
}
// extractHeaders reads Apple session headers from an HTTP response into the session
// Does NOT acquire s.mu - caller is responsible for locking if needed
func (s *Session) extractHeaders(resp *http.Response) {
s.mergeCookies(resp.Cookies())
if val := resp.Header.Get("X-Apple-ID-Account-Country"); val != "" {
s.AccountCountry = val
}
@@ -77,7 +180,17 @@ func (s *Session) Request(ctx context.Context, opts rest.Opts, request any, resp
if val := resp.Header.Get("X-Apple-Auth-Attributes"); val != "" {
s.AuthAttributes = val
}
}
// Request makes a JSON API request and extracts session headers
func (s *Session) Request(ctx context.Context, opts rest.Opts, request any, response any) (*http.Response, error) {
resp, err := s.srv.CallJSON(ctx, &opts, &request, &response)
if err != nil {
return resp, err
}
s.mu.Lock()
s.extractHeaders(resp)
s.mu.Unlock()
return resp, nil
}
@@ -89,13 +202,8 @@ func (s *Session) Requires2FA() bool {
return s.AccountInfo.DsInfo != nil && s.AccountInfo.DsInfo.HsaVersion == 2 && s.AccountInfo.HsaChallengeRequired
}
// SignIn performs SRP-based authentication against Apple's idmsa endpoint.
// SignIn performs SRP-based authentication against Apple's idmsa endpoint
func (s *Session) SignIn(ctx context.Context, appleID, password string) error {
// Apple's SRP implementation expects a lowercase account name.
// The old plaintext flow didn't need this because the server normalized
// it, but SRP uses the username in client-side proof computation (M1).
appleID = strings.ToLower(appleID)
// Step 1: Initialize the auth session
if err := s.authStart(ctx); err != nil {
return fmt.Errorf("authStart: %w", err)
@@ -106,8 +214,11 @@ func (s *Session) SignIn(ctx context.Context, appleID, password string) error {
return fmt.Errorf("authFederate: %w", err)
}
// Step 3: SRP init — send client public value A, get salt + B
client := newSRPClient()
// Step 3: SRP init - send client public value A, get salt + B
client, err := newSRPClient()
if err != nil {
return fmt.Errorf("newSRPClient: %w", err)
}
aBase64 := base64.StdEncoding.EncodeToString(client.getABytes())
initResp, err := s.authSRPInit(ctx, aBase64, appleID)
@@ -130,9 +241,11 @@ func (s *Session) SignIn(ctx context.Context, appleID, password string) error {
if err != nil {
return fmt.Errorf("derivePassword: %w", err)
}
client.processChallenge([]byte(appleID), derivedKey, salt, serverB)
if err := client.processChallenge([]byte(appleID), derivedKey, salt, serverB); err != nil {
return fmt.Errorf("processChallenge: %w", err)
}
// Step 5: Complete — send M1, M2 proofs
// Step 5: Complete - send M1, M2 proofs
m1Base64 := base64.StdEncoding.EncodeToString(client.M1)
m2Base64 := base64.StdEncoding.EncodeToString(client.M2)
@@ -143,7 +256,7 @@ func (s *Session) SignIn(ctx context.Context, appleID, password string) error {
return nil
}
// authStart initializes the SRP auth session by hitting the authorize/signin endpoint.
// authStart initializes the SRP auth session by hitting the authorize/signin endpoint
func (s *Session) authStart(ctx context.Context) error {
if s.FrameID == "" {
s.FrameID = strings.ToLower(uuid.New().String())
@@ -184,20 +297,11 @@ func (s *Session) authStart(ctx context.Context) error {
return fmt.Errorf("authStart: unexpected status %s", resp.Status)
}
if val := resp.Header.Get("X-Apple-Auth-Attributes"); val != "" {
s.AuthAttributes = val
}
if val := resp.Header.Get("scnt"); val != "" {
s.Scnt = val
}
if val := resp.Header.Get("X-Apple-ID-Session-Id"); val != "" {
s.SessionID = val
}
s.extractHeaders(resp)
return nil
}
// authFederate submits the account name to Apple's federate endpoint.
// authFederate submits the account name to Apple's federate endpoint
func (s *Session) authFederate(ctx context.Context, accountName string) error {
values := map[string]any{
"accountName": accountName,
@@ -224,16 +328,7 @@ func (s *Session) authFederate(ctx context.Context, accountName string) error {
}
_ = resp.Body.Close()
if val := resp.Header.Get("X-Apple-Auth-Attributes"); val != "" {
s.AuthAttributes = val
}
if val := resp.Header.Get("scnt"); val != "" {
s.Scnt = val
}
if val := resp.Header.Get("X-Apple-ID-Session-Id"); val != "" {
s.SessionID = val
}
s.extractHeaders(resp)
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("authFederate: unexpected status %s", resp.Status)
}
@@ -241,7 +336,7 @@ func (s *Session) authFederate(ctx context.Context, accountName string) error {
}
// authSRPInit sends the client's public value A to the server and retrieves
// the salt, server public value B, iteration count, protocol, and challenge.
// the salt, server public value B, iteration count, protocol, and challenge
func (s *Session) authSRPInit(ctx context.Context, aBase64, accountName string) (*srpInitResponse, error) {
values := map[string]any{
"a": aBase64,
@@ -267,18 +362,12 @@ func (s *Session) authSRPInit(ctx context.Context, aBase64, accountName string)
return nil, err
}
if val := resp.Header.Get("scnt"); val != "" {
s.Scnt = val
}
if val := resp.Header.Get("X-Apple-ID-Session-Id"); val != "" {
s.SessionID = val
}
s.extractHeaders(resp)
return &initResp, nil
}
// authSRPComplete sends the SRP proofs M1 and M2 to complete authentication.
// Returns nil on success (200 or 409/2FA needed).
// authSRPComplete sends the SRP proofs M1 and M2 to complete authentication
// Returns nil on success (200 or 409/2FA needed)
func (s *Session) authSRPComplete(ctx context.Context, accountName, m1Base64, m2Base64, c string) error {
trustTokens := []string{}
if s.TrustToken != "" {
@@ -305,7 +394,6 @@ func (s *Session) authSRPComplete(ctx context.Context, accountName, m1Base64, m2
ExtraHeaders: s.getSRPAuthHeaders(),
RootURL: authEndpoint,
IgnoreStatus: true,
NoResponse: true,
Body: body,
}
@@ -313,51 +401,65 @@ func (s *Session) authSRPComplete(ctx context.Context, accountName, m1Base64, m2
if err != nil {
return err
}
respBody, _ := io.ReadAll(resp.Body)
_ = resp.Body.Close()
// Extract updated headers
if val := resp.Header.Get("X-Apple-Auth-Attributes"); val != "" {
s.AuthAttributes = val
}
if val := resp.Header.Get("X-Apple-Session-Token"); val != "" {
s.SessionToken = val
}
if val := resp.Header.Get("scnt"); val != "" {
s.Scnt = val
}
if val := resp.Header.Get("X-Apple-ID-Session-Id"); val != "" {
s.SessionID = val
}
if val := resp.Header.Get("X-Apple-ID-Account-Country"); val != "" {
s.AccountCountry = val
}
if val := resp.Header.Get("X-Apple-TwoSV-Trust-Token"); val != "" {
s.TrustToken = val
}
s.extractHeaders(resp)
switch resp.StatusCode {
case http.StatusOK:
fs.Debugf("icloud", "SRP sign in successful")
fs.Debugf(nil, "iclouddrive: SRP sign in successful")
return nil
case http.StatusConflict:
// 409 = 2FA required, this is expected
fs.Debugf("icloud", "SRP sign in requires 2FA")
// 409 = 2FA required
fs.Debugf(nil, "iclouddrive: SRP sign in requires 2FA, response: %s", respBody)
s.needs2FA = true
return nil
case http.StatusPreconditionFailed:
// 412 = non-2FA account needs repair/complete step
fs.Debugf(nil, "iclouddrive: SRP sign in returned 412, attempting repair/complete")
return s.authRepairComplete(ctx)
case http.StatusForbidden:
return fmt.Errorf("sign in failed: incorrect username or password")
default:
return fmt.Errorf("sign in failed: %s", resp.Status)
return fmt.Errorf("sign in failed: %s: %s", resp.Status, respBody)
}
}
// getAuthOrigin returns the origin URL for auth requests.
// Supports both global (idmsa.apple.com) and China (idmsa.apple.com.cn) endpoints.
// authRepairComplete handles the repair flow for non-2FA accounts that return 412
func (s *Session) authRepairComplete(ctx context.Context) error {
body, err := IntoReader(map[string]any{})
if err != nil {
return err
}
opts := rest.Opts{
Method: "POST",
Path: "/repair/complete",
ExtraHeaders: s.getSRPAuthHeaders(),
RootURL: authEndpoint,
IgnoreStatus: true,
NoResponse: true,
Body: body,
}
resp, err := s.srv.Call(ctx, &opts)
if err != nil {
return fmt.Errorf("repair/complete failed: %w", err)
}
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("repair/complete returned %s", resp.Status)
}
s.extractHeaders(resp)
fs.Debugf(nil, "iclouddrive: repair/complete successful")
return nil
}
// getAuthOrigin returns the origin URL for auth requests
// Supports both global (idmsa.apple.com) and China (idmsa.apple.com.cn) endpoints
func getAuthOrigin() string {
return strings.TrimSuffix(authEndpoint, "/appleauth/auth")
}
// getSRPAuthHeaders returns headers needed for SRP auth requests.
// getSRPAuthHeaders returns headers needed for SRP auth requests
func (s *Session) getSRPAuthHeaders() map[string]string {
frameTag := "auth-" + s.FrameID
authOrigin := getAuthOrigin()
@@ -414,14 +516,110 @@ func (s *Session) AuthWithToken(ctx context.Context) error {
}
resp, err := s.Request(ctx, opts, nil, &s.AccountInfo)
if err == nil {
s.Cookies = resp.Cookies()
if err != nil {
return err
}
fs.Debugf(nil, "iclouddrive: accountLogin response cookies: %v", cookieDebugSummaries(resp.Cookies()))
fs.Debugf(nil, "iclouddrive: session cookie jar after accountLogin: %v", cookieJarDebugSummaries(s.Cookies))
// Acquire PCS cookies if Advanced Data Protection is enabled
if ws := s.AccountInfo.Webservices["ckdatabasews"]; ws != nil && ws.PcsRequired {
fs.Debugf(nil, "iclouddrive: ADP detected (pcsRequired=true)")
if s.hasPCSCookies() {
fs.Debugf(nil, "iclouddrive: PCS cookies already present, skipping acquisition")
} else {
if err := s.acquirePCSCookies(ctx); err != nil {
return err
}
}
} else {
fs.Debugf(nil, "iclouddrive: no ADP (pcsRequired=false)")
}
return nil
}
// hasPCSCookies checks if the required PCS cookies for Photos are already present
func (s *Session) hasPCSCookies() bool {
var hasPhotos, hasSharing bool
for _, c := range s.Cookies {
switch c.Name {
case "X-APPLE-WEBAUTH-PCS-Photos":
hasPhotos = true
case "X-APPLE-WEBAUTH-PCS-Sharing":
hasSharing = true
}
}
return hasPhotos && hasSharing
}
// acquirePCSCookies requests PCS cookies for ADP-enabled accounts
// May require user approval on a trusted device (polls every 10s, max 5 min)
func (s *Session) acquirePCSCookies(ctx context.Context) error {
fs.Logf(nil, "iclouddrive: Advanced Data Protection enabled, requesting PCS cookies")
const maxAttempts = 30 // 30 * 10s = 5 minutes max
for attempt := 0; attempt < maxAttempts; attempt++ {
fs.Debugf(nil, "iclouddrive: requestPCS outgoing cookies: %v", cookieJarDebugSummaries(s.Cookies))
values := map[string]any{
"appName": "photos",
"derivedFromUserAction": true,
}
body, err := IntoReader(values)
if err != nil {
return fmt.Errorf("requestPCS: %w", err)
}
opts := rest.Opts{
Method: "POST",
Path: "/requestPCS",
ExtraHeaders: s.GetHeaders(map[string]string{}),
RootURL: setupEndpoint,
}
opts.Body = body
var pcsResp struct {
Status string `json:"status"`
Message string `json:"message"`
}
resp, err := s.Request(ctx, opts, nil, &pcsResp)
if err != nil {
return fmt.Errorf("requestPCS: %w", err)
}
fs.Debugf(nil, "iclouddrive: requestPCS response cookies: %v", cookieDebugSummaries(resp.Cookies()))
fs.Debugf(nil, "iclouddrive: requestPCS response: status=%q message=%q cookies=%d",
pcsResp.Status, pcsResp.Message, len(resp.Cookies()))
if pcsResp.Status == "success" {
if !s.hasPCSCookies() {
return fmt.Errorf("requestPCS: server returned success but PCS cookies missing")
}
fs.Logf(nil, "iclouddrive: PCS cookies acquired")
return nil
}
// Device consent required - poll until approved
fs.Logf(nil, "iclouddrive: waiting for device approval for PCS (%s)", pcsResp.Message)
select {
case <-ctx.Done():
return ctx.Err()
case <-time.After(10 * time.Second):
}
}
return fmt.Errorf("requestPCS: timed out waiting for device approval after 5 minutes")
}
// RequestPushNotification explicitly requests a push notification to trusted devices
// Required for iOS 26.4+ where the SRP 409 response no longer auto-pushes
func (s *Session) RequestPushNotification(ctx context.Context) error {
opts := rest.Opts{
Method: "PUT",
Path: "/verify/trusteddevice/securitycode",
ExtraHeaders: s.GetAuthHeaders(map[string]string{}),
RootURL: authEndpoint,
NoResponse: true,
}
_, err := s.Request(ctx, opts, nil, nil)
return err
}
// Validate2FACode validates the 2FA code
// Validate2FACode validates the 2FA code from a trusted device push notification
func (s *Session) Validate2FACode(ctx context.Context, code string) error {
values := map[string]any{"securityCode": map[string]string{"code": code}}
body, err := IntoReader(values)
@@ -450,6 +648,132 @@ func (s *Session) Validate2FACode(ctx context.Context, code string) error {
return fmt.Errorf("validate2FACode failed: %w", err)
}
// TrustedPhoneNumber represents a phone number that can receive SMS verification codes
type TrustedPhoneNumber struct {
ID int `json:"id"`
NumberWithDialCode string `json:"numberWithDialCode"`
ObfuscatedNumber string `json:"obfuscatedNumber"`
PushMode string `json:"pushMode"`
NonFTEU bool `json:"nonFTEU"`
}
// AuthStateResponse is the response from GET /appleauth/auth after sign-in
// Some accounts return fields at top level, others nest them under phoneNumberVerification
type AuthStateResponse struct {
TrustedPhoneNumbers []TrustedPhoneNumber `json:"trustedPhoneNumbers"`
TrustedPhoneNumber *TrustedPhoneNumber `json:"trustedPhoneNumber"`
NoTrustedDevices bool `json:"noTrustedDevices"`
AuthenticationType string `json:"authenticationType"`
Hsa2Account bool `json:"hsa2Account"`
PhoneNumberVerification *AuthStateResponse `json:"phoneNumberVerification"`
}
// GetAuthState retrieves the current auth state including trusted phone numbers for SMS 2FA
func (s *Session) GetAuthState(ctx context.Context) (*AuthStateResponse, error) {
opts := rest.Opts{
Method: "GET",
Path: "",
ExtraHeaders: s.GetAuthHeaders(map[string]string{}),
RootURL: authEndpoint,
ContentLength: int64Ptr(0),
}
// Use srv.Call directly to capture the raw response body for debugging
resp, err := s.srv.Call(ctx, &opts)
if err != nil {
return nil, fmt.Errorf("getAuthState: %w", err)
}
s.mu.Lock()
s.extractHeaders(resp)
s.mu.Unlock()
body, readErr := io.ReadAll(resp.Body)
_ = resp.Body.Close()
if readErr != nil {
return nil, fmt.Errorf("getAuthState read: %w", readErr)
}
fs.Debugf(nil, "iclouddrive: auth state response summary: %s", authStateBodySummary(body))
var state AuthStateResponse
if err := json.Unmarshal(body, &state); err != nil {
fs.Debugf(nil, "iclouddrive: auth state parse failed: %s", authStateBodySummary(body))
return nil, fmt.Errorf("getAuthState unmarshal: %w", err)
}
// Some accounts nest auth data under phoneNumberVerification
if state.PhoneNumberVerification != nil && state.AuthenticationType == "" {
fs.Debugf(nil, "iclouddrive: unwrapping phoneNumberVerification envelope")
n := state.PhoneNumberVerification
state.TrustedPhoneNumbers = n.TrustedPhoneNumbers
state.TrustedPhoneNumber = n.TrustedPhoneNumber
state.NoTrustedDevices = n.NoTrustedDevices
state.AuthenticationType = n.AuthenticationType
state.Hsa2Account = n.Hsa2Account
state.PhoneNumberVerification = nil
}
fs.Debugf(nil, "iclouddrive: auth state: type=%s hsa2=%v noTrustedDevices=%v phones=%d phoneSingular=%v",
state.AuthenticationType, state.Hsa2Account, state.NoTrustedDevices,
len(state.TrustedPhoneNumbers), state.TrustedPhoneNumber != nil)
// Fall back to singular trustedPhoneNumber when plural array is empty
// Some accounts return only the singular form (SMS-only, no trusted devices)
if len(state.TrustedPhoneNumbers) == 0 && state.TrustedPhoneNumber != nil {
fs.Debugf(nil, "iclouddrive: using singular trustedPhoneNumber (id=%d) as fallback",
state.TrustedPhoneNumber.ID)
state.TrustedPhoneNumbers = []TrustedPhoneNumber{*state.TrustedPhoneNumber}
}
return &state, nil
}
// RequestSMSCode triggers SMS code delivery to a trusted phone number
func (s *Session) RequestSMSCode(ctx context.Context, phoneID int, mode string) error {
values := map[string]any{
"phoneNumber": map[string]any{"id": phoneID},
"mode": mode,
}
body, err := IntoReader(values)
if err != nil {
return err
}
opts := rest.Opts{
Method: "PUT",
Path: "/verify/phone",
ExtraHeaders: s.GetAuthHeaders(map[string]string{}),
RootURL: authEndpoint,
Body: body,
NoResponse: true,
}
_, err = s.Request(ctx, opts, nil, nil)
if err != nil {
return fmt.Errorf("requestSMSCode: %w", err)
}
return nil
}
// ValidateSMSCode validates a 2FA code received via SMS
func (s *Session) ValidateSMSCode(ctx context.Context, code string, phoneID int, mode string) error {
values := map[string]any{
"securityCode": map[string]string{"code": code},
"phoneNumber": map[string]any{"id": phoneID},
"mode": mode,
}
body, err := IntoReader(values)
if err != nil {
return err
}
opts := rest.Opts{
Method: "POST",
Path: "/verify/phone/securitycode",
ExtraHeaders: s.GetAuthHeaders(map[string]string{}),
RootURL: authEndpoint,
Body: body,
NoResponse: true,
}
_, err = s.Request(ctx, opts, nil, nil)
if err == nil {
if err := s.TrustSession(ctx); err != nil {
return err
}
return nil
}
return fmt.Errorf("validateSMSCode: %w", err)
}
// TrustSession trusts the session
func (s *Session) TrustSession(ctx context.Context) error {
opts := rest.Opts{
@@ -458,7 +782,7 @@ func (s *Session) TrustSession(ctx context.Context) error {
ExtraHeaders: s.GetAuthHeaders(map[string]string{}),
RootURL: authEndpoint,
NoResponse: true,
ContentLength: common.Int64(0),
ContentLength: int64Ptr(0),
}
_, err := s.Request(ctx, opts, nil, nil)
@@ -476,25 +800,26 @@ func (s *Session) ValidateSession(ctx context.Context) error {
Path: "/validate",
ExtraHeaders: s.GetHeaders(map[string]string{}),
RootURL: setupEndpoint,
ContentLength: common.Int64(0),
ContentLength: int64Ptr(0),
}
_, err := s.Request(ctx, opts, nil, &s.AccountInfo)
if err != nil {
return fmt.Errorf("validateSession failed: %w", err)
}
s.needs2FA = false
return nil
}
// GetAuthHeaders returns the authentication headers for the session.
// Used for 2FA validation and trust requests to idmsa.apple.com.
// GetAuthHeaders returns the authentication headers for the session
// Used for 2FA validation and trust requests to idmsa.apple.com
func (s *Session) GetAuthHeaders(overwrite map[string]string) map[string]string {
headers := s.getSRPAuthHeaders()
maps.Copy(headers, overwrite)
return headers
}
// GetHeaders Gets the authentication headers required for a request
// GetHeaders returns the authentication headers required for a request
func (s *Session) GetHeaders(overwrite map[string]string) map[string]string {
headers := GetCommonHeaders(map[string]string{})
headers["Cookie"] = s.GetCookieString()
@@ -502,17 +827,26 @@ func (s *Session) GetHeaders(overwrite map[string]string) map[string]string {
return headers
}
// GetCookieString returns the cookie header string for the session.
// GetCookieString returns the cookie header string for the session
func (s *Session) GetCookieString() string {
cookieHeader := ""
// we only care about name and value.
var b strings.Builder
first := true
for _, cookie := range s.Cookies {
cookieHeader = cookieHeader + cookie.Name + "=" + cookie.Value + ";"
if cookie.Value == "" {
continue
}
if !first {
b.WriteString("; ")
}
first = false
b.WriteString(cookie.Name)
b.WriteByte('=')
b.WriteString(cookie.Value)
}
return cookieHeader
return b.String()
}
// GetCommonHeaders generates common HTTP headers with optional overwrite.
// GetCommonHeaders generates common HTTP headers with optional overwrite
func GetCommonHeaders(overwrite map[string]string) map[string]string {
headers := map[string]string{
"Content-Type": "application/json",
@@ -524,32 +858,9 @@ func GetCommonHeaders(overwrite map[string]string) map[string]string {
return headers
}
// MergeCookies merges two slices of http.Cookies, ensuring no duplicates are added.
func MergeCookies(left []*http.Cookie, right []*http.Cookie) ([]*http.Cookie, error) {
var hashes []string
for _, cookie := range right {
hashes = append(hashes, cookie.Raw)
}
for _, cookie := range left {
if !slices.Contains(hashes, cookie.Raw) {
right = append(right, cookie)
}
}
return right, nil
}
func int64Ptr(v int64) *int64 { return &v }
// GetCookiesForDomain filters the provided cookies based on the domain of the given URL.
func GetCookiesForDomain(url *url.URL, cookies []*http.Cookie) ([]*http.Cookie, error) {
var domainCookies []*http.Cookie
for _, cookie := range cookies {
if strings.HasSuffix(url.Host, cookie.Domain) {
domainCookies = append(domainCookies, cookie)
}
}
return domainCookies, nil
}
// NewSession creates a new Session instance with default values.
// NewSession creates a new Session instance with default values
func NewSession() *Session {
session := &Session{
FrameID: strings.ToLower(uuid.New().String()),
@@ -564,117 +875,20 @@ func NewSession() *Session {
return session
}
// AccountInfo represents an account info
// AccountInfo represents the subset of Apple's account response we actually use
// json.Unmarshal silently ignores extra fields in the response
type AccountInfo struct {
DsInfo *ValidateDataDsInfo `json:"dsInfo"`
HasMinimumDeviceForPhotosWeb bool `json:"hasMinimumDeviceForPhotosWeb"`
ICDPEnabled bool `json:"iCDPEnabled"`
Webservices map[string]*webService `json:"webservices"`
PcsEnabled bool `json:"pcsEnabled"`
TermsUpdateNeeded bool `json:"termsUpdateNeeded"`
ConfigBag struct {
Urls struct {
AccountCreateUI string `json:"accountCreateUI"`
AccountLoginUI string `json:"accountLoginUI"`
AccountLogin string `json:"accountLogin"`
AccountRepairUI string `json:"accountRepairUI"`
DownloadICloudTerms string `json:"downloadICloudTerms"`
RepairDone string `json:"repairDone"`
AccountAuthorizeUI string `json:"accountAuthorizeUI"`
VettingURLForEmail string `json:"vettingUrlForEmail"`
AccountCreate string `json:"accountCreate"`
GetICloudTerms string `json:"getICloudTerms"`
VettingURLForPhone string `json:"vettingUrlForPhone"`
} `json:"urls"`
AccountCreateEnabled bool `json:"accountCreateEnabled"`
} `json:"configBag"`
HsaTrustedBrowser bool `json:"hsaTrustedBrowser"`
AppsOrder []string `json:"appsOrder"`
Version int `json:"version"`
IsExtendedLogin bool `json:"isExtendedLogin"`
PcsServiceIdentitiesIncluded bool `json:"pcsServiceIdentitiesIncluded"`
IsRepairNeeded bool `json:"isRepairNeeded"`
HsaChallengeRequired bool `json:"hsaChallengeRequired"`
RequestInfo struct {
Country string `json:"country"`
TimeZone string `json:"timeZone"`
Region string `json:"region"`
} `json:"requestInfo"`
PcsDeleted bool `json:"pcsDeleted"`
ICloudInfo struct {
SafariBookmarksHasMigratedToCloudKit bool `json:"SafariBookmarksHasMigratedToCloudKit"`
} `json:"iCloudInfo"`
Apps map[string]*ValidateDataApp `json:"apps"`
DsInfo *dsInfo `json:"dsInfo"`
Webservices map[string]*webService `json:"webservices"`
HsaChallengeRequired bool `json:"hsaChallengeRequired"`
}
// ValidateDataDsInfo represents an validation info
type ValidateDataDsInfo struct {
HsaVersion int `json:"hsaVersion"`
LastName string `json:"lastName"`
ICDPEnabled bool `json:"iCDPEnabled"`
TantorMigrated bool `json:"tantorMigrated"`
Dsid string `json:"dsid"`
HsaEnabled bool `json:"hsaEnabled"`
IsHideMyEmailSubscriptionActive bool `json:"isHideMyEmailSubscriptionActive"`
IroncadeMigrated bool `json:"ironcadeMigrated"`
Locale string `json:"locale"`
BrZoneConsolidated bool `json:"brZoneConsolidated"`
ICDRSCapableDeviceList string `json:"ICDRSCapableDeviceList"`
IsManagedAppleID bool `json:"isManagedAppleID"`
IsCustomDomainsFeatureAvailable bool `json:"isCustomDomainsFeatureAvailable"`
IsHideMyEmailFeatureAvailable bool `json:"isHideMyEmailFeatureAvailable"`
ContinueOnDeviceEligibleDeviceInfo []string `json:"ContinueOnDeviceEligibleDeviceInfo"`
Gilligvited bool `json:"gilligvited"`
AppleIDAliases []any `json:"appleIdAliases"`
UbiquityEOLEnabled bool `json:"ubiquityEOLEnabled"`
IsPaidDeveloper bool `json:"isPaidDeveloper"`
CountryCode string `json:"countryCode"`
NotificationID string `json:"notificationId"`
PrimaryEmailVerified bool `json:"primaryEmailVerified"`
ADsID string `json:"aDsID"`
Locked bool `json:"locked"`
ICDRSCapableDeviceCount int `json:"ICDRSCapableDeviceCount"`
HasICloudQualifyingDevice bool `json:"hasICloudQualifyingDevice"`
PrimaryEmail string `json:"primaryEmail"`
AppleIDEntries []struct {
IsPrimary bool `json:"isPrimary"`
Type string `json:"type"`
Value string `json:"value"`
} `json:"appleIdEntries"`
GilliganEnabled bool `json:"gilligan-enabled"`
IsWebAccessAllowed bool `json:"isWebAccessAllowed"`
FullName string `json:"fullName"`
MailFlags struct {
IsThreadingAvailable bool `json:"isThreadingAvailable"`
IsSearchV2Provisioned bool `json:"isSearchV2Provisioned"`
SCKMail bool `json:"sCKMail"`
IsMppSupportedInCurrentCountry bool `json:"isMppSupportedInCurrentCountry"`
} `json:"mailFlags"`
LanguageCode string `json:"languageCode"`
AppleID string `json:"appleId"`
HasUnreleasedOS bool `json:"hasUnreleasedOS"`
AnalyticsOptInStatus bool `json:"analyticsOptInStatus"`
FirstName string `json:"firstName"`
ICloudAppleIDAlias string `json:"iCloudAppleIdAlias"`
NotesMigrated bool `json:"notesMigrated"`
BeneficiaryInfo struct {
IsBeneficiary bool `json:"isBeneficiary"`
} `json:"beneficiaryInfo"`
HasPaymentInfo bool `json:"hasPaymentInfo"`
PcsDelet bool `json:"pcsDelet"`
AppleIDAlias string `json:"appleIdAlias"`
BrMigrated bool `json:"brMigrated"`
StatusCode int `json:"statusCode"`
FamilyEligible bool `json:"familyEligible"`
// dsInfo holds the account metadata fields we read
type dsInfo struct {
HsaVersion int `json:"hsaVersion"`
}
// ValidateDataApp represents an app
type ValidateDataApp struct {
CanLaunchWithOneFactor bool `json:"canLaunchWithOneFactor"`
IsQualifiedForBeta bool `json:"isQualifiedForBeta"`
}
// WebService represents a web service
// webService represents a web service
type webService struct {
PcsRequired bool `json:"pcsRequired"`
URL string `json:"url"`
+40
View File
@@ -0,0 +1,40 @@
package api
import (
"net/http"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestExtractHeadersMergesCookies(t *testing.T) {
s := NewSession()
s.Cookies = []*http.Cookie{{Name: "existing", Value: "old"}}
resp := &http.Response{Header: make(http.Header)}
resp.Header.Add("Set-Cookie", (&http.Cookie{Name: "existing", Value: "new"}).String())
resp.Header.Add("Set-Cookie", (&http.Cookie{Name: "fresh", Value: "value"}).String())
resp.Header.Set("X-Apple-Session-Token", "session-token")
s.extractHeaders(resp)
require.Len(t, s.Cookies, 2)
assert.Equal(t, "new", s.Cookies[0].Value)
assert.Equal(t, "fresh", s.Cookies[1].Name)
assert.Equal(t, "session-token", s.SessionToken)
}
func TestExtractHeadersDeletesEmptyCookies(t *testing.T) {
s := NewSession()
s.Cookies = []*http.Cookie{{Name: "X-APPLE-WEBAUTH-HSA-LOGIN", Value: "stale"}, {Name: "keep", Value: "value"}}
resp := &http.Response{Header: make(http.Header)}
resp.Header.Add("Set-Cookie", (&http.Cookie{Name: "X-APPLE-WEBAUTH-HSA-LOGIN", Value: ""}).String())
s.extractHeaders(resp)
require.Len(t, s.Cookies, 1)
assert.Equal(t, "keep", s.Cookies[0].Name)
assert.Equal(t, "keep=value", s.GetCookieString())
}
+17 -9
View File
@@ -46,10 +46,10 @@ type srpClient struct {
}
// newSRPClient generates a random 32-byte secret and computes the public value A.
func newSRPClient() *srpClient {
func newSRPClient() (*srpClient, error) {
secret := make([]byte, 32)
if _, err := rand.Read(secret); err != nil {
panic(fmt.Sprintf("srp: rand.Read failed: %v", err))
return nil, fmt.Errorf("srp: rand.Read failed: %w", err)
}
a := new(big.Int).SetBytes(secret)
@@ -60,7 +60,7 @@ func newSRPClient() *srpClient {
a: a,
A: A,
k: k,
}
}, nil
}
// getABytes returns the padded public value A.
@@ -71,16 +71,21 @@ func (c *srpClient) getABytes() []byte {
// processChallenge computes the session key and proof values from the server challenge.
// username is the Apple ID, derivedKey is the output of derivePassword, salt and B come
// from the server's init response (raw bytes, already base64-decoded).
func (c *srpClient) processChallenge(username, derivedKey, salt, serverB []byte) {
func (c *srpClient) processChallenge(username, derivedKey, salt, serverB []byte) error {
B := new(big.Int).SetBytes(serverB)
// Validate B
// Validate B (RFC 5054)
if B.Cmp(big.NewInt(0)) <= 0 || B.Cmp(srpN) >= 0 {
panic("srp: invalid server-supplied B, must be 1..N-1")
return fmt.Errorf("srp: invalid server-supplied B, must be 1..N-1")
}
x := calculateX(salt, derivedKey)
u := calculateU(c.A, B)
// Validate u (RFC 5054)
if u.Sign() == 0 {
return fmt.Errorf("srp: calculated u is zero, aborting")
}
S := calculateS(c.k, x, c.a, B, u)
c.K = calculateK(S)
@@ -88,6 +93,7 @@ func (c *srpClient) processChallenge(username, derivedKey, salt, serverB []byte)
bBytes := padToN(B)
c.M1 = calculateM1(username, salt, aBytes, bBytes, c.K)
c.M2 = calculateM2(aBytes, c.M1, c.K)
return nil
}
// derivePassword performs Apple's password key derivation.
@@ -98,10 +104,12 @@ func derivePassword(password string, salt []byte, iterations int, protocol strin
var passInput string
switch protocol {
case "s2k":
passInput = string(passHash[:])
case "s2k_fo":
passInput = hex.EncodeToString(passHash[:])
default: // "s2k"
passInput = string(passHash[:])
default:
return nil, fmt.Errorf("unsupported SRP protocol: %q", protocol)
}
return pbkdf2.Key(sha256.New, passInput, salt, iterations, 32)
}
@@ -142,7 +150,7 @@ func getMultiplier() *big.Int {
}
// calculateX computes x = H(salt | H(":" | password))
// Apple's variant: NoUserNameInX — username is omitted from the inner hash.
// Apple's variant: NoUserNameInX - username is omitted from the inner hash.
// The "password" here is actually the derived key from derivePassword.
func calculateX(salt, derivedKey []byte) *big.Int {
h := srpHashFunc()
+39 -7
View File
@@ -27,7 +27,7 @@ func TestPadToN(t *testing.T) {
})
t.Run("preserves value that fills N", func(t *testing.T) {
// Use N itself — already 256 bytes
// Use N itself - already 256 bytes
result := padToN(srpN)
assert.Equal(t, srpNLenBytes, len(result))
assert.Equal(t, srpN.Bytes(), result)
@@ -52,7 +52,7 @@ func TestGetMultiplier(t *testing.T) {
k2 := getMultiplier()
assert.Equal(t, 0, k.Cmp(k2), "multiplier must be deterministic")
// k = H(N | pad(g)) — verify by manual computation
// k = H(N | pad(g)) - verify by manual computation
h := srpHashFunc()
nBytes := srpN.Bytes()
gBytes := srpG.Bytes()
@@ -123,6 +123,12 @@ func TestDerivePassword(t *testing.T) {
require.NoError(t, err)
assert.Equal(t, 32, len(key))
})
t.Run("unknown protocol returns error", func(t *testing.T) {
_, err := derivePassword(password, salt, 1000, "unknown")
assert.Error(t, err)
assert.Contains(t, err.Error(), "unsupported SRP protocol")
})
}
func TestCalculateX(t *testing.T) {
@@ -256,7 +262,8 @@ func TestCalculateM2(t *testing.T) {
}
func TestNewSRPClient(t *testing.T) {
client := newSRPClient()
client, err := newSRPClient()
require.NoError(t, err)
// A must be non-zero and in range [1, N-1]
assert.True(t, client.A.Sign() > 0, "A must be positive")
@@ -273,7 +280,8 @@ func TestNewSRPClient(t *testing.T) {
assert.Equal(t, 0, client.A.Cmp(recoveredA))
// Two clients should have different secrets (probabilistic, but 2^256 collision chance)
client2 := newSRPClient()
client2, err := newSRPClient()
require.NoError(t, err)
assert.NotEqual(t, 0, client.a.Cmp(client2.a), "two clients must have different secrets")
}
@@ -315,7 +323,8 @@ func TestProcessChallenge(t *testing.T) {
serverB := padToN(B)
client.processChallenge(username, derivedKey, salt, serverB)
err := client.processChallenge(username, derivedKey, salt, serverB)
require.NoError(t, err)
// M1, M2, K must be populated
assert.NotNil(t, client.M1)
@@ -328,13 +337,14 @@ func TestProcessChallenge(t *testing.T) {
// M1 and M2 must be different
assert.NotEqual(t, client.M1, client.M2)
// Results must be deterministic — run again with same inputs
// Results must be deterministic - run again with same inputs
client2 := &srpClient{
a: new(big.Int).Set(a),
A: new(big.Int).Set(A),
k: new(big.Int).Set(k),
}
client2.processChallenge(username, derivedKey, salt, serverB)
err = client2.processChallenge(username, derivedKey, salt, serverB)
require.NoError(t, err)
assert.Equal(t, client.M1, client2.M1, "M1 must be deterministic")
assert.Equal(t, client.M2, client2.M2, "M2 must be deterministic")
@@ -352,6 +362,28 @@ func TestProcessChallenge(t *testing.T) {
assert.Equal(t, client.K, kServer, "client and server must derive the same session key")
}
func TestProcessChallenge_InvalidB(t *testing.T) {
k := getMultiplier()
client := &srpClient{
a: big.NewInt(42),
A: new(big.Int).Exp(srpG, big.NewInt(42), srpN),
k: k,
}
username := []byte("test@apple.com")
derivedKey := make([]byte, 32)
salt := []byte("salt")
t.Run("B=0 rejected", func(t *testing.T) {
err := client.processChallenge(username, derivedKey, salt, padToN(big.NewInt(0)))
assert.Error(t, err)
})
t.Run("B=N rejected", func(t *testing.T) {
err := client.processChallenge(username, derivedKey, salt, padToN(srpN))
assert.Error(t, err)
})
}
func TestSRPGroupParameters(t *testing.T) {
// Verify the 2048-bit prime starts with known hex prefix
nHex := hex.EncodeToString(srpN.Bytes())
+495
View File
@@ -0,0 +1,495 @@
//go:build !plan9 && !solaris
// Package iclouddrive provides access to iCloud Drive and Photos
package iclouddrive
import (
"context"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"net/http"
"os"
"strconv"
"strings"
"github.com/rclone/rclone/backend/iclouddrive/api"
"github.com/rclone/rclone/fs"
"github.com/rclone/rclone/fs/config"
"github.com/rclone/rclone/fs/config/configmap"
"github.com/rclone/rclone/fs/config/configstruct"
"github.com/rclone/rclone/fs/config/obscure"
"github.com/rclone/rclone/lib/encoder"
)
const configAuthSession = "_auth_session"
// configAuthState holds session fields that must be preserved across
// Config() state machine steps (push 2FA and SMS both need it)
type configAuthState struct {
Scnt string `json:"s"`
SessionID string `json:"i"`
AuthAttributes string `json:"a"`
FrameID string `json:"f"`
SessionToken string `json:"t"`
ClientID string `json:"c"`
AccountCountry string `json:"ac"`
Phones []smsPhone `json:"p,omitempty"`
}
type smsPhone struct {
ID int `json:"id"`
Num string `json:"n"`
Mode string `json:"m"`
}
func saveAuthSession(m configmap.Mapper, s *api.Session, phones []smsPhone) {
st := configAuthState{
Scnt: s.Scnt,
SessionID: s.SessionID,
AuthAttributes: s.AuthAttributes,
FrameID: s.FrameID,
SessionToken: s.SessionToken,
ClientID: s.ClientID,
AccountCountry: s.AccountCountry,
Phones: phones,
}
data, err := json.Marshal(st)
if err != nil {
fs.Debugf(nil, "iclouddrive: failed to marshal auth session: %v", err)
return
}
m.Set(configAuthSession, base64.StdEncoding.EncodeToString(data))
}
func loadAuthSession(m configmap.Mapper) (*configAuthState, error) {
raw, _ := m.Get(configAuthSession)
if raw == "" {
return nil, errors.New("auth session state lost, please reconfigure")
}
data, err := base64.StdEncoding.DecodeString(raw)
if err != nil {
return nil, fmt.Errorf("corrupt auth session state: %w", err)
}
var st configAuthState
if err := json.Unmarshal(data, &st); err != nil {
return nil, fmt.Errorf("invalid auth session state: %w", err)
}
return &st, nil
}
func restoreAuthSession(icloud *api.Client, st *configAuthState) {
icloud.Session.Scnt = st.Scnt
icloud.Session.SessionID = st.SessionID
icloud.Session.AuthAttributes = st.AuthAttributes
icloud.Session.FrameID = st.FrameID
icloud.Session.SessionToken = st.SessionToken
icloud.Session.ClientID = st.ClientID
icloud.Session.AccountCountry = st.AccountCountry
}
// resumeConfigClient recreates an API client and restores session state saved
// from a previous Config() step. Used across 2FA states to avoid re-authenticating
func resumeConfigClient(m configmap.Mapper, appleid, password, trustToken, clientID string, cookies []*http.Cookie) (*api.Client, *configAuthState, error) {
st, err := loadAuthSession(m)
if err != nil {
return nil, nil, err
}
icloud, err := api.New(appleid, password, trustToken, clientID, cookies, nil, "_config")
if err != nil {
return nil, nil, err
}
restoreAuthSession(icloud, st)
return icloud, st, nil
}
// saveAuthCredentials persists trust token, cookies, and clears session state
// after successful 2FA validation
func saveAuthCredentials(m configmap.Mapper, icloud *api.Client, name string) {
m.Set(configTrustToken, icloud.Session.TrustToken)
m.Set(configCookies, icloud.Session.GetCookieString())
m.Set(configAuthSession, "")
api.ClearCacheDir(name)
}
// triggerSMSFlow handles phone selection and SMS triggering
// Single phone: triggers SMS immediately, returns code prompt
// Multiple phones: saves session + phone list, returns phone picker
func triggerSMSFlow(ctx context.Context, icloud *api.Client, phones []api.TrustedPhoneNumber, m configmap.Mapper) (*fs.ConfigOut, error) {
smsPhones := make([]smsPhone, len(phones))
for i, p := range phones {
mode := p.PushMode
if mode == "" {
mode = "sms"
}
smsPhones[i] = smsPhone{ID: p.ID, Num: p.NumberWithDialCode, Mode: mode}
}
if len(phones) == 1 {
p := smsPhones[0]
if err := icloud.Session.RequestSMSCode(ctx, p.ID, p.Mode); err != nil {
return nil, fmt.Errorf("failed to send SMS code: %w", err)
}
saveAuthSession(m, icloud.Session, nil)
nextState := fmt.Sprintf("2fa_sms_%d_%s", p.ID, p.Mode)
return fs.ConfigInput(nextState, "config_2fa_sms", fmt.Sprintf("Enter the verification code sent to %s", p.Num))
}
// Multiple phones - save session and present picker
saveAuthSession(m, icloud.Session, smsPhones)
items := make([]fs.OptionExample, len(smsPhones))
for i, p := range smsPhones {
items[i] = fs.OptionExample{
Value: fmt.Sprintf("%d_%s", p.ID, p.Mode),
Help: p.Num,
}
}
return fs.ConfigChooseExclusiveFixed("2fa_sms_select", "config_2fa_phone", "Select phone number for SMS verification", items)
}
const (
configService = "service"
// Service types
serviceDrive = "drive"
servicePhotos = "photos"
)
// ServiceOptions defines the configuration for service selection
type ServiceOptions struct {
Service string `config:"service"`
}
// Register with rclone
func init() {
fs.Register(&fs.RegInfo{
Name: "iclouddrive",
Description: "iCloud Drive and Photos",
Config: Config,
NewFs: NewServiceFs,
MetadataInfo: &fs.MetadataInfo{
System: map[string]fs.MetadataHelp{
"width": {
Help: "Image width in pixels",
Type: "int",
ReadOnly: true,
},
"height": {
Help: "Image height in pixels",
Type: "int",
ReadOnly: true,
},
"added-time": {
Help: "Time the item was added to the iCloud library",
Type: "RFC 3339",
Example: "2006-01-02T15:04:05Z",
ReadOnly: true,
},
"favorite": {
Help: "Whether the item is marked as favorite",
Type: "bool",
ReadOnly: true,
},
"hidden": {
Help: "Whether the item is hidden",
Type: "bool",
ReadOnly: true,
},
},
Help: "Metadata is read-only and available for the Photos service only.",
},
Options: []fs.Option{{
Name: configService,
Help: "iCloud service to use.",
Required: true,
Default: serviceDrive,
Examples: []fs.OptionExample{{
Value: serviceDrive,
Help: "iCloud Drive",
}, {
Value: servicePhotos,
Help: "iCloud Photos",
}},
}, {
Name: configAppleID,
Help: "Apple ID.",
Required: true,
Sensitive: true,
}, {
Name: configPassword,
Help: "Password.",
Required: true,
IsPassword: true,
Sensitive: true,
}, {
Name: configTrustToken,
Help: "Trust token for session authentication.",
IsPassword: false,
Required: false,
Sensitive: true,
Hide: fs.OptionHideBoth,
}, {
Name: configCookies,
Help: "Session cookies.",
Required: false,
Advanced: false,
Sensitive: true,
Hide: fs.OptionHideBoth,
}, {
Name: configClientID,
Help: "Client ID for iCloud API access.",
Required: false,
Advanced: true,
Default: "d39ba9916b7251055b22c7f910e2ea796ee65e98b2ddecea8f5dde8d9d1a815d",
}, {
Name: config.ConfigEncoding,
Help: config.ConfigEncodingHelp,
Advanced: true,
Default: (encoder.Display |
encoder.EncodeBackSlash |
encoder.EncodeSlash |
encoder.EncodeInvalidUtf8),
}},
})
}
// Config handles the authentication and configuration flow
func Config(ctx context.Context, name string, m configmap.Mapper, config fs.ConfigIn) (*fs.ConfigOut, error) {
var err error
appleid, _ := m.Get(configAppleID)
if appleid == "" {
return nil, errors.New("an Apple ID is required")
}
password, _ := m.Get(configPassword)
if password != "" {
password, err = obscure.Reveal(password)
if err != nil {
return nil, err
}
}
trustToken, _ := m.Get(configTrustToken)
cookieRaw, _ := m.Get(configCookies)
clientID, _ := m.Get(configClientID)
cookies := ReadCookies(cookieRaw)
switch {
case config.State == "":
// Force fresh SRP authentication - ignore stale trust token and cookies
// so that reconnect always prompts for 2FA
m.Set(configAuthSession, "")
icloud, err := api.New(appleid, password, "", clientID, nil, nil, "_config")
if err != nil {
return nil, err
}
if err := icloud.Authenticate(ctx); err != nil {
return nil, err
}
m.Set(configCookies, icloud.Session.GetCookieString())
if icloud.Session.Requires2FA() {
// Check if user has no trusted devices - auto-trigger SMS if so
authState, err := icloud.Session.GetAuthState(ctx)
if err == nil && authState.NoTrustedDevices && len(authState.TrustedPhoneNumbers) > 0 {
return triggerSMSFlow(ctx, icloud, authState.TrustedPhoneNumbers, m)
}
// Explicitly request push to trusted devices - required for iOS 26.4+
// where the SRP 409 no longer auto-pushes. GET /appleauth/auth above
// may also trigger a push (cosmetic double on pre-26.4, harmless)
if err := icloud.Session.RequestPushNotification(ctx); err != nil {
fs.Debugf(nil, "iclouddrive: push notification request failed (SMS fallback available): %v", err)
} else {
fs.Debugf(nil, "iclouddrive: push notification requested to trusted devices")
}
// Save session state so 2fa_do can validate without re-authenticating
// Push codes are account-scoped so session reuse is not strictly required,
// but it avoids a redundant SRP roundtrip and a second push on pre-26.4
saveAuthSession(m, icloud.Session, nil)
return fs.ConfigInput("2fa_do", "config_2fa", "Two-factor authentication: enter your 2FA code or type 'sms' for a text message")
}
// Auth succeeded without 2FA - save updated credentials and clear stale cache
m.Set(configTrustToken, icloud.Session.TrustToken)
api.ClearCacheDir(name)
return nil, nil
case config.State == "2fa_do":
code := config.Result
if code == "" {
return fs.ConfigError("authenticate", "2FA codes can't be blank")
}
// Restore session from initial sign-in instead of re-authenticating
// This avoids a redundant SRP roundtrip and extra push on pre-26.4
icloud, _, err := resumeConfigClient(m, appleid, password, trustToken, clientID, cookies)
if err != nil {
return nil, err
}
if strings.EqualFold(code, "sms") {
authState, err := icloud.Session.GetAuthState(ctx)
if err != nil {
return nil, fmt.Errorf("failed to get trusted phone numbers: %w", err)
}
if len(authState.TrustedPhoneNumbers) == 0 {
return nil, errors.New("no trusted phone numbers on this account")
}
return triggerSMSFlow(ctx, icloud, authState.TrustedPhoneNumbers, m)
}
if err := icloud.Session.Validate2FACode(ctx, code); err != nil {
return nil, err
}
saveAuthCredentials(m, icloud, name)
return nil, nil
case config.State == "2fa_sms_select":
// User selected a phone from ConfigChooseExclusiveFixed (value: "ID_mode")
idStr, mode, _ := strings.Cut(config.Result, "_")
phoneID, err := strconv.Atoi(idStr)
if err != nil {
m.Set(configAuthSession, "")
return nil, fmt.Errorf("invalid phone selection %q", config.Result)
}
if mode == "" {
mode = "sms"
}
icloud, smsState, err := resumeConfigClient(m, appleid, password, trustToken, clientID, cookies)
if err != nil {
return nil, err
}
// Find the selected phone's display number
var phoneNum string
for _, p := range smsState.Phones {
if p.ID == phoneID {
phoneNum = p.Num
break
}
}
if err := icloud.Session.RequestSMSCode(ctx, phoneID, mode); err != nil {
m.Set(configAuthSession, "")
return nil, fmt.Errorf("failed to send SMS code: %w", err)
}
saveAuthSession(m, icloud.Session, nil)
nextState := fmt.Sprintf("2fa_sms_%d_%s", phoneID, mode)
return fs.ConfigInput(nextState, "config_2fa_sms", fmt.Sprintf("Enter the verification code sent to %s", phoneNum))
case strings.HasPrefix(config.State, "2fa_sms_"):
code := config.Result
if code == "" {
return fs.ConfigError("authenticate", "SMS code can't be blank")
}
// State encodes phone ID and mode: "2fa_sms_<ID>_<mode>"
suffix := strings.TrimPrefix(config.State, "2fa_sms_")
idStr, mode, _ := strings.Cut(suffix, "_")
phoneID, err := strconv.Atoi(idStr)
if err != nil {
return nil, fmt.Errorf("invalid phone ID in state %q: %w", config.State, err)
}
if mode == "" {
mode = "sms"
}
icloud, _, err := resumeConfigClient(m, appleid, password, trustToken, clientID, cookies)
if err != nil {
return nil, err
}
if err := icloud.Session.ValidateSMSCode(ctx, code, phoneID, mode); err != nil {
m.Set(configAuthSession, "")
return nil, err
}
saveAuthCredentials(m, icloud, name)
return nil, nil
default:
return nil, fmt.Errorf("unknown state %q", config.State)
}
}
// newICloudClient parses options, authenticates, and returns a ready client
// Shared between NewFs (Drive) and NewFsPhotos (Photos) to avoid duplication
func newICloudClient(ctx context.Context, name string, m configmap.Mapper) (*api.Client, *Options, error) {
opt := new(Options)
err := configstruct.Set(m, opt)
if err != nil {
return nil, nil, err
}
if opt.Password != "" {
opt.Password, err = obscure.Reveal(opt.Password)
if err != nil {
return nil, nil, fmt.Errorf("couldn't decrypt user password: %w", err)
}
}
if opt.TrustToken == "" {
return nil, nil, fmt.Errorf("missing icloud trust token: try refreshing it with \"rclone config reconnect %s:\"", name)
}
cookies := ReadCookies(opt.Cookies)
callback := func(session *api.Session) {
m.Set(configCookies, session.GetCookieString())
}
icloud, err := api.New(
opt.AppleID,
opt.Password,
opt.TrustToken,
opt.ClientID,
cookies,
callback,
name,
)
if err != nil {
return nil, nil, err
}
if err := icloud.Authenticate(ctx); err != nil {
return nil, nil, err
}
if icloud.Session.Requires2FA() {
return nil, nil, errors.New("trust token expired, please reauth")
}
return icloud, opt, nil
}
// disconnectClient clears authentication state and removes disk caches
// Shared between Drive Fs and Photos Fs Disconnect() implementations
func disconnectClient(m configmap.Mapper, icloud *api.Client) error {
m.Set(configTrustToken, "")
m.Set(configCookies, "")
m.Set(configAuthSession, "")
return os.RemoveAll(icloud.CacheDir())
}
// NewServiceFs creates a filesystem instance for the selected service
func NewServiceFs(ctx context.Context, name, root string, m configmap.Mapper) (fs.Fs, error) {
// Parse the service selection
opt := new(ServiceOptions)
err := configstruct.Set(m, opt)
if err != nil {
return nil, err
}
// Set default service if not specified
if opt.Service == "" {
opt.Service = serviceDrive
}
// Route to the appropriate backend
switch opt.Service {
case serviceDrive:
// Create Drive filesystem
return NewFs(ctx, name, root, m)
case servicePhotos:
// Create Photos filesystem
return NewFsPhotos(ctx, name, root, m)
default:
return nil, fmt.Errorf("invalid service selection: %s (must be 'drive' or 'photos')", opt.Service)
}
}
+9 -162
View File
@@ -9,17 +9,13 @@ import (
"path"
"errors"
"fmt"
"io"
"net/http"
"strings"
"time"
"github.com/rclone/rclone/fs"
"github.com/rclone/rclone/fs/config"
"github.com/rclone/rclone/fs/config/configmap"
"github.com/rclone/rclone/fs/config/configstruct"
"github.com/rclone/rclone/fs/config/obscure"
"github.com/rclone/rclone/fs/fserrors"
"github.com/rclone/rclone/backend/iclouddrive/api"
@@ -51,61 +47,10 @@ const (
decayConstant = 2
)
// Register with Fs
func init() {
fs.Register(&fs.RegInfo{
Name: "iclouddrive",
Description: "iCloud Drive",
Config: Config,
NewFs: NewFs,
Options: []fs.Option{{
Name: configAppleID,
Help: "Apple ID.",
Required: true,
Sensitive: true,
}, {
Name: configPassword,
Help: "Password.",
Required: true,
IsPassword: true,
Sensitive: true,
}, {
Name: configTrustToken,
Help: "Trust token (internal use)",
IsPassword: false,
Required: false,
Sensitive: true,
Hide: fs.OptionHideBoth,
}, {
Name: configCookies,
Help: "cookies (internal use only)",
Required: false,
Advanced: false,
Sensitive: true,
Hide: fs.OptionHideBoth,
}, {
Name: configClientID,
Help: "Client id",
Required: false,
Advanced: true,
Default: "d39ba9916b7251055b22c7f910e2ea796ee65e98b2ddecea8f5dde8d9d1a815d",
}, {
Name: config.ConfigEncoding,
Help: config.ConfigEncodingHelp,
Advanced: true,
Default: (encoder.Display |
//encoder.EncodeDot |
encoder.EncodeBackSlash |
encoder.EncodeInvalidUtf8),
}},
})
}
// Options defines the configuration for this backend
type Options struct {
AppleID string `config:"apple_id"`
Password string `config:"password"`
Photos bool `config:"photos"`
TrustToken string `config:"trust_token"`
Cookies string `config:"cookies"`
ClientID string `config:"client_id"`
@@ -118,6 +63,7 @@ type Fs struct {
root string // the path we are working on.
rootID string
opt Options // parsed config options
m configmap.Mapper // config map for persisting auth state
features *fs.Features // optional features
dirCache *dircache.DirCache // Map of directory path to directory id
icloud *api.Client
@@ -139,72 +85,6 @@ type Object struct {
downloadURL string
}
// Config configures the iCloud remote.
func Config(ctx context.Context, name string, m configmap.Mapper, config fs.ConfigIn) (*fs.ConfigOut, error) {
var err error
appleid, _ := m.Get(configAppleID)
if appleid == "" {
return nil, errors.New("a apple ID is required")
}
password, _ := m.Get(configPassword)
if password != "" {
password, err = obscure.Reveal(password)
if err != nil {
return nil, err
}
}
trustToken, _ := m.Get(configTrustToken)
cookieRaw, _ := m.Get(configCookies)
clientID, _ := m.Get(configClientID)
cookies := ReadCookies(cookieRaw)
switch config.State {
case "":
icloud, err := api.New(appleid, password, trustToken, clientID, cookies, nil)
if err != nil {
return nil, err
}
if err := icloud.Authenticate(ctx); err != nil {
return nil, err
}
m.Set(configCookies, icloud.Session.GetCookieString())
if icloud.Session.Requires2FA() {
return fs.ConfigInput("2fa_do", "config_2fa", "Two-factor authentication: please enter your 2FA code")
}
return nil, nil
case "2fa_do":
code := config.Result
if code == "" {
return fs.ConfigError("authenticate", "2FA codes can't be blank")
}
icloud, err := api.New(appleid, password, trustToken, clientID, cookies, nil)
if err != nil {
return nil, err
}
if err := icloud.SignIn(ctx); err != nil {
return nil, err
}
if err := icloud.Session.Validate2FACode(ctx, code); err != nil {
return nil, err
}
m.Set(configTrustToken, icloud.Session.TrustToken)
m.Set(configCookies, icloud.Session.GetCookieString())
return nil, nil
case "2fa_error":
if config.Result == "true" {
return fs.ConfigGoto("2fa")
}
return nil, errors.New("2fa authentication failed")
}
return nil, fmt.Errorf("unknown state %q", config.State)
}
// find item by path. Will not return any children for the item
func (f *Fs) findItem(ctx context.Context, dir string) (item *api.DriveItem, found bool, err error) {
var resp *http.Response
@@ -797,51 +677,11 @@ func retryResultUnknown(ctx context.Context, resp *http.Response, err error) (bo
// NewFs constructs an Fs from the path, container:path
func NewFs(ctx context.Context, name, root string, m configmap.Mapper) (fs.Fs, error) {
// Parse config into Options struct
opt := new(Options)
err := configstruct.Set(m, opt)
icloud, opt, err := newICloudClient(ctx, name, m)
if err != nil {
return nil, err
}
if opt.Password != "" {
var err error
opt.Password, err = obscure.Reveal(opt.Password)
if err != nil {
return nil, fmt.Errorf("couldn't decrypt user password: %w", err)
}
}
if opt.TrustToken == "" {
return nil, fmt.Errorf("missing icloud trust token: try refreshing it with \"rclone config reconnect %s:\"", name)
}
cookies := ReadCookies(opt.Cookies)
callback := func(session *api.Session) {
m.Set(configCookies, session.GetCookieString())
}
icloud, err := api.New(
opt.AppleID,
opt.Password,
opt.TrustToken,
opt.ClientID,
cookies,
callback,
)
if err != nil {
return nil, err
}
if err := icloud.Authenticate(ctx); err != nil {
return nil, err
}
if icloud.Session.Requires2FA() {
return nil, errors.New("trust token expired, please reauth")
}
root = strings.Trim(root, "/")
f := &Fs{
@@ -850,6 +690,7 @@ func NewFs(ctx context.Context, name, root string, m configmap.Mapper) (fs.Fs, e
icloud: icloud,
rootID: "FOLDER::com.apple.CloudDocs::root",
opt: *opt,
m: m,
pacer: fs.NewPacer(ctx, pacer.NewDefault(pacer.MinSleep(minSleep), pacer.MaxSleep(maxSleep), pacer.DecayConstant(decayConstant))),
}
f.features = (&fs.Features{
@@ -1164,9 +1005,15 @@ func (o *Object) Update(ctx context.Context, in io.Reader, src fs.ObjectInfo, op
return nil
}
// Disconnect clears authentication state and removes disk caches
func (f *Fs) Disconnect(ctx context.Context) error {
return disconnectClient(f.m, f.icloud)
}
// Check interfaces are satisfied
var (
_ fs.Fs = &Fs{}
_ fs.Disconnecter = (*Fs)(nil)
_ fs.Mover = (*Fs)(nil)
_ fs.Purger = (*Fs)(nil)
_ fs.DirMover = (*Fs)(nil)
+892
View File
@@ -0,0 +1,892 @@
//go:build !plan9 && !solaris
package iclouddrive
import (
"context"
"fmt"
"io"
"net/http"
"path"
"strconv"
"strings"
"sync"
"time"
"github.com/rclone/rclone/backend/iclouddrive/api"
"github.com/rclone/rclone/fs"
"github.com/rclone/rclone/fs/config/configmap"
"github.com/rclone/rclone/fs/fshttp"
"github.com/rclone/rclone/fs/hash"
"github.com/rclone/rclone/fs/list"
"github.com/rclone/rclone/lib/dircache"
"github.com/rclone/rclone/lib/pacer"
)
const rootID = "photos-root"
// PhotosFs represents a remote iCloud Photos server
type PhotosFs struct {
name string
root string
opt Options
features *fs.Features
icloud *api.Client
m configmap.Mapper
pacer *fs.Pacer
dirCache *dircache.DirCache
httpClient *http.Client
startTime time.Time
mu sync.Mutex
photos *api.PhotosService
}
// PhotosObject describes an iCloud Photos object
type PhotosObject struct {
fs *PhotosFs
remote string
size int64
modTime time.Time
masterID string // CloudKit recordName for fresh URL lookup (master or asset depending on resource)
zone string // CloudKit zone (e.g. "PrimarySync")
resourceKey string // CloudKit resource field (resOriginalRes or resOriginalVidComplRes)
width int
height int
addedDate int64
isFavorite bool
isHidden bool
}
// NewFsPhotos constructs an Fs for Photos from the path, container:path
func NewFsPhotos(ctx context.Context, name, root string, m configmap.Mapper) (fs.Fs, error) {
icloud, opt, err := newICloudClient(ctx, name, m)
if err != nil {
return nil, err
}
root = strings.Trim(root, "/")
f := &PhotosFs{
name: name,
root: root,
opt: *opt,
icloud: icloud,
m: m,
pacer: fs.NewPacer(ctx, pacer.NewDefault(pacer.MinSleep(minSleep), pacer.MaxSleep(maxSleep), pacer.DecayConstant(decayConstant))),
httpClient: fshttp.NewClient(ctx),
startTime: time.Now(),
}
f.features = (&fs.Features{
CanHaveEmptyDirectories: true,
PartialUploads: false,
ReadMimeType: false,
ReadMetadata: true,
}).Fill(ctx, f)
f.dirCache = dircache.New(root, rootID, f)
err = f.dirCache.FindRoot(ctx, false)
if err != nil {
// Check if root points to a file (e.g. PrimarySync/Videos/file.mp4)
newRoot, remote := dircache.SplitPath(root)
tempF := &PhotosFs{
name: f.name,
root: newRoot,
opt: f.opt,
features: f.features,
icloud: f.icloud,
pacer: f.pacer,
httpClient: f.httpClient,
startTime: f.startTime,
photos: f.photos, // nil here; photosService() creates lazily on first use
}
tempF.dirCache = dircache.New(newRoot, rootID, tempF)
if err2 := tempF.dirCache.FindRoot(ctx, false); err2 != nil {
return f, nil
}
_, err2 := tempF.NewObject(ctx, remote)
if err2 != nil {
return f, nil
}
// Root is a file - adjust f to parent dir, signal ErrorIsFile
f.root = newRoot
f.dirCache = tempF.dirCache
return f, fs.ErrorIsFile
}
return f, nil
}
// photosService returns the PhotosService, creating it on first call
func (f *PhotosFs) photosService(ctx context.Context) (*api.PhotosService, error) {
f.mu.Lock()
defer f.mu.Unlock()
if f.photos == nil {
var err error
f.photos, err = api.NewPhotosService(ctx, f.icloud, f.pacer, shouldRetry)
if err != nil {
return nil, err
}
}
return f.photos, nil
}
// Name of the remote (as passed into NewFs)
func (f *PhotosFs) Name() string {
return f.name
}
// Root of the remote (as passed into NewFs)
func (f *PhotosFs) Root() string {
return f.opt.Enc.ToStandardPath(f.root)
}
// String converts this Fs to a string
func (f *PhotosFs) String() string {
return fmt.Sprintf("iCloud Photos root '%s'", f.root)
}
// Precision of the object storage system
func (f *PhotosFs) Precision() time.Duration {
return time.Second
}
// Hashes returns the supported hash sets
func (f *PhotosFs) Hashes() hash.Set {
return hash.Set(hash.None)
}
// Features returns the optional features of this Fs
func (f *PhotosFs) Features() *fs.Features {
return f.features
}
// List the objects and directories in dir into entries
func (f *PhotosFs) List(ctx context.Context, dir string) (entries fs.DirEntries, err error) {
photosService, err := f.photosService(ctx)
if err != nil {
return nil, fmt.Errorf("failed to get photos service: %w", err)
}
dirID, err := f.dirCache.FindDir(ctx, dir, false)
if err != nil {
return nil, err
}
switch {
case dirID == rootID:
// List libraries
libraries, err := photosService.GetLibraries(ctx)
if err != nil {
return nil, err
}
albumCounts, err := photosService.GetLibraryAlbumCounts(ctx)
if err != nil {
fs.Debugf(f, "Failed to get library album counts: %v", err)
}
for libraryName := range libraries {
count := int64(-1)
if albumCounts != nil {
if c, ok := albumCounts[libraryName]; ok {
count = c
}
}
d := fs.NewDir(path.Join(dir, f.opt.Enc.ToStandardName(libraryName)), f.startTime).SetItems(count)
entries = append(entries, d)
}
case strings.HasPrefix(dirID, "lib:"):
// List albums in a library
libraryName := strings.TrimPrefix(dirID, "lib:")
libraries, err := photosService.GetLibraries(ctx)
if err != nil {
return nil, err
}
library, exists := libraries[libraryName]
if !exists {
return nil, fs.ErrorDirNotFound
}
albums, err := library.GetAlbums(ctx)
if err != nil {
return nil, err
}
counts, err := library.GetAlbumCounts(ctx)
if err != nil {
fs.Debugf(f, "Failed to get album counts for library %q: %v", libraryName, err)
}
for albumName := range albums {
count := int64(-1)
if counts != nil {
if c, ok := counts[albumName]; ok {
count = c
}
}
d := fs.NewDir(path.Join(dir, f.opt.Enc.ToStandardName(albumName)), f.startTime).SetItems(count)
entries = append(entries, d)
}
case strings.HasPrefix(dirID, "album:"):
// List contents of an album or folder
libraryName, albumPath, ok := parseAlbumDirID(dirID)
if !ok {
return nil, fs.ErrorDirNotFound
}
album, err := f.resolveAlbum(ctx, photosService, libraryName, albumPath)
if err != nil {
return nil, err
}
// Folders list their child albums as subdirectories
if album.IsFolder {
for childName, child := range album.Children {
d := fs.NewDir(path.Join(dir, f.opt.Enc.ToStandardName(childName)), f.startTime)
if child.IsFolder {
d.SetItems(int64(len(child.Children)))
}
entries = append(entries, d)
}
return entries, nil
}
photos, err := album.GetPhotos(ctx)
if err != nil {
return nil, err
}
for _, photo := range photos {
if photo.Filename == "" {
continue
}
encodedName := f.opt.Enc.ToStandardName(photo.Filename)
remotePath := encodedName
if dir != "" {
remotePath = path.Join(dir, encodedName)
}
o := f.newPhotosObject(remotePath, photo, libraryName)
entries = append(entries, o)
}
default:
return nil, fs.ErrorDirNotFound
}
return entries, nil
}
// NewObject finds the Object at remote. If it can't be found
// it returns the error fs.ErrorObjectNotFound
func (f *PhotosFs) NewObject(ctx context.Context, remote string) (fs.Object, error) {
dir, leaf := dircache.SplitPath(remote)
filename := f.opt.Enc.FromStandardName(leaf)
dirID, err := f.dirCache.FindDir(ctx, dir, false)
if err != nil {
fs.Debugf(f, "NewObject(%s): FindDir: %v", remote, err)
return nil, fs.ErrorObjectNotFound
}
if !strings.HasPrefix(dirID, "album:") {
return nil, fs.ErrorObjectNotFound
}
libraryName, albumPath, ok := parseAlbumDirID(dirID)
if !ok {
return nil, fs.ErrorObjectNotFound
}
photosService, err := f.photosService(ctx)
if err != nil {
return nil, fmt.Errorf("failed to get photos service: %w", err)
}
album, err := f.resolveAlbum(ctx, photosService, libraryName, albumPath)
if err != nil || album.IsFolder {
fs.Debugf(f, "NewObject(%s): album resolve failed or is folder", remote)
return nil, fs.ErrorObjectNotFound
}
photo, err := album.GetPhotoByName(ctx, filename)
if err != nil {
fs.Debugf(f, "NewObject(%s): %v", remote, err)
return nil, fs.ErrorObjectNotFound
}
return f.newPhotosObject(remote, photo, libraryName), nil
}
// newPhotosObject creates a PhotosObject from a Photo and zone
func (f *PhotosFs) newPhotosObject(remote string, photo *api.Photo, zone string) *PhotosObject {
return &PhotosObject{
fs: f,
remote: remote,
size: photo.Size,
modTime: time.UnixMilli(photo.AssetDate),
masterID: photo.ID,
zone: zone,
resourceKey: photo.ResourceKey,
width: photo.Width,
height: photo.Height,
addedDate: photo.AddedDate,
isFavorite: photo.IsFavorite,
isHidden: photo.IsHidden,
}
}
// Put is not supported for this read-only backend
func (f *PhotosFs) Put(ctx context.Context, in io.Reader, src fs.ObjectInfo, options ...fs.OpenOption) (fs.Object, error) {
return nil, fs.ErrorNotImplemented
}
// Mkdir is not supported for this read-only backend
func (f *PhotosFs) Mkdir(ctx context.Context, dir string) error {
return fs.ErrorNotImplemented
}
// Rmdir is not supported for this read-only backend
func (f *PhotosFs) Rmdir(ctx context.Context, dir string) error {
return fs.ErrorNotImplemented
}
// FindLeaf finds a directory of name leaf in the folder with ID pathID
func (f *PhotosFs) FindLeaf(ctx context.Context, pathID, leaf string) (pathIDOut string, found bool, err error) {
decodedLeaf := f.opt.Enc.FromStandardName(leaf)
photosService, err := f.photosService(ctx)
if err != nil {
return "", false, fmt.Errorf("failed to get photos service: %w", err)
}
if pathID == rootID {
libraries, err := photosService.GetLibraries(ctx)
if err != nil {
return "", false, err
}
if _, exists := libraries[decodedLeaf]; exists {
return "lib:" + decodedLeaf, true, nil
}
return "", false, nil
}
if strings.HasPrefix(pathID, "lib:") {
libraryName := strings.TrimPrefix(pathID, "lib:")
libraries, err := photosService.GetLibraries(ctx)
if err != nil {
return "", false, err
}
library, exists := libraries[libraryName]
if !exists {
return "", false, fs.ErrorDirNotFound
}
albums, err := library.GetAlbums(ctx)
if err != nil {
return "", false, err
}
if _, exists := albums[decodedLeaf]; exists {
albumID := "album:" + libraryName + ":" + decodedLeaf
return albumID, true, nil
}
return "", false, nil
}
if strings.HasPrefix(pathID, "album:") {
// Check if this is a folder containing child albums
libraryName, albumPath, ok := parseAlbumDirID(pathID)
if !ok {
return "", false, nil
}
album, err := f.resolveAlbum(ctx, photosService, libraryName, albumPath)
if err != nil {
return "", false, nil
}
if album.IsFolder {
if _, exists := album.Children[decodedLeaf]; exists {
childID := "album:" + libraryName + ":" + albumPath + "/" + decodedLeaf
return childID, true, nil
}
}
return "", false, nil
}
return "", false, nil
}
// CreateDir makes a directory with pathID as parent and name leaf
func (f *PhotosFs) CreateDir(ctx context.Context, pathID, leaf string) (newID string, err error) {
return "", fs.ErrorNotImplemented
}
// Fs returns the parent Fs
func (o *PhotosObject) Fs() fs.Info {
return o.fs
}
// Return a string version
func (o *PhotosObject) String() string {
if o == nil {
return "<nil>"
}
return o.remote
}
// Remote returns the remote path
func (o *PhotosObject) Remote() string {
return o.remote
}
// ModTime returns the modification time of the object
func (o *PhotosObject) ModTime(ctx context.Context) time.Time {
return o.modTime
}
// Size returns the size of an object in bytes
func (o *PhotosObject) Size() int64 {
return o.size
}
// Storable returns a boolean as to whether this object is storable
func (o *PhotosObject) Storable() bool {
return true
}
// Hash returns the hash of an object returning a lowercase hex string
func (o *PhotosObject) Hash(ctx context.Context, ty hash.Type) (string, error) {
return "", hash.ErrUnsupported
}
// SetModTime sets the modification time of the object
func (o *PhotosObject) SetModTime(ctx context.Context, modTime time.Time) error {
return fs.ErrorCantSetModTime
}
// Metadata returns metadata for the photo object
func (o *PhotosObject) Metadata(ctx context.Context) (fs.Metadata, error) {
metadata := make(fs.Metadata, 5)
if o.width > 0 {
metadata["width"] = strconv.Itoa(o.width)
}
if o.height > 0 {
metadata["height"] = strconv.Itoa(o.height)
}
if o.addedDate > 0 {
metadata["added-time"] = time.UnixMilli(o.addedDate).UTC().Format(time.RFC3339)
}
metadata["favorite"] = strconv.FormatBool(o.isFavorite)
metadata["hidden"] = strconv.FormatBool(o.isHidden)
return metadata, nil
}
// Open an object for read, fetching a fresh download URL via records/lookup
func (o *PhotosObject) Open(ctx context.Context, options ...fs.OpenOption) (io.ReadCloser, error) {
photosService, err := o.fs.photosService(ctx)
if err != nil {
return nil, fmt.Errorf("failed to get photos service: %w", err)
}
downloadURL, err := photosService.LookupDownloadURL(ctx, o.masterID, o.zone, o.resourceKey)
if err != nil {
return nil, fmt.Errorf("failed to get download URL for %q: %w", o.remote, err)
}
fs.FixRangeOption(options, o.size)
var resp *http.Response
err = o.fs.pacer.Call(func() (bool, error) {
if resp != nil {
_ = resp.Body.Close()
resp = nil
}
req, err := http.NewRequestWithContext(ctx, "GET", downloadURL, nil)
if err != nil {
return false, fmt.Errorf("failed to create request: %w", err)
}
fs.OpenOptionAddHTTPHeaders(req.Header, options)
resp, err = o.fs.httpClient.Do(req)
return shouldRetry(ctx, resp, err)
})
if err != nil {
return nil, err
}
if resp.StatusCode != http.StatusOK && resp.StatusCode != http.StatusPartialContent {
_ = resp.Body.Close()
return nil, fmt.Errorf("download %q failed: %s", o.remote, resp.Status)
}
return resp.Body, nil
}
// Update is not supported for this read-only backend
func (o *PhotosObject) Update(ctx context.Context, in io.Reader, src fs.ObjectInfo, options ...fs.OpenOption) error {
return fs.ErrorNotImplemented
}
// Remove is not supported for this read-only backend
func (o *PhotosObject) Remove(ctx context.Context) error {
return fs.ErrorNotImplemented
}
// resolveAlbum finds an album by path, traversing folder hierarchy
// albumPath can be "AlbumName" or "FolderName/AlbumName" for nested albums
func (f *PhotosFs) resolveAlbum(ctx context.Context, ps *api.PhotosService, libraryName, albumPath string) (*api.Album, error) {
libraries, err := ps.GetLibraries(ctx)
if err != nil {
return nil, err
}
library, exists := libraries[libraryName]
if !exists {
return nil, fs.ErrorDirNotFound
}
albums, err := library.GetAlbums(ctx)
if err != nil {
return nil, err
}
return resolveAlbumPath(albums, albumPath)
}
// resolveAlbumPath finds an album inside an album tree by slash-separated path
func resolveAlbumPath(albums map[string]*api.Album, albumPath string) (*api.Album, error) {
parts := strings.Split(albumPath, "/")
current := albums
for i, part := range parts {
album, exists := current[part]
if !exists {
return nil, fs.ErrorDirNotFound
}
if i == len(parts)-1 {
return album, nil
}
if !album.IsFolder || album.Children == nil {
return nil, fs.ErrorDirNotFound
}
current = album.Children
}
return nil, fs.ErrorDirNotFound
}
// parseAlbumDirID extracts library and album names from "album:lib:album" dirID
func parseAlbumDirID(dirID string) (libraryName, albumName string, ok bool) {
parts := strings.SplitN(strings.TrimPrefix(dirID, "album:"), ":", 2)
if len(parts) != 2 {
return "", "", false
}
return parts[0], parts[1], true
}
// DirCacheFlush resets the directory cache - used in testing as an
// optional interface
func (f *PhotosFs) DirCacheFlush() {
f.dirCache.ResetRoot()
// Also flush API-layer caches so albums/photos are re-fetched
// DirCacheFlusher interface has no ctx param; use a bounded timeout
// to avoid hanging indefinitely if photosService triggers first init
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
if ps, err := f.photosService(ctx); err == nil {
ps.FlushCaches()
}
}
// listRAlbumJob represents an album to be listed by a ListR worker
type listRAlbumJob struct {
album *api.Album
zone string
dirPath string // path relative to Fs root (e.g. "Videos" when f.root="PrimarySync")
}
// ListR lists the objects and directories of the Fs starting from dir
// recursively into out, calling callback for each batch of entries
// Albums are listed in parallel using a goroutine pool
func (f *PhotosFs) ListR(ctx context.Context, dir string, callback fs.ListRCallback) error {
ps, err := f.photosService(ctx)
if err != nil {
return err
}
helper := list.NewHelper(callback)
var mu sync.Mutex // protects helper (not thread-safe)
addEntry := func(entry fs.DirEntry) error {
mu.Lock()
defer mu.Unlock()
return helper.Add(entry)
}
// Resolve dir through dirCache - this respects f.root
dirID, err := f.dirCache.FindDir(ctx, dir, false)
if err != nil {
return err
}
// Collect album jobs, scoped to the resolved directory
var jobs []listRAlbumJob
// collectAlbumJobs emits directory entries for albums/folders under basePath
// and collects leaf albums as jobs for parallel photo listing
// Recurses into folders to handle arbitrary nesting depth
var collectAlbumJobs func(albums map[string]*api.Album, zone, basePath string) error
collectAlbumJobs = func(albums map[string]*api.Album, zone, basePath string) error {
for albumName, album := range albums {
albumPath := path.Join(basePath, f.opt.Enc.ToStandardName(albumName))
if err := addEntry(fs.NewDir(albumPath, f.startTime)); err != nil {
return err
}
if album.IsFolder {
if err := collectAlbumJobs(album.Children, zone, albumPath); err != nil {
return err
}
} else {
jobs = append(jobs, listRAlbumJob{album: album, zone: zone, dirPath: albumPath})
}
}
return nil
}
switch {
case dirID == rootID:
// Root level - emit libraries, then all albums
libraries, err := ps.GetLibraries(ctx)
if err != nil {
return err
}
for libName, lib := range libraries {
libPath := path.Join(dir, f.opt.Enc.ToStandardName(libName))
if err := addEntry(fs.NewDir(libPath, f.startTime)); err != nil {
return err
}
albums, err := lib.GetAlbums(ctx)
if err != nil {
return err
}
if err := collectAlbumJobs(albums, libName, libPath); err != nil {
return err
}
}
case strings.HasPrefix(dirID, "lib:"):
// Library level - emit and collect all albums
libraryName := strings.TrimPrefix(dirID, "lib:")
libraries, err := ps.GetLibraries(ctx)
if err != nil {
return err
}
library, exists := libraries[libraryName]
if !exists {
return fs.ErrorDirNotFound
}
albums, err := library.GetAlbums(ctx)
if err != nil {
return err
}
if err := collectAlbumJobs(albums, libraryName, dir); err != nil {
return err
}
case strings.HasPrefix(dirID, "album:"):
// Album or folder level
libraryName, albumPath, ok := parseAlbumDirID(dirID)
if !ok {
return fs.ErrorDirNotFound
}
album, err := f.resolveAlbum(ctx, ps, libraryName, albumPath)
if err != nil {
return err
}
if album.IsFolder {
if err := collectAlbumJobs(album.Children, libraryName, dir); err != nil {
return err
}
} else {
// Single album - just list its photos
jobs = append(jobs, listRAlbumJob{album: album, zone: libraryName, dirPath: dir})
}
default:
return fs.ErrorDirNotFound
}
if len(jobs) == 0 {
return helper.Flush()
}
// Fan out album photo listing across goroutines
ctx, cancel := context.WithCancel(ctx)
defer cancel()
jobCh := make(chan listRAlbumJob, len(jobs))
for _, job := range jobs {
jobCh <- job
}
close(jobCh)
workers := fs.GetConfig(ctx).Checkers
if len(jobs) < workers {
workers = len(jobs)
}
errs := make(chan error, workers)
for range workers {
go func() {
for job := range jobCh {
photos, err := job.album.GetPhotos(ctx)
if err != nil {
errs <- err
cancel()
return
}
for _, photo := range photos {
if photo.Filename == "" {
continue
}
remotePath := path.Join(job.dirPath, f.opt.Enc.ToStandardName(photo.Filename))
o := f.newPhotosObject(remotePath, photo, job.zone)
if err := addEntry(o); err != nil {
errs <- err
cancel()
return
}
}
}
errs <- nil
}()
}
for range workers {
if e := <-errs; e != nil && err == nil {
err = e
}
}
if err != nil {
return err
}
return helper.Flush()
}
func notifyAlbumTree(albums map[string]*api.Album, base string, notifyFunc func(string, fs.EntryType)) {
for name, album := range albums {
albumPath := name
if base != "" {
albumPath = path.Join(base, name)
}
notifyFunc(albumPath, fs.EntryDirectory)
if album.IsFolder {
notifyAlbumTree(album.Children, albumPath, notifyFunc)
}
}
}
func (f *PhotosFs) notifyZoneChange(ctx context.Context, ps *api.PhotosService, zone string, notifyFunc func(string, fs.EntryType)) {
if f.root == "" {
notifyFunc(zone, fs.EntryDirectory)
return
}
libraryName, albumPath, _ := strings.Cut(f.root, "/")
if libraryName != zone {
return
}
// Always invalidate the mounted root when its backing zone changes
notifyFunc("", fs.EntryDirectory)
libraries, err := ps.GetLibraries(ctx)
if err != nil {
return
}
library, ok := libraries[zone]
if !ok {
return
}
albums, err := library.GetAlbums(ctx)
if err != nil {
return
}
if albumPath == "" {
notifyAlbumTree(albums, "", notifyFunc)
return
}
album, err := resolveAlbumPath(albums, albumPath)
if err != nil {
return
}
if album.IsFolder {
notifyAlbumTree(album.Children, "", notifyFunc)
}
}
// ChangeNotify polls for changes and notifies the VFS when directories are modified
func (f *PhotosFs) ChangeNotify(ctx context.Context, notifyFunc func(string, fs.EntryType), pollIntervalChan <-chan time.Duration) {
go func() {
var ticker *time.Ticker
var tickerC <-chan time.Time
for {
select {
case pollInterval, ok := <-pollIntervalChan:
if !ok {
if ticker != nil {
ticker.Stop()
}
return
}
if ticker != nil {
ticker.Stop()
tickerC = nil
}
if pollInterval > 0 {
ticker = time.NewTicker(pollInterval)
tickerC = ticker.C
}
case <-tickerC:
ps, err := f.photosService(ctx)
if err != nil {
fs.Debugf(f, "ChangeNotify: failed to get photos service: %v", err)
continue
}
changedZones := ps.PollForChanges(ctx)
for _, zone := range changedZones {
f.notifyZoneChange(ctx, ps, zone, notifyFunc)
}
case <-ctx.Done():
if ticker != nil {
ticker.Stop()
}
return
}
}
}()
}
// Disconnect clears authentication state and removes disk caches
func (f *PhotosFs) Disconnect(ctx context.Context) error {
return disconnectClient(f.m, f.icloud)
}
// Check interfaces are satisfied
var (
_ fs.Fs = (*PhotosFs)(nil)
_ fs.Disconnecter = (*PhotosFs)(nil)
_ fs.ListRer = (*PhotosFs)(nil)
_ fs.ChangeNotifier = (*PhotosFs)(nil)
_ fs.DirCacheFlusher = (*PhotosFs)(nil)
_ fs.Object = (*PhotosObject)(nil)
_ fs.Metadataer = (*PhotosObject)(nil)
)
+516
View File
@@ -0,0 +1,516 @@
//go:build !plan9 && !solaris
package iclouddrive
import (
"context"
"sort"
"testing"
"time"
"github.com/rclone/rclone/backend/iclouddrive/api"
"github.com/rclone/rclone/fs"
"github.com/rclone/rclone/lib/dircache"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func newTestPhotosFs() *PhotosFs {
return &PhotosFs{
name: "test-icp",
root: "",
opt: Options{},
}
}
func TestSmartAlbumDefinitions(t *testing.T) {
expected := map[string]struct {
ObjectType string
ListType string
}{
"All Photos": {"CPLAssetByAssetDateWithoutHiddenOrDeleted", "CPLAssetAndMasterByAssetDateWithoutHiddenOrDeleted"},
"Time-lapse": {"CPLAssetInSmartAlbumByAssetDate:Timelapse", "CPLAssetAndMasterInSmartAlbumByAssetDate"},
"Videos": {"CPLAssetInSmartAlbumByAssetDate:Video", "CPLAssetAndMasterInSmartAlbumByAssetDate"},
"Slo-mo": {"CPLAssetInSmartAlbumByAssetDate:Slomo", "CPLAssetAndMasterInSmartAlbumByAssetDate"},
"Bursts": {"CPLAssetBurstStackAssetByAssetDate", "CPLBurstStackAssetAndMasterByAssetDate"},
"Favorites": {"CPLAssetInSmartAlbumByAssetDate:Favorite", "CPLAssetAndMasterInSmartAlbumByAssetDate"},
"Panoramas": {"CPLAssetInSmartAlbumByAssetDate:Panorama", "CPLAssetAndMasterInSmartAlbumByAssetDate"},
"Screenshots": {"CPLAssetInSmartAlbumByAssetDate:Screenshot", "CPLAssetAndMasterInSmartAlbumByAssetDate"},
"Live": {"CPLAssetInSmartAlbumByAssetDate:Live", "CPLAssetAndMasterInSmartAlbumByAssetDate"},
"Recently Deleted": {"CPLAssetDeletedByExpungedDate", "CPLAssetAndMasterDeletedByExpungedDate"},
"Hidden": {"CPLAssetHiddenByAssetDate", "CPLAssetAndMasterHiddenByAssetDate"},
"Portrait": {"CPLAssetInSmartAlbumByAssetDate:Depth", "CPLAssetAndMasterInSmartAlbumByAssetDate"},
"Long Exposure": {"CPLAssetInSmartAlbumByAssetDate:Exposure", "CPLAssetAndMasterInSmartAlbumByAssetDate"},
"Animated": {"CPLAssetInSmartAlbumByAssetDate:Animated", "CPLAssetAndMasterInSmartAlbumByAssetDate"},
}
for name, exp := range expected {
t.Run(name, func(t *testing.T) {
album, exists := api.SmartAlbums[name]
require.True(t, exists, "smart album %q must be defined", name)
assert.Equal(t, exp.ObjectType, album.ObjectType)
assert.Equal(t, exp.ListType, album.ListType)
})
}
assert.Equal(t, len(expected), len(api.SmartAlbums))
}
func TestSmartAlbumFilters(t *testing.T) {
filtered := map[string]string{
"Time-lapse": "TIMELAPSE",
"Videos": "VIDEO",
"Slo-mo": "SLOMO",
"Favorites": "FAVORITE",
"Panoramas": "PANORAMA",
"Screenshots": "SCREENSHOT",
"Live": "LIVE",
"Portrait": "DEPTH",
"Long Exposure": "EXPOSURE",
"Animated": "ANIMATED",
}
for name, filterVal := range filtered {
t.Run(name, func(t *testing.T) {
album := api.SmartAlbums[name]
require.NotEmpty(t, album.Filters, "album %q must have filters", name)
assert.Equal(t, "smartAlbum", album.Filters[0].FieldName)
assert.Equal(t, "EQUALS", album.Filters[0].Comparator)
fv, ok := album.Filters[0].FieldValue.(map[string]string)
require.True(t, ok, "filter value must be map[string]string")
assert.Equal(t, filterVal, fv["value"])
})
}
unfiltered := []string{"All Photos", "Bursts", "Recently Deleted", "Hidden"}
for _, name := range unfiltered {
t.Run(name+"_no_filter", func(t *testing.T) {
album := api.SmartAlbums[name]
assert.Empty(t, album.Filters, "album %q must not have filters", name)
})
}
}
func TestParseAlbumDirID(t *testing.T) {
lib, album, ok := parseAlbumDirID("album:PrimarySync:Favorites")
assert.True(t, ok)
assert.Equal(t, "PrimarySync", lib)
assert.Equal(t, "Favorites", album)
// Colon in album name
lib, album, ok = parseAlbumDirID("album:PrimarySync:My:Album")
assert.True(t, ok)
assert.Equal(t, "PrimarySync", lib)
assert.Equal(t, "My:Album", album)
// Missing album
_, _, ok = parseAlbumDirID("album:PrimarySync")
assert.False(t, ok)
// Empty
_, _, ok = parseAlbumDirID("album:")
assert.False(t, ok)
}
func TestPhotosObject_Metadata(t *testing.T) {
o := &PhotosObject{
fs: &PhotosFs{name: "test"},
remote: "PrimarySync/All Photos/IMG_0001.HEIC",
size: 4200000,
modTime: time.Date(2025, 6, 15, 14, 30, 0, 0, time.UTC),
masterID: "master-001",
zone: "PrimarySync",
width: 4032,
height: 3024,
addedDate: 1718459400000, // 2024-06-15T13:50:00Z in millis
isFavorite: true,
isHidden: false,
}
metadata, err := o.Metadata(context.Background())
require.NoError(t, err)
assert.Equal(t, "4032", metadata["width"])
assert.Equal(t, "3024", metadata["height"])
assert.Equal(t, "2024-06-15T13:50:00Z", metadata["added-time"])
assert.Equal(t, "true", metadata["favorite"])
assert.Equal(t, "false", metadata["hidden"])
}
func TestPhotosObject_MetadataZeroDimensions(t *testing.T) {
// Live Photo .MOV companion - no width/height, not favorite
o := &PhotosObject{
fs: &PhotosFs{name: "test"},
remote: "PrimarySync/Live/IMG_3031.MOV",
}
metadata, err := o.Metadata(context.Background())
require.NoError(t, err)
_, hasWidth := metadata["width"]
_, hasHeight := metadata["height"]
assert.False(t, hasWidth, "zero width should be omitted")
assert.False(t, hasHeight, "zero height should be omitted")
assert.Equal(t, "false", metadata["favorite"])
assert.Equal(t, "false", metadata["hidden"])
}
// newTestPhotosService builds a PhotosService with pre-populated libraries and albums
// for testing resolveAlbum and FindLeaf without HTTP calls
func newTestPhotosService() *api.PhotosService {
return api.NewTestPhotosService(map[string]map[string]*api.Album{
"PrimarySync": {
"All Photos": {Name: "All Photos", ObjectType: "CPLAssetByAssetDateWithoutHiddenOrDeleted"},
"Videos": {Name: "Videos", ObjectType: "CPLAssetInSmartAlbumByAssetDate:Video"},
"UserAlbum": {Name: "UserAlbum", ObjectType: "CPLContainerRelationNotDeletedByAssetDate:rec1", RecordName: "rec1"},
"Folder": {
Name: "Folder", RecordName: "folder1", IsFolder: true,
Children: map[string]*api.Album{
"ChildAlbum": {Name: "ChildAlbum", ObjectType: "CPLContainerRelationNotDeletedByAssetDate:rec2", RecordName: "rec2"},
"NestedFolder": {
Name: "NestedFolder", RecordName: "folder2", IsFolder: true,
Children: map[string]*api.Album{
"LeafAlbum": {Name: "LeafAlbum", ObjectType: "CPLContainerRelationNotDeletedByAssetDate:rec3", RecordName: "rec3"},
},
},
},
},
},
})
}
func TestResolveAlbum(t *testing.T) {
f := newTestPhotosFs()
f.photos = newTestPhotosService()
ctx := context.Background()
t.Run("simple album", func(t *testing.T) {
album, err := f.resolveAlbum(ctx, f.photos, "PrimarySync", "Videos")
require.NoError(t, err)
assert.Equal(t, "Videos", album.Name)
})
t.Run("nested folder child", func(t *testing.T) {
album, err := f.resolveAlbum(ctx, f.photos, "PrimarySync", "Folder/ChildAlbum")
require.NoError(t, err)
assert.Equal(t, "ChildAlbum", album.Name)
})
t.Run("two-level nesting", func(t *testing.T) {
album, err := f.resolveAlbum(ctx, f.photos, "PrimarySync", "Folder/NestedFolder/LeafAlbum")
require.NoError(t, err)
assert.Equal(t, "LeafAlbum", album.Name)
})
t.Run("folder itself", func(t *testing.T) {
album, err := f.resolveAlbum(ctx, f.photos, "PrimarySync", "Folder")
require.NoError(t, err)
assert.True(t, album.IsFolder)
})
t.Run("missing album", func(t *testing.T) {
_, err := f.resolveAlbum(ctx, f.photos, "PrimarySync", "NoSuchAlbum")
assert.ErrorIs(t, err, fs.ErrorDirNotFound)
})
t.Run("missing library", func(t *testing.T) {
_, err := f.resolveAlbum(ctx, f.photos, "NoSuchLib", "Videos")
assert.ErrorIs(t, err, fs.ErrorDirNotFound)
})
t.Run("traverse into non-folder", func(t *testing.T) {
_, err := f.resolveAlbum(ctx, f.photos, "PrimarySync", "Videos/SubAlbum")
assert.ErrorIs(t, err, fs.ErrorDirNotFound)
})
}
func TestFindLeaf(t *testing.T) {
f := newTestPhotosFs()
f.photos = newTestPhotosService()
ctx := context.Background()
t.Run("root to library", func(t *testing.T) {
id, found, err := f.FindLeaf(ctx, rootID, "PrimarySync")
require.NoError(t, err)
assert.True(t, found)
assert.Equal(t, "lib:PrimarySync", id)
})
t.Run("root to missing library", func(t *testing.T) {
_, found, err := f.FindLeaf(ctx, rootID, "NoSuchLib")
require.NoError(t, err)
assert.False(t, found)
})
t.Run("library to album", func(t *testing.T) {
id, found, err := f.FindLeaf(ctx, "lib:PrimarySync", "Videos")
require.NoError(t, err)
assert.True(t, found)
assert.Equal(t, "album:PrimarySync:Videos", id)
})
t.Run("library to missing album", func(t *testing.T) {
_, found, err := f.FindLeaf(ctx, "lib:PrimarySync", "NoSuchAlbum")
require.NoError(t, err)
assert.False(t, found)
})
t.Run("missing library returns ErrorDirNotFound", func(t *testing.T) {
_, _, err := f.FindLeaf(ctx, "lib:NoSuchLib", "Videos")
assert.ErrorIs(t, err, fs.ErrorDirNotFound)
})
t.Run("folder to child album", func(t *testing.T) {
id, found, err := f.FindLeaf(ctx, "album:PrimarySync:Folder", "ChildAlbum")
require.NoError(t, err)
assert.True(t, found)
assert.Equal(t, "album:PrimarySync:Folder/ChildAlbum", id)
})
t.Run("non-folder album returns not found", func(t *testing.T) {
_, found, err := f.FindLeaf(ctx, "album:PrimarySync:Videos", "SubAlbum")
require.NoError(t, err)
assert.False(t, found)
})
t.Run("unknown prefix returns not found", func(t *testing.T) {
_, found, err := f.FindLeaf(ctx, "unknown:prefix", "leaf")
require.NoError(t, err)
assert.False(t, found)
})
}
func TestResolveAlbum_EmptyPath(t *testing.T) {
f := newTestPhotosFs()
f.photos = newTestPhotosService()
ctx := context.Background()
// parseAlbumDirID("album:PrimarySync:") returns ("PrimarySync", "", true)
// This empty album path feeds into resolveAlbum
_, err := f.resolveAlbum(ctx, f.photos, "PrimarySync", "")
assert.ErrorIs(t, err, fs.ErrorDirNotFound, "empty album path should return ErrorDirNotFound")
}
func TestNewObject_ErrorPaths(t *testing.T) {
f := newTestPhotosFs()
f.photos = newTestPhotosService()
ctx := context.Background()
// Pre-populate photoCache on the "Videos" album so GetPhotoByName works
// without HTTP. Access via the test PhotosService's internal structure
libs, _ := f.photos.GetLibraries(ctx)
albums, _ := libs["PrimarySync"].GetAlbums(ctx)
videos := albums["Videos"]
videos.SetTestPhotoCache(map[string]*api.Photo{
"existing.mp4": {ID: "m1", Filename: "existing.mp4", Size: 1024, ResourceKey: "resOriginalRes"},
})
// NewObject depends on dircache.FindDir which requires FindRoot + HTTP
// Test the reachable paths through resolveAlbum and GetPhotoByName
// Test resolveAlbum → folder → GetPhotoByName on folder (should fail)
album, err := f.resolveAlbum(ctx, f.photos, "PrimarySync", "Folder")
require.NoError(t, err)
assert.True(t, album.IsFolder, "Folder should be a folder")
// NewObject would return ErrorObjectNotFound for a folder path
// Test GetPhotoByName on album with populated cache
photo, err := videos.GetPhotoByName(ctx, "existing.mp4")
require.NoError(t, err)
assert.Equal(t, "m1", photo.ID)
// Test GetPhotoByName cache miss
_, err = videos.GetPhotoByName(ctx, "nonexistent.mp4")
assert.Error(t, err)
assert.Contains(t, err.Error(), "nonexistent.mp4")
}
func TestParseAlbumDirID_Exhaustive(t *testing.T) {
tests := []struct {
input string
lib, album string
ok bool
}{
{"album:PrimarySync:Videos", "PrimarySync", "Videos", true},
{"album:PrimarySync:Folder/Child", "PrimarySync", "Folder/Child", true},
{"album:PrimarySync:Name:With:Colons", "PrimarySync", "Name:With:Colons", true},
{"album:PrimarySync:", "PrimarySync", "", true},
{"album:", "", "", false},
{"notalbum:foo:bar", "notalbum", "foo:bar", true}, // strips "album:" prefix literally, "notalbum:" stays
{"", "", "", false},
}
for _, tt := range tests {
lib, album, ok := parseAlbumDirID(tt.input)
if tt.ok {
assert.True(t, ok, "input=%q", tt.input)
assert.Equal(t, tt.lib, lib, "input=%q", tt.input)
assert.Equal(t, tt.album, album, "input=%q", tt.input)
} else {
assert.False(t, ok, "input=%q", tt.input)
}
}
}
// setEmptyPhotoCaches recursively sets empty photo caches on all leaf albums
// so GetPhotos returns without HTTP (service=nil test fast path)
func setEmptyPhotoCaches(albums map[string]*api.Album) {
for _, album := range albums {
if album.IsFolder {
setEmptyPhotoCaches(album.Children)
} else {
album.SetTestPhotoCache(map[string]*api.Photo{})
}
}
}
func TestListR_NestedFolderRecursion(t *testing.T) {
f := newTestPhotosFs()
f.photos = newTestPhotosService()
f.dirCache = dircache.New("", rootID, f)
f.features = (&fs.Features{}).Fill(context.Background(), f)
f.startTime = time.Now()
ctx := context.Background()
err := f.dirCache.FindRoot(ctx, false)
require.NoError(t, err)
// Pre-populate empty photo caches on all leaf albums
libs, err := f.photos.GetLibraries(ctx)
require.NoError(t, err)
for _, lib := range libs {
albums, err := lib.GetAlbums(ctx)
require.NoError(t, err)
setEmptyPhotoCaches(albums)
}
var dirs []string
err = f.ListR(ctx, "", func(entries fs.DirEntries) error {
for _, entry := range entries {
if _, ok := entry.(fs.Directory); ok {
dirs = append(dirs, entry.Remote())
}
}
return nil
})
require.NoError(t, err)
sort.Strings(dirs)
// All directories must be present, including deeply nested ones
assert.Contains(t, dirs, "PrimarySync")
assert.Contains(t, dirs, "PrimarySync/All Photos")
assert.Contains(t, dirs, "PrimarySync/Videos")
assert.Contains(t, dirs, "PrimarySync/Folder")
assert.Contains(t, dirs, "PrimarySync/Folder/ChildAlbum")
assert.Contains(t, dirs, "PrimarySync/Folder/NestedFolder")
assert.Contains(t, dirs, "PrimarySync/Folder/NestedFolder/LeafAlbum")
}
func TestListR_FolderLevel(t *testing.T) {
f := &PhotosFs{
name: "test",
root: "PrimarySync/Folder",
opt: Options{},
startTime: time.Now(),
}
f.features = (&fs.Features{}).Fill(context.Background(), f)
f.photos = newTestPhotosService()
f.dirCache = dircache.New("PrimarySync/Folder", rootID, f)
ctx := context.Background()
err := f.dirCache.FindRoot(ctx, false)
require.NoError(t, err)
libs, _ := f.photos.GetLibraries(ctx)
for _, lib := range libs {
albums, _ := lib.GetAlbums(ctx)
setEmptyPhotoCaches(albums)
}
var dirs []string
err = f.ListR(ctx, "", func(entries fs.DirEntries) error {
for _, entry := range entries {
if _, ok := entry.(fs.Directory); ok {
dirs = append(dirs, entry.Remote())
}
}
return nil
})
require.NoError(t, err)
// When rooted at a folder, should see children recursively
assert.Contains(t, dirs, "ChildAlbum")
assert.Contains(t, dirs, "NestedFolder")
assert.Contains(t, dirs, "NestedFolder/LeafAlbum")
}
func TestNotifyZoneChange(t *testing.T) {
ctx := context.Background()
ps := newTestPhotosService()
tests := []struct {
name string
root string
zone string
want []string
}{
{
name: "top level root",
root: "",
zone: "PrimarySync",
want: []string{"PrimarySync"},
},
{
name: "library root",
root: "PrimarySync",
zone: "PrimarySync",
want: []string{"", "All Photos", "Folder", "Folder/ChildAlbum", "Folder/NestedFolder", "Folder/NestedFolder/LeafAlbum", "UserAlbum", "Videos"},
},
{
name: "folder root",
root: "PrimarySync/Folder",
zone: "PrimarySync",
want: []string{"", "ChildAlbum", "NestedFolder", "NestedFolder/LeafAlbum"},
},
{
name: "nested folder root",
root: "PrimarySync/Folder/NestedFolder",
zone: "PrimarySync",
want: []string{"", "LeafAlbum"},
},
{
name: "leaf album root",
root: "PrimarySync/Folder/NestedFolder/LeafAlbum",
zone: "PrimarySync",
want: []string{""},
},
{
name: "missing nested album still invalidates root",
root: "PrimarySync/Folder/NoSuchAlbum",
zone: "PrimarySync",
want: []string{""},
},
{
name: "different zone ignored",
root: "PrimarySync/Folder/NestedFolder",
zone: "SharedSync",
want: nil,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
f := newTestPhotosFs()
f.root = tt.root
var got []string
f.notifyZoneChange(ctx, ps, tt.zone, func(remote string, entryType fs.EntryType) {
assert.Equal(t, fs.EntryDirectory, entryType)
got = append(got, remote)
})
sort.Strings(got)
want := append([]string(nil), tt.want...)
sort.Strings(want)
assert.Equal(t, want, got)
})
}
}
+135 -24
View File
@@ -1,20 +1,25 @@
---
title: "iCloud Drive"
description: "Rclone docs for iCloud Drive"
title: "iCloud Drive and Photos"
description: "Rclone docs for iCloud Drive and Photos"
versionIntroduced: "v1.69"
---
# iCloud Drive
# {{< icon "fa fa-cloud" >}} iCloud Drive and Photos
## Configuration
The initial setup for an iCloud Drive backend involves getting a trust token/session.
This can be done by simply using the regular iCloud password, and accepting the code
prompt on another iCloud connected device.
The initial setup for an iCloud backend involves getting a trust token/session.
This uses your regular Apple ID password plus 2FA, either from a trusted
device prompt or an SMS code sent to a trusted phone number.
**IMPORTANT**: App-specific passwords are not accepted. Only use your
regular Apple ID password and 2FA.
This backend serves two Apple services:
- `drive` - iCloud Drive (default)
- `photos` - iCloud Photos
`rclone config` walks you through the token creation. The trust token is valid
for 30 days. After which you will have to reauthenticate with `rclone reconnect`
or `rclone config`.
@@ -30,10 +35,12 @@ The authentication flow is:
1. rclone initiates a session with Apple's identity service
2. An SRP key exchange takes place (your password is used locally to derive a key)
3. Apple sends a 2FA prompt to your trusted devices
3. Apple sends a 2FA prompt to your trusted devices, or lets you request an
SMS code
4. After you enter the 2FA code, rclone receives a trust token for future sessions
Here is an example of how to make a remote called `iclouddrive`. First run:
Here is an example of how to make a Photos remote called `icloudphotos`.
For iCloud Drive, leave `service` at its default `drive` value. First run:
```console
rclone config
@@ -47,7 +54,7 @@ n) New remote
s) Set configuration password
q) Quit config
n/s/q> n
name> iclouddrive
name> icloudphotos
Option Storage.
Type of storage to configure.
Choose a number from below, or type in your own value.
@@ -56,6 +63,15 @@ XX / iCloud Drive
\ (iclouddrive)
[snip]
Storage> iclouddrive
Option service.
iCloud service to use.
Choose a number from below, or type in your own value of type string.
Press Enter for the default (drive).
1 / iCloud Drive
\ (drive)
2 / iCloud Photos
\ (photos)
service> 2
Option apple_id.
Apple ID.
Enter a value.
@@ -75,13 +91,14 @@ y) Yes
n) No (default)
y/n> n
Option config_2fa.
Two-factor authentication: please enter your 2FA code
Two-factor authentication: enter your 2FA code or type 'sms' for a text message
Enter a value.
config_2fa> 2FACODE
Remote config
--------------------
[iclouddrive]
[icloudphotos]
- type: iclouddrive
- service: photos
- apple_id: APPLEID
- password: *** ENCRYPTED ***
- cookies: ****************************
@@ -93,30 +110,124 @@ d) Delete this remote
y/e/d> y
```
## iCloud Photos
The iCloud Drive backend also supports accessing iCloud Photos by setting the
`service` option to `photos`:
```console
rclone lsd iclouddrive: --iclouddrive-service photos
```
This presents a read-only hierarchy rooted at photo libraries:
- **Level 1**: Photo libraries — your personal library (`PrimarySync`) and
any Shared Photo Library (`SharedSync-XXXX`)
- **Level 2+**: Albums and folders within a library, nested recursively as in
Apple Photos
- **Leaf level**: Photos/videos within an album, including Live Photo `.MOV`
companions
Examples:
```console
# List libraries
rclone lsd iclouddrive: --iclouddrive-service photos
# List albums in your primary library
rclone lsd iclouddrive:PrimarySync/ --iclouddrive-service photos
# List photos in an album
rclone ls iclouddrive:PrimarySync/All\ Photos/ --iclouddrive-service photos
# Download a photo
rclone copy iclouddrive:PrimarySync/Favorites/IMG_0001.HEIC /tmp/ --iclouddrive-service photos
```
You can either:
- set `service = photos` in `rclone config` for a dedicated Photos remote
- keep `service = drive` and pass `--iclouddrive-service photos` when needed
### Metadata
With `--metadata`, Photos entries expose these read-only metadata keys:
- `width`
- `height`
- `added-time`
- `favorite`
- `hidden`
These metadata keys are only available when `service = photos`.
### Caching
iCloud Photos caches album listings to disk for fast subsequent access.
On the first run, listing a large album uses parallel `startRank`
partitions to fetch pages concurrently. After that, a lightweight change
check (~200ms) determines whether the cache is still valid.
Cache location: `~/.cache/rclone/iclouddrive-photos/<remote>/<zone>/`
To clear the cache: delete that directory or run `rclone config reconnect`.
### FUSE mounts
For mounting iCloud Photos via `rclone mount`, the following flags
are recommended:
rclone mount remote: /mnt/photos \
--iclouddrive-service photos \
--vfs-refresh \
--dir-cache-time 1h \
--vfs-cache-mode full \
--attr-timeout 1m \
--read-only
- `--vfs-refresh` pre-warms directory caches in the background on mount
start so that albums are ready when you browse them
- `--dir-cache-time 1h` extends the in-memory cache lifetime beyond the
default 5 minutes (change detection is fast, so this is safe)
- `--vfs-cache-mode full` caches downloaded photos and videos to local
disk for fast repeated access
- `--attr-timeout 1m` reduces kernel attribute lookups (safe because
the backend is read-only)
- `--read-only` prevents confusing write errors
The first listing of a very large album (e.g. 75,000 items in "All
Photos") can take several minutes due to API pagination limits. This
happens once — subsequent listings use the disk cache.
### Limitations
iCloud Photos is read-only. Upload, delete, rename, and move operations
are not supported.
## Advanced Data Protection
ADP is currently unsupported and need to be disabled
Advanced Data Protection is supported.
On iPhone, Settings `>` Apple Account `>` iCloud `>` 'Access iCloud Data on the Web'
must be ON, and 'Advanced Data Protection' OFF.
must be ON.
If ADP is enabled on your account, rclone requests PCS cookies after 2FA.
Apple may require approval on a trusted device before those cookies are issued.
## Troubleshooting
### Missing PCS cookies from the request
### PCS cookie errors with ADP
This means you have Advanced Data Protection (ADP) turned on. This is not supported
at the moment. If you want to use rclone you will have to turn it off. See above
for how to turn it off.
If you see `Missing PCS cookies from the request` or a `requestPCS:` error,
the ADP approval flow did not complete successfully.
You will need to clear the `cookies` and the `trust_token` fields in the config.
Or you can delete the remote config and start again.
Check that 'Access iCloud Data on the Web' is enabled and approve any prompt
on your trusted device.
You should then run `rclone reconnect remote:`.
Then run `rclone reconnect remote:`.
Note that changing the ADP setting may not take effect immediately - you may
need to wait a few hours or a day before you can get rclone to work - keep
clearing the config entry and running `rclone reconnect remote:` until rclone
functions properly.
If the remote still has stale auth state, clear the `cookies` and
`trust_token` fields in the config, or delete and recreate the remote.
<!-- autogenerated options start - DO NOT EDIT - instead edit fs.RegInfo in backend/iclouddrive/iclouddrive.go and run make backenddocs to verify --> <!-- markdownlint-disable-line line-length -->
### Standard options