iclouddrive: add read only iCloud Photos support and SRP authentication
Add read-only iCloud Photos support to the existing iclouddrive backend via `service = photos` config option. Also includes auth improvements on top of #9209's SRP authentication. **Photos features:** - 3-level hierarchy: libraries (Personal + Shared Photo Library) → albums → photos/videos - server-side smart albums (All Photos, Videos, Favorites, Screenshots, Live, Bursts, Panoramas, Slo-mo, Time-lapse, Portrait, Long Exposure, Animated, Hidden, Recently Deleted) - User-created albums and nested album folders - Live Photo `.MOV` companions as first-class entries - Edited photo versions (`-edited` suffix) and RAW alternatives - Duplicate filename dedup for camera counter wrap collisions - Parallel cold listing for large albums - Delta sync via CloudKit `changes/zone` - warm listings near-instant from disk cache - Disk cache (libraries, albums, photos) with atomic writes for crash safety - `ChangeNotify` support for FUSE mounts via `changes/zone` polling - `ListR` support for `--fast-list` and recursive operations - `--metadata` support - width, height, added-time, favorite, hidden - Fresh download URLs per file - no stale URL failures on long copies - FUSE mount documentation with recommended flags **Auth improvements over #9209:** - SMS 2FA fallback for users without trusted Apple devices - Explicit push notification request - fixes iOS/macOS 26.4+ where 409 no longer auto-pushes - Thread safety for concurrent FUSE callers (mutexes on session and client state) - Session endpoint caching - skips ~5s `/validate` round-trip on warm start - `Disconnect` support - clears auth state + disk cache - PCS cookie support for Advanced Data Protection accounts, including trusted-device approval for PCS cookies Built on @coughlanio's Photos PoC (Closes #8734) and @mikegillan's SRP auth (#9209). Fixes #7982 Co-authored-by: Chris Coughlan <chris@coughlan.io>
This commit is contained in:
@@ -1,4 +1,4 @@
|
||||
// Package api provides functionality for interacting with the iCloud API.
|
||||
// Package api provides functionality for interacting with the iCloud API
|
||||
package api
|
||||
|
||||
import (
|
||||
@@ -8,16 +8,19 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
"github.com/rclone/rclone/fs"
|
||||
"github.com/rclone/rclone/fs/config"
|
||||
"github.com/rclone/rclone/fs/fshttp"
|
||||
"github.com/rclone/rclone/lib/rest"
|
||||
)
|
||||
|
||||
const (
|
||||
baseEndpoint = "https://www.icloud.com"
|
||||
homeEndpoint = "https://www.icloud.com"
|
||||
setupEndpoint = "https://setup.icloud.com/setup/ws/1"
|
||||
authEndpoint = "https://idmsa.apple.com/appleauth/auth"
|
||||
)
|
||||
@@ -28,26 +31,21 @@ type sessionSave func(*Session)
|
||||
type Client struct {
|
||||
appleID string
|
||||
password string
|
||||
remoteName string // rclone remote name, used for cache namespacing
|
||||
srv *rest.Client
|
||||
Session *Session
|
||||
sessionSaveCallback sessionSave
|
||||
|
||||
drive *DriveService
|
||||
mu sync.Mutex // protects drive and Authenticate
|
||||
}
|
||||
|
||||
// New creates a new Client instance with the provided Apple ID, password, trust token, cookies, and session save callback.
|
||||
//
|
||||
// Parameters:
|
||||
// - appleID: the Apple ID of the user.
|
||||
// - password: the password of the user.
|
||||
// - trustToken: the trust token for the session.
|
||||
// - clientID: the client id for the session.
|
||||
// - cookies: the cookies for the session.
|
||||
// - sessionSaveCallback: the callback function to save the session.
|
||||
func New(appleID, password, trustToken string, clientID string, cookies []*http.Cookie, sessionSaveCallback sessionSave) (*Client, error) {
|
||||
// New creates a new iCloud API client and initializes its HTTP session
|
||||
func New(appleID, password, trustToken string, clientID string, cookies []*http.Cookie, sessionSaveCallback sessionSave, remoteName string) (*Client, error) {
|
||||
icloud := &Client{
|
||||
appleID: appleID,
|
||||
appleID: strings.ToLower(appleID), // Apple SRP requires lowercase in client-side proof
|
||||
password: password,
|
||||
remoteName: filepath.Base(remoteName),
|
||||
srv: rest.NewClient(fshttp.NewClient(context.Background())),
|
||||
Session: NewSession(),
|
||||
sessionSaveCallback: sessionSaveCallback,
|
||||
@@ -59,10 +57,12 @@ func New(appleID, password, trustToken string, clientID string, cookies []*http.
|
||||
return icloud, nil
|
||||
}
|
||||
|
||||
// DriveService returns the DriveService instance associated with the Client.
|
||||
// DriveService returns the DriveService instance, creating it on first call
|
||||
func (c *Client) DriveService() (*DriveService, error) {
|
||||
var err error
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
if c.drive == nil {
|
||||
var err error
|
||||
c.drive, err = NewDriveService(c)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -71,11 +71,7 @@ func (c *Client) DriveService() (*DriveService, error) {
|
||||
return c.drive, nil
|
||||
}
|
||||
|
||||
// Request makes a request and retries it if the session is invalid.
|
||||
//
|
||||
// This function is the main entry point for making requests to the iCloud
|
||||
// API. If the initial request returns a 401 (Unauthorized), it will try to
|
||||
// reauthenticate and retry the request.
|
||||
// Request makes a request to the iCloud API, re-authenticating on 401/421
|
||||
func (c *Client) Request(ctx context.Context, opts rest.Opts, request any, response any) (resp *http.Response, err error) {
|
||||
resp, err = c.Session.Request(ctx, opts, request, response)
|
||||
if err != nil && resp != nil {
|
||||
@@ -89,55 +85,93 @@ func (c *Client) Request(ctx context.Context, opts rest.Opts, request any, respo
|
||||
if c.Session.Requires2FA() {
|
||||
return nil, errors.New("trust token expired, please reauth")
|
||||
}
|
||||
return c.RequestNoReAuth(ctx, opts, request, response)
|
||||
return c.Session.Request(ctx, opts, request, response)
|
||||
}
|
||||
}
|
||||
return resp, err
|
||||
}
|
||||
|
||||
// RequestNoReAuth makes a request without re-authenticating.
|
||||
//
|
||||
// This function is useful when you have a session that is already
|
||||
// authenticated, but you need to make a request without triggering
|
||||
// a re-authentication.
|
||||
func (c *Client) RequestNoReAuth(ctx context.Context, opts rest.Opts, request any, response any) (resp *http.Response, err error) {
|
||||
// Make the request without re-authenticating
|
||||
resp, err = c.Session.Request(ctx, opts, request, response)
|
||||
return resp, err
|
||||
}
|
||||
|
||||
// Authenticate authenticates the client with the iCloud API.
|
||||
// Authenticate authenticates the client, reusing existing session if valid
|
||||
func (c *Client) Authenticate(ctx context.Context) error {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
// Skip /validate round-trip when saved session has cookies + service endpoints
|
||||
// Native client behavior: use cached session, reauth lazily on 401/421
|
||||
if c.Session.Cookies != nil && len(c.Session.AccountInfo.Webservices) > 0 {
|
||||
fs.Debugf(nil, "iclouddrive: reusing saved session")
|
||||
return nil
|
||||
}
|
||||
// Try loading cached service endpoints to avoid /validate round-trip (~5s)
|
||||
if c.Session.Cookies != nil && c.loadCachedWebservices() {
|
||||
fs.Debugf(nil, "iclouddrive: reusing session with cached endpoints")
|
||||
return nil
|
||||
}
|
||||
if c.Session.Cookies != nil {
|
||||
if err := c.Session.ValidateSession(ctx); err == nil {
|
||||
fs.Debugf("icloud", "Valid session, no need to reauth")
|
||||
fs.Debugf(nil, "iclouddrive: valid session, no need to reauth")
|
||||
c.saveCachedWebservices()
|
||||
return nil
|
||||
}
|
||||
c.Session.Cookies = nil
|
||||
}
|
||||
|
||||
fs.Debugf("icloud", "Authenticating as %s\n", c.appleID)
|
||||
fs.Debugf(nil, "iclouddrive: authenticating")
|
||||
err := c.Session.SignIn(ctx, c.appleID, c.password)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// If 2FA is required, don't try AuthWithToken yet — the caller
|
||||
// must complete 2FA first, then call AuthWithToken.
|
||||
// If 2FA is required, skip AuthWithToken - caller must complete 2FA first
|
||||
if c.Session.Requires2FA() {
|
||||
return nil
|
||||
}
|
||||
|
||||
err = c.Session.AuthWithToken(ctx)
|
||||
if err == nil && c.sessionSaveCallback != nil {
|
||||
c.sessionSaveCallback(c.Session)
|
||||
if err == nil {
|
||||
c.saveCachedWebservices()
|
||||
if c.sessionSaveCallback != nil {
|
||||
c.sessionSaveCallback(c.Session)
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
// SignIn signs in the client using the provided context and credentials.
|
||||
func (c *Client) SignIn(ctx context.Context) error {
|
||||
return c.Session.SignIn(ctx, c.appleID, c.password)
|
||||
// loadCachedWebservices loads service endpoints from disk cache
|
||||
func (c *Client) loadCachedWebservices() bool {
|
||||
data, err := os.ReadFile(filepath.Join(config.GetCacheDir(), cacheSubdir, c.remoteName, "webservices.json"))
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
var ws map[string]*webService
|
||||
if err := json.Unmarshal(data, &ws); err != nil {
|
||||
return false
|
||||
}
|
||||
if len(ws) == 0 {
|
||||
return false
|
||||
}
|
||||
c.Session.AccountInfo.Webservices = ws
|
||||
return true
|
||||
}
|
||||
|
||||
// saveCachedWebservices persists service endpoints to disk
|
||||
func (c *Client) saveCachedWebservices() {
|
||||
if len(c.Session.AccountInfo.Webservices) == 0 {
|
||||
return
|
||||
}
|
||||
saveJSONCache(filepath.Join(config.GetCacheDir(), cacheSubdir, c.remoteName), "webservices.json", c.Session.AccountInfo.Webservices)
|
||||
}
|
||||
|
||||
// CacheDir returns the disk cache directory for this remote
|
||||
func (c *Client) CacheDir() string {
|
||||
return filepath.Join(config.GetCacheDir(), cacheSubdir, c.remoteName)
|
||||
}
|
||||
|
||||
// ClearCacheDir removes all disk cache files for a remote
|
||||
func ClearCacheDir(remoteName string) {
|
||||
dir := filepath.Join(config.GetCacheDir(), cacheSubdir, filepath.Base(remoteName))
|
||||
if err := os.RemoveAll(dir); err != nil {
|
||||
fs.Debugf(nil, "iclouddrive: failed to clear cache: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// IntoReader marshals the provided values into a JSON encoded reader
|
||||
@@ -155,19 +189,19 @@ type RequestError struct {
|
||||
Text string
|
||||
}
|
||||
|
||||
// Error satisfy the error interface.
|
||||
// Error satisfies the error interface
|
||||
func (e *RequestError) Error() string {
|
||||
return fmt.Sprintf("%s: %s", e.Text, e.Status)
|
||||
}
|
||||
|
||||
func newRequestError(Status string, Text string) *RequestError {
|
||||
func newRequestError(status string, text string) *RequestError {
|
||||
return &RequestError{
|
||||
Status: strings.ToLower(Status),
|
||||
Text: Text,
|
||||
Status: strings.ToLower(status),
|
||||
Text: text,
|
||||
}
|
||||
}
|
||||
|
||||
// newErr orf makes a new error from sprintf parameters.
|
||||
func newRequestErrorf(Status string, Text string, Parameters ...any) *RequestError {
|
||||
return newRequestError(strings.ToLower(Status), fmt.Sprintf(Text, Parameters...))
|
||||
// newRequestErrorf makes a new error from sprintf parameters
|
||||
func newRequestErrorf(status string, text string, params ...any) *RequestError {
|
||||
return newRequestError(status, fmt.Sprintf(text, params...))
|
||||
}
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
+441
-227
@@ -2,16 +2,21 @@ package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"maps"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"slices"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/oracle/oci-go-sdk/v65/common"
|
||||
"github.com/rclone/rclone/fs"
|
||||
"github.com/rclone/rclone/fs/fshttp"
|
||||
"github.com/rclone/rclone/lib/rest"
|
||||
@@ -32,6 +37,7 @@ type Session struct {
|
||||
Cookies []*http.Cookie `json:"cookies"`
|
||||
AccountInfo AccountInfo `json:"account_info"`
|
||||
|
||||
mu sync.Mutex `json:"-"` // protects session fields during concurrent Request calls
|
||||
srv *rest.Client `json:"-"`
|
||||
needs2FA bool `json:"-"` // set when SRP signin returns 409
|
||||
}
|
||||
@@ -45,20 +51,117 @@ type srpInitResponse struct {
|
||||
C string `json:"c"`
|
||||
}
|
||||
|
||||
// String returns the session as a string
|
||||
// func (s *Session) String() string {
|
||||
// jsession, _ := json.Marshal(s)
|
||||
// return string(jsession)
|
||||
// }
|
||||
|
||||
// Request makes a request
|
||||
func (s *Session) Request(ctx context.Context, opts rest.Opts, request any, response any) (*http.Response, error) {
|
||||
resp, err := s.srv.CallJSON(ctx, &opts, &request, &response)
|
||||
|
||||
if err != nil {
|
||||
return resp, err
|
||||
func cookieValueFingerprint(value string) string {
|
||||
if value == "" {
|
||||
return ""
|
||||
}
|
||||
sum := sha256.Sum256([]byte(value))
|
||||
return hex.EncodeToString(sum[:6])
|
||||
}
|
||||
|
||||
func authStateBodySummary(body []byte) string {
|
||||
summary := []string{fmt.Sprintf("bytes=%d", len(body)), "hash=" + cookieValueFingerprint(string(body))}
|
||||
var top map[string]json.RawMessage
|
||||
if err := json.Unmarshal(body, &top); err != nil {
|
||||
return strings.Join(summary, " ")
|
||||
}
|
||||
keys := make([]string, 0, len(top))
|
||||
for key := range top {
|
||||
keys = append(keys, key)
|
||||
}
|
||||
slices.Sort(keys)
|
||||
summary = append(summary, fmt.Sprintf("keys=%v", keys))
|
||||
if _, ok := top["phoneNumberVerification"]; ok {
|
||||
summary = append(summary, "wrapped=true")
|
||||
}
|
||||
return strings.Join(summary, " ")
|
||||
}
|
||||
|
||||
func cookieDebugSummary(c *http.Cookie) string {
|
||||
parts := []string{c.Name}
|
||||
if c.Value == "" {
|
||||
parts = append(parts, "empty", "len=0")
|
||||
} else {
|
||||
parts = append(parts, "set", fmt.Sprintf("len=%d", len(c.Value)), "hash="+cookieValueFingerprint(c.Value))
|
||||
}
|
||||
if c.Path != "" {
|
||||
parts = append(parts, "path="+c.Path)
|
||||
}
|
||||
if c.Domain != "" {
|
||||
parts = append(parts, "domain="+c.Domain)
|
||||
}
|
||||
if c.MaxAge != 0 {
|
||||
parts = append(parts, fmt.Sprintf("maxAge=%d", c.MaxAge))
|
||||
}
|
||||
if !c.Expires.IsZero() {
|
||||
parts = append(parts, "expires")
|
||||
}
|
||||
return strings.Join(parts, ",")
|
||||
}
|
||||
|
||||
func cookieDebugSummaries(cookies []*http.Cookie) []string {
|
||||
if len(cookies) == 0 {
|
||||
return nil
|
||||
}
|
||||
out := make([]string, 0, len(cookies))
|
||||
for _, c := range cookies {
|
||||
out = append(out, cookieDebugSummary(c))
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func cookieJarDebugSummaries(cookies []*http.Cookie) []string {
|
||||
if len(cookies) == 0 {
|
||||
return nil
|
||||
}
|
||||
out := make([]string, 0, len(cookies))
|
||||
for _, c := range cookies {
|
||||
summary := c.Name
|
||||
if c.Value == "" {
|
||||
summary += ",empty,len=0"
|
||||
} else {
|
||||
summary += fmt.Sprintf(",len=%d,hash=%s", len(c.Value), cookieValueFingerprint(c.Value))
|
||||
}
|
||||
out = append(out, summary)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (s *Session) mergeCookies(cookies []*http.Cookie) {
|
||||
if len(cookies) == 0 {
|
||||
return
|
||||
}
|
||||
existing := make(map[string]int, len(s.Cookies))
|
||||
for i, c := range s.Cookies {
|
||||
existing[c.Name] = i
|
||||
}
|
||||
for _, c := range cookies {
|
||||
if c.Value == "" {
|
||||
if idx, ok := existing[c.Name]; ok {
|
||||
fs.Debugf(nil, "iclouddrive: deleting empty auth cookie: %s", cookieDebugSummary(c))
|
||||
s.Cookies = append(s.Cookies[:idx], s.Cookies[idx+1:]...)
|
||||
delete(existing, c.Name)
|
||||
for j := idx; j < len(s.Cookies); j++ {
|
||||
existing[s.Cookies[j].Name] = j
|
||||
}
|
||||
} else {
|
||||
fs.Debugf(nil, "iclouddrive: ignoring empty auth cookie tombstone for missing cookie: %s", cookieDebugSummary(c))
|
||||
}
|
||||
continue
|
||||
}
|
||||
if idx, ok := existing[c.Name]; ok {
|
||||
s.Cookies[idx] = c
|
||||
} else {
|
||||
s.Cookies = append(s.Cookies, c)
|
||||
existing[c.Name] = len(s.Cookies) - 1
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// extractHeaders reads Apple session headers from an HTTP response into the session
|
||||
// Does NOT acquire s.mu - caller is responsible for locking if needed
|
||||
func (s *Session) extractHeaders(resp *http.Response) {
|
||||
s.mergeCookies(resp.Cookies())
|
||||
if val := resp.Header.Get("X-Apple-ID-Account-Country"); val != "" {
|
||||
s.AccountCountry = val
|
||||
}
|
||||
@@ -77,7 +180,17 @@ func (s *Session) Request(ctx context.Context, opts rest.Opts, request any, resp
|
||||
if val := resp.Header.Get("X-Apple-Auth-Attributes"); val != "" {
|
||||
s.AuthAttributes = val
|
||||
}
|
||||
}
|
||||
|
||||
// Request makes a JSON API request and extracts session headers
|
||||
func (s *Session) Request(ctx context.Context, opts rest.Opts, request any, response any) (*http.Response, error) {
|
||||
resp, err := s.srv.CallJSON(ctx, &opts, &request, &response)
|
||||
if err != nil {
|
||||
return resp, err
|
||||
}
|
||||
s.mu.Lock()
|
||||
s.extractHeaders(resp)
|
||||
s.mu.Unlock()
|
||||
return resp, nil
|
||||
}
|
||||
|
||||
@@ -89,13 +202,8 @@ func (s *Session) Requires2FA() bool {
|
||||
return s.AccountInfo.DsInfo != nil && s.AccountInfo.DsInfo.HsaVersion == 2 && s.AccountInfo.HsaChallengeRequired
|
||||
}
|
||||
|
||||
// SignIn performs SRP-based authentication against Apple's idmsa endpoint.
|
||||
// SignIn performs SRP-based authentication against Apple's idmsa endpoint
|
||||
func (s *Session) SignIn(ctx context.Context, appleID, password string) error {
|
||||
// Apple's SRP implementation expects a lowercase account name.
|
||||
// The old plaintext flow didn't need this because the server normalized
|
||||
// it, but SRP uses the username in client-side proof computation (M1).
|
||||
appleID = strings.ToLower(appleID)
|
||||
|
||||
// Step 1: Initialize the auth session
|
||||
if err := s.authStart(ctx); err != nil {
|
||||
return fmt.Errorf("authStart: %w", err)
|
||||
@@ -106,8 +214,11 @@ func (s *Session) SignIn(ctx context.Context, appleID, password string) error {
|
||||
return fmt.Errorf("authFederate: %w", err)
|
||||
}
|
||||
|
||||
// Step 3: SRP init — send client public value A, get salt + B
|
||||
client := newSRPClient()
|
||||
// Step 3: SRP init - send client public value A, get salt + B
|
||||
client, err := newSRPClient()
|
||||
if err != nil {
|
||||
return fmt.Errorf("newSRPClient: %w", err)
|
||||
}
|
||||
aBase64 := base64.StdEncoding.EncodeToString(client.getABytes())
|
||||
|
||||
initResp, err := s.authSRPInit(ctx, aBase64, appleID)
|
||||
@@ -130,9 +241,11 @@ func (s *Session) SignIn(ctx context.Context, appleID, password string) error {
|
||||
if err != nil {
|
||||
return fmt.Errorf("derivePassword: %w", err)
|
||||
}
|
||||
client.processChallenge([]byte(appleID), derivedKey, salt, serverB)
|
||||
if err := client.processChallenge([]byte(appleID), derivedKey, salt, serverB); err != nil {
|
||||
return fmt.Errorf("processChallenge: %w", err)
|
||||
}
|
||||
|
||||
// Step 5: Complete — send M1, M2 proofs
|
||||
// Step 5: Complete - send M1, M2 proofs
|
||||
m1Base64 := base64.StdEncoding.EncodeToString(client.M1)
|
||||
m2Base64 := base64.StdEncoding.EncodeToString(client.M2)
|
||||
|
||||
@@ -143,7 +256,7 @@ func (s *Session) SignIn(ctx context.Context, appleID, password string) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// authStart initializes the SRP auth session by hitting the authorize/signin endpoint.
|
||||
// authStart initializes the SRP auth session by hitting the authorize/signin endpoint
|
||||
func (s *Session) authStart(ctx context.Context) error {
|
||||
if s.FrameID == "" {
|
||||
s.FrameID = strings.ToLower(uuid.New().String())
|
||||
@@ -184,20 +297,11 @@ func (s *Session) authStart(ctx context.Context) error {
|
||||
return fmt.Errorf("authStart: unexpected status %s", resp.Status)
|
||||
}
|
||||
|
||||
if val := resp.Header.Get("X-Apple-Auth-Attributes"); val != "" {
|
||||
s.AuthAttributes = val
|
||||
}
|
||||
if val := resp.Header.Get("scnt"); val != "" {
|
||||
s.Scnt = val
|
||||
}
|
||||
if val := resp.Header.Get("X-Apple-ID-Session-Id"); val != "" {
|
||||
s.SessionID = val
|
||||
}
|
||||
|
||||
s.extractHeaders(resp)
|
||||
return nil
|
||||
}
|
||||
|
||||
// authFederate submits the account name to Apple's federate endpoint.
|
||||
// authFederate submits the account name to Apple's federate endpoint
|
||||
func (s *Session) authFederate(ctx context.Context, accountName string) error {
|
||||
values := map[string]any{
|
||||
"accountName": accountName,
|
||||
@@ -224,16 +328,7 @@ func (s *Session) authFederate(ctx context.Context, accountName string) error {
|
||||
}
|
||||
_ = resp.Body.Close()
|
||||
|
||||
if val := resp.Header.Get("X-Apple-Auth-Attributes"); val != "" {
|
||||
s.AuthAttributes = val
|
||||
}
|
||||
if val := resp.Header.Get("scnt"); val != "" {
|
||||
s.Scnt = val
|
||||
}
|
||||
if val := resp.Header.Get("X-Apple-ID-Session-Id"); val != "" {
|
||||
s.SessionID = val
|
||||
}
|
||||
|
||||
s.extractHeaders(resp)
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return fmt.Errorf("authFederate: unexpected status %s", resp.Status)
|
||||
}
|
||||
@@ -241,7 +336,7 @@ func (s *Session) authFederate(ctx context.Context, accountName string) error {
|
||||
}
|
||||
|
||||
// authSRPInit sends the client's public value A to the server and retrieves
|
||||
// the salt, server public value B, iteration count, protocol, and challenge.
|
||||
// the salt, server public value B, iteration count, protocol, and challenge
|
||||
func (s *Session) authSRPInit(ctx context.Context, aBase64, accountName string) (*srpInitResponse, error) {
|
||||
values := map[string]any{
|
||||
"a": aBase64,
|
||||
@@ -267,18 +362,12 @@ func (s *Session) authSRPInit(ctx context.Context, aBase64, accountName string)
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if val := resp.Header.Get("scnt"); val != "" {
|
||||
s.Scnt = val
|
||||
}
|
||||
if val := resp.Header.Get("X-Apple-ID-Session-Id"); val != "" {
|
||||
s.SessionID = val
|
||||
}
|
||||
|
||||
s.extractHeaders(resp)
|
||||
return &initResp, nil
|
||||
}
|
||||
|
||||
// authSRPComplete sends the SRP proofs M1 and M2 to complete authentication.
|
||||
// Returns nil on success (200 or 409/2FA needed).
|
||||
// authSRPComplete sends the SRP proofs M1 and M2 to complete authentication
|
||||
// Returns nil on success (200 or 409/2FA needed)
|
||||
func (s *Session) authSRPComplete(ctx context.Context, accountName, m1Base64, m2Base64, c string) error {
|
||||
trustTokens := []string{}
|
||||
if s.TrustToken != "" {
|
||||
@@ -305,7 +394,6 @@ func (s *Session) authSRPComplete(ctx context.Context, accountName, m1Base64, m2
|
||||
ExtraHeaders: s.getSRPAuthHeaders(),
|
||||
RootURL: authEndpoint,
|
||||
IgnoreStatus: true,
|
||||
NoResponse: true,
|
||||
Body: body,
|
||||
}
|
||||
|
||||
@@ -313,51 +401,65 @@ func (s *Session) authSRPComplete(ctx context.Context, accountName, m1Base64, m2
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
respBody, _ := io.ReadAll(resp.Body)
|
||||
_ = resp.Body.Close()
|
||||
|
||||
// Extract updated headers
|
||||
if val := resp.Header.Get("X-Apple-Auth-Attributes"); val != "" {
|
||||
s.AuthAttributes = val
|
||||
}
|
||||
if val := resp.Header.Get("X-Apple-Session-Token"); val != "" {
|
||||
s.SessionToken = val
|
||||
}
|
||||
if val := resp.Header.Get("scnt"); val != "" {
|
||||
s.Scnt = val
|
||||
}
|
||||
if val := resp.Header.Get("X-Apple-ID-Session-Id"); val != "" {
|
||||
s.SessionID = val
|
||||
}
|
||||
if val := resp.Header.Get("X-Apple-ID-Account-Country"); val != "" {
|
||||
s.AccountCountry = val
|
||||
}
|
||||
if val := resp.Header.Get("X-Apple-TwoSV-Trust-Token"); val != "" {
|
||||
s.TrustToken = val
|
||||
}
|
||||
s.extractHeaders(resp)
|
||||
|
||||
switch resp.StatusCode {
|
||||
case http.StatusOK:
|
||||
fs.Debugf("icloud", "SRP sign in successful")
|
||||
fs.Debugf(nil, "iclouddrive: SRP sign in successful")
|
||||
return nil
|
||||
case http.StatusConflict:
|
||||
// 409 = 2FA required, this is expected
|
||||
fs.Debugf("icloud", "SRP sign in requires 2FA")
|
||||
// 409 = 2FA required
|
||||
fs.Debugf(nil, "iclouddrive: SRP sign in requires 2FA, response: %s", respBody)
|
||||
s.needs2FA = true
|
||||
return nil
|
||||
case http.StatusPreconditionFailed:
|
||||
// 412 = non-2FA account needs repair/complete step
|
||||
fs.Debugf(nil, "iclouddrive: SRP sign in returned 412, attempting repair/complete")
|
||||
return s.authRepairComplete(ctx)
|
||||
case http.StatusForbidden:
|
||||
return fmt.Errorf("sign in failed: incorrect username or password")
|
||||
default:
|
||||
return fmt.Errorf("sign in failed: %s", resp.Status)
|
||||
return fmt.Errorf("sign in failed: %s: %s", resp.Status, respBody)
|
||||
}
|
||||
}
|
||||
|
||||
// getAuthOrigin returns the origin URL for auth requests.
|
||||
// Supports both global (idmsa.apple.com) and China (idmsa.apple.com.cn) endpoints.
|
||||
// authRepairComplete handles the repair flow for non-2FA accounts that return 412
|
||||
func (s *Session) authRepairComplete(ctx context.Context) error {
|
||||
body, err := IntoReader(map[string]any{})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
opts := rest.Opts{
|
||||
Method: "POST",
|
||||
Path: "/repair/complete",
|
||||
ExtraHeaders: s.getSRPAuthHeaders(),
|
||||
RootURL: authEndpoint,
|
||||
IgnoreStatus: true,
|
||||
NoResponse: true,
|
||||
Body: body,
|
||||
}
|
||||
resp, err := s.srv.Call(ctx, &opts)
|
||||
if err != nil {
|
||||
return fmt.Errorf("repair/complete failed: %w", err)
|
||||
}
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return fmt.Errorf("repair/complete returned %s", resp.Status)
|
||||
}
|
||||
s.extractHeaders(resp)
|
||||
fs.Debugf(nil, "iclouddrive: repair/complete successful")
|
||||
return nil
|
||||
}
|
||||
|
||||
// getAuthOrigin returns the origin URL for auth requests
|
||||
// Supports both global (idmsa.apple.com) and China (idmsa.apple.com.cn) endpoints
|
||||
func getAuthOrigin() string {
|
||||
return strings.TrimSuffix(authEndpoint, "/appleauth/auth")
|
||||
}
|
||||
|
||||
// getSRPAuthHeaders returns headers needed for SRP auth requests.
|
||||
// getSRPAuthHeaders returns headers needed for SRP auth requests
|
||||
func (s *Session) getSRPAuthHeaders() map[string]string {
|
||||
frameTag := "auth-" + s.FrameID
|
||||
authOrigin := getAuthOrigin()
|
||||
@@ -414,14 +516,110 @@ func (s *Session) AuthWithToken(ctx context.Context) error {
|
||||
}
|
||||
|
||||
resp, err := s.Request(ctx, opts, nil, &s.AccountInfo)
|
||||
if err == nil {
|
||||
s.Cookies = resp.Cookies()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fs.Debugf(nil, "iclouddrive: accountLogin response cookies: %v", cookieDebugSummaries(resp.Cookies()))
|
||||
fs.Debugf(nil, "iclouddrive: session cookie jar after accountLogin: %v", cookieJarDebugSummaries(s.Cookies))
|
||||
|
||||
// Acquire PCS cookies if Advanced Data Protection is enabled
|
||||
if ws := s.AccountInfo.Webservices["ckdatabasews"]; ws != nil && ws.PcsRequired {
|
||||
fs.Debugf(nil, "iclouddrive: ADP detected (pcsRequired=true)")
|
||||
if s.hasPCSCookies() {
|
||||
fs.Debugf(nil, "iclouddrive: PCS cookies already present, skipping acquisition")
|
||||
} else {
|
||||
if err := s.acquirePCSCookies(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
} else {
|
||||
fs.Debugf(nil, "iclouddrive: no ADP (pcsRequired=false)")
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// hasPCSCookies checks if the required PCS cookies for Photos are already present
|
||||
func (s *Session) hasPCSCookies() bool {
|
||||
var hasPhotos, hasSharing bool
|
||||
for _, c := range s.Cookies {
|
||||
switch c.Name {
|
||||
case "X-APPLE-WEBAUTH-PCS-Photos":
|
||||
hasPhotos = true
|
||||
case "X-APPLE-WEBAUTH-PCS-Sharing":
|
||||
hasSharing = true
|
||||
}
|
||||
}
|
||||
return hasPhotos && hasSharing
|
||||
}
|
||||
|
||||
// acquirePCSCookies requests PCS cookies for ADP-enabled accounts
|
||||
// May require user approval on a trusted device (polls every 10s, max 5 min)
|
||||
func (s *Session) acquirePCSCookies(ctx context.Context) error {
|
||||
fs.Logf(nil, "iclouddrive: Advanced Data Protection enabled, requesting PCS cookies")
|
||||
const maxAttempts = 30 // 30 * 10s = 5 minutes max
|
||||
for attempt := 0; attempt < maxAttempts; attempt++ {
|
||||
fs.Debugf(nil, "iclouddrive: requestPCS outgoing cookies: %v", cookieJarDebugSummaries(s.Cookies))
|
||||
values := map[string]any{
|
||||
"appName": "photos",
|
||||
"derivedFromUserAction": true,
|
||||
}
|
||||
body, err := IntoReader(values)
|
||||
if err != nil {
|
||||
return fmt.Errorf("requestPCS: %w", err)
|
||||
}
|
||||
opts := rest.Opts{
|
||||
Method: "POST",
|
||||
Path: "/requestPCS",
|
||||
ExtraHeaders: s.GetHeaders(map[string]string{}),
|
||||
RootURL: setupEndpoint,
|
||||
}
|
||||
opts.Body = body
|
||||
var pcsResp struct {
|
||||
Status string `json:"status"`
|
||||
Message string `json:"message"`
|
||||
}
|
||||
resp, err := s.Request(ctx, opts, nil, &pcsResp)
|
||||
if err != nil {
|
||||
return fmt.Errorf("requestPCS: %w", err)
|
||||
}
|
||||
fs.Debugf(nil, "iclouddrive: requestPCS response cookies: %v", cookieDebugSummaries(resp.Cookies()))
|
||||
fs.Debugf(nil, "iclouddrive: requestPCS response: status=%q message=%q cookies=%d",
|
||||
pcsResp.Status, pcsResp.Message, len(resp.Cookies()))
|
||||
if pcsResp.Status == "success" {
|
||||
if !s.hasPCSCookies() {
|
||||
return fmt.Errorf("requestPCS: server returned success but PCS cookies missing")
|
||||
}
|
||||
fs.Logf(nil, "iclouddrive: PCS cookies acquired")
|
||||
return nil
|
||||
}
|
||||
// Device consent required - poll until approved
|
||||
fs.Logf(nil, "iclouddrive: waiting for device approval for PCS (%s)", pcsResp.Message)
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
case <-time.After(10 * time.Second):
|
||||
}
|
||||
}
|
||||
return fmt.Errorf("requestPCS: timed out waiting for device approval after 5 minutes")
|
||||
}
|
||||
|
||||
// RequestPushNotification explicitly requests a push notification to trusted devices
|
||||
// Required for iOS 26.4+ where the SRP 409 response no longer auto-pushes
|
||||
func (s *Session) RequestPushNotification(ctx context.Context) error {
|
||||
opts := rest.Opts{
|
||||
Method: "PUT",
|
||||
Path: "/verify/trusteddevice/securitycode",
|
||||
ExtraHeaders: s.GetAuthHeaders(map[string]string{}),
|
||||
RootURL: authEndpoint,
|
||||
NoResponse: true,
|
||||
}
|
||||
|
||||
_, err := s.Request(ctx, opts, nil, nil)
|
||||
return err
|
||||
}
|
||||
|
||||
// Validate2FACode validates the 2FA code
|
||||
// Validate2FACode validates the 2FA code from a trusted device push notification
|
||||
func (s *Session) Validate2FACode(ctx context.Context, code string) error {
|
||||
values := map[string]any{"securityCode": map[string]string{"code": code}}
|
||||
body, err := IntoReader(values)
|
||||
@@ -450,6 +648,132 @@ func (s *Session) Validate2FACode(ctx context.Context, code string) error {
|
||||
return fmt.Errorf("validate2FACode failed: %w", err)
|
||||
}
|
||||
|
||||
// TrustedPhoneNumber represents a phone number that can receive SMS verification codes
|
||||
type TrustedPhoneNumber struct {
|
||||
ID int `json:"id"`
|
||||
NumberWithDialCode string `json:"numberWithDialCode"`
|
||||
ObfuscatedNumber string `json:"obfuscatedNumber"`
|
||||
PushMode string `json:"pushMode"`
|
||||
NonFTEU bool `json:"nonFTEU"`
|
||||
}
|
||||
|
||||
// AuthStateResponse is the response from GET /appleauth/auth after sign-in
|
||||
// Some accounts return fields at top level, others nest them under phoneNumberVerification
|
||||
type AuthStateResponse struct {
|
||||
TrustedPhoneNumbers []TrustedPhoneNumber `json:"trustedPhoneNumbers"`
|
||||
TrustedPhoneNumber *TrustedPhoneNumber `json:"trustedPhoneNumber"`
|
||||
NoTrustedDevices bool `json:"noTrustedDevices"`
|
||||
AuthenticationType string `json:"authenticationType"`
|
||||
Hsa2Account bool `json:"hsa2Account"`
|
||||
PhoneNumberVerification *AuthStateResponse `json:"phoneNumberVerification"`
|
||||
}
|
||||
|
||||
// GetAuthState retrieves the current auth state including trusted phone numbers for SMS 2FA
|
||||
func (s *Session) GetAuthState(ctx context.Context) (*AuthStateResponse, error) {
|
||||
opts := rest.Opts{
|
||||
Method: "GET",
|
||||
Path: "",
|
||||
ExtraHeaders: s.GetAuthHeaders(map[string]string{}),
|
||||
RootURL: authEndpoint,
|
||||
ContentLength: int64Ptr(0),
|
||||
}
|
||||
// Use srv.Call directly to capture the raw response body for debugging
|
||||
resp, err := s.srv.Call(ctx, &opts)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("getAuthState: %w", err)
|
||||
}
|
||||
s.mu.Lock()
|
||||
s.extractHeaders(resp)
|
||||
s.mu.Unlock()
|
||||
body, readErr := io.ReadAll(resp.Body)
|
||||
_ = resp.Body.Close()
|
||||
if readErr != nil {
|
||||
return nil, fmt.Errorf("getAuthState read: %w", readErr)
|
||||
}
|
||||
fs.Debugf(nil, "iclouddrive: auth state response summary: %s", authStateBodySummary(body))
|
||||
var state AuthStateResponse
|
||||
if err := json.Unmarshal(body, &state); err != nil {
|
||||
fs.Debugf(nil, "iclouddrive: auth state parse failed: %s", authStateBodySummary(body))
|
||||
return nil, fmt.Errorf("getAuthState unmarshal: %w", err)
|
||||
}
|
||||
// Some accounts nest auth data under phoneNumberVerification
|
||||
if state.PhoneNumberVerification != nil && state.AuthenticationType == "" {
|
||||
fs.Debugf(nil, "iclouddrive: unwrapping phoneNumberVerification envelope")
|
||||
n := state.PhoneNumberVerification
|
||||
state.TrustedPhoneNumbers = n.TrustedPhoneNumbers
|
||||
state.TrustedPhoneNumber = n.TrustedPhoneNumber
|
||||
state.NoTrustedDevices = n.NoTrustedDevices
|
||||
state.AuthenticationType = n.AuthenticationType
|
||||
state.Hsa2Account = n.Hsa2Account
|
||||
state.PhoneNumberVerification = nil
|
||||
}
|
||||
fs.Debugf(nil, "iclouddrive: auth state: type=%s hsa2=%v noTrustedDevices=%v phones=%d phoneSingular=%v",
|
||||
state.AuthenticationType, state.Hsa2Account, state.NoTrustedDevices,
|
||||
len(state.TrustedPhoneNumbers), state.TrustedPhoneNumber != nil)
|
||||
// Fall back to singular trustedPhoneNumber when plural array is empty
|
||||
// Some accounts return only the singular form (SMS-only, no trusted devices)
|
||||
if len(state.TrustedPhoneNumbers) == 0 && state.TrustedPhoneNumber != nil {
|
||||
fs.Debugf(nil, "iclouddrive: using singular trustedPhoneNumber (id=%d) as fallback",
|
||||
state.TrustedPhoneNumber.ID)
|
||||
state.TrustedPhoneNumbers = []TrustedPhoneNumber{*state.TrustedPhoneNumber}
|
||||
}
|
||||
return &state, nil
|
||||
}
|
||||
|
||||
// RequestSMSCode triggers SMS code delivery to a trusted phone number
|
||||
func (s *Session) RequestSMSCode(ctx context.Context, phoneID int, mode string) error {
|
||||
values := map[string]any{
|
||||
"phoneNumber": map[string]any{"id": phoneID},
|
||||
"mode": mode,
|
||||
}
|
||||
body, err := IntoReader(values)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
opts := rest.Opts{
|
||||
Method: "PUT",
|
||||
Path: "/verify/phone",
|
||||
ExtraHeaders: s.GetAuthHeaders(map[string]string{}),
|
||||
RootURL: authEndpoint,
|
||||
Body: body,
|
||||
NoResponse: true,
|
||||
}
|
||||
_, err = s.Request(ctx, opts, nil, nil)
|
||||
if err != nil {
|
||||
return fmt.Errorf("requestSMSCode: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// ValidateSMSCode validates a 2FA code received via SMS
|
||||
func (s *Session) ValidateSMSCode(ctx context.Context, code string, phoneID int, mode string) error {
|
||||
values := map[string]any{
|
||||
"securityCode": map[string]string{"code": code},
|
||||
"phoneNumber": map[string]any{"id": phoneID},
|
||||
"mode": mode,
|
||||
}
|
||||
body, err := IntoReader(values)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
opts := rest.Opts{
|
||||
Method: "POST",
|
||||
Path: "/verify/phone/securitycode",
|
||||
ExtraHeaders: s.GetAuthHeaders(map[string]string{}),
|
||||
RootURL: authEndpoint,
|
||||
Body: body,
|
||||
NoResponse: true,
|
||||
}
|
||||
_, err = s.Request(ctx, opts, nil, nil)
|
||||
if err == nil {
|
||||
if err := s.TrustSession(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("validateSMSCode: %w", err)
|
||||
}
|
||||
|
||||
// TrustSession trusts the session
|
||||
func (s *Session) TrustSession(ctx context.Context) error {
|
||||
opts := rest.Opts{
|
||||
@@ -458,7 +782,7 @@ func (s *Session) TrustSession(ctx context.Context) error {
|
||||
ExtraHeaders: s.GetAuthHeaders(map[string]string{}),
|
||||
RootURL: authEndpoint,
|
||||
NoResponse: true,
|
||||
ContentLength: common.Int64(0),
|
||||
ContentLength: int64Ptr(0),
|
||||
}
|
||||
|
||||
_, err := s.Request(ctx, opts, nil, nil)
|
||||
@@ -476,25 +800,26 @@ func (s *Session) ValidateSession(ctx context.Context) error {
|
||||
Path: "/validate",
|
||||
ExtraHeaders: s.GetHeaders(map[string]string{}),
|
||||
RootURL: setupEndpoint,
|
||||
ContentLength: common.Int64(0),
|
||||
ContentLength: int64Ptr(0),
|
||||
}
|
||||
_, err := s.Request(ctx, opts, nil, &s.AccountInfo)
|
||||
if err != nil {
|
||||
return fmt.Errorf("validateSession failed: %w", err)
|
||||
}
|
||||
|
||||
s.needs2FA = false
|
||||
return nil
|
||||
}
|
||||
|
||||
// GetAuthHeaders returns the authentication headers for the session.
|
||||
// Used for 2FA validation and trust requests to idmsa.apple.com.
|
||||
// GetAuthHeaders returns the authentication headers for the session
|
||||
// Used for 2FA validation and trust requests to idmsa.apple.com
|
||||
func (s *Session) GetAuthHeaders(overwrite map[string]string) map[string]string {
|
||||
headers := s.getSRPAuthHeaders()
|
||||
maps.Copy(headers, overwrite)
|
||||
return headers
|
||||
}
|
||||
|
||||
// GetHeaders Gets the authentication headers required for a request
|
||||
// GetHeaders returns the authentication headers required for a request
|
||||
func (s *Session) GetHeaders(overwrite map[string]string) map[string]string {
|
||||
headers := GetCommonHeaders(map[string]string{})
|
||||
headers["Cookie"] = s.GetCookieString()
|
||||
@@ -502,17 +827,26 @@ func (s *Session) GetHeaders(overwrite map[string]string) map[string]string {
|
||||
return headers
|
||||
}
|
||||
|
||||
// GetCookieString returns the cookie header string for the session.
|
||||
// GetCookieString returns the cookie header string for the session
|
||||
func (s *Session) GetCookieString() string {
|
||||
cookieHeader := ""
|
||||
// we only care about name and value.
|
||||
var b strings.Builder
|
||||
first := true
|
||||
for _, cookie := range s.Cookies {
|
||||
cookieHeader = cookieHeader + cookie.Name + "=" + cookie.Value + ";"
|
||||
if cookie.Value == "" {
|
||||
continue
|
||||
}
|
||||
if !first {
|
||||
b.WriteString("; ")
|
||||
}
|
||||
first = false
|
||||
b.WriteString(cookie.Name)
|
||||
b.WriteByte('=')
|
||||
b.WriteString(cookie.Value)
|
||||
}
|
||||
return cookieHeader
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// GetCommonHeaders generates common HTTP headers with optional overwrite.
|
||||
// GetCommonHeaders generates common HTTP headers with optional overwrite
|
||||
func GetCommonHeaders(overwrite map[string]string) map[string]string {
|
||||
headers := map[string]string{
|
||||
"Content-Type": "application/json",
|
||||
@@ -524,32 +858,9 @@ func GetCommonHeaders(overwrite map[string]string) map[string]string {
|
||||
return headers
|
||||
}
|
||||
|
||||
// MergeCookies merges two slices of http.Cookies, ensuring no duplicates are added.
|
||||
func MergeCookies(left []*http.Cookie, right []*http.Cookie) ([]*http.Cookie, error) {
|
||||
var hashes []string
|
||||
for _, cookie := range right {
|
||||
hashes = append(hashes, cookie.Raw)
|
||||
}
|
||||
for _, cookie := range left {
|
||||
if !slices.Contains(hashes, cookie.Raw) {
|
||||
right = append(right, cookie)
|
||||
}
|
||||
}
|
||||
return right, nil
|
||||
}
|
||||
func int64Ptr(v int64) *int64 { return &v }
|
||||
|
||||
// GetCookiesForDomain filters the provided cookies based on the domain of the given URL.
|
||||
func GetCookiesForDomain(url *url.URL, cookies []*http.Cookie) ([]*http.Cookie, error) {
|
||||
var domainCookies []*http.Cookie
|
||||
for _, cookie := range cookies {
|
||||
if strings.HasSuffix(url.Host, cookie.Domain) {
|
||||
domainCookies = append(domainCookies, cookie)
|
||||
}
|
||||
}
|
||||
return domainCookies, nil
|
||||
}
|
||||
|
||||
// NewSession creates a new Session instance with default values.
|
||||
// NewSession creates a new Session instance with default values
|
||||
func NewSession() *Session {
|
||||
session := &Session{
|
||||
FrameID: strings.ToLower(uuid.New().String()),
|
||||
@@ -564,117 +875,20 @@ func NewSession() *Session {
|
||||
return session
|
||||
}
|
||||
|
||||
// AccountInfo represents an account info
|
||||
// AccountInfo represents the subset of Apple's account response we actually use
|
||||
// json.Unmarshal silently ignores extra fields in the response
|
||||
type AccountInfo struct {
|
||||
DsInfo *ValidateDataDsInfo `json:"dsInfo"`
|
||||
HasMinimumDeviceForPhotosWeb bool `json:"hasMinimumDeviceForPhotosWeb"`
|
||||
ICDPEnabled bool `json:"iCDPEnabled"`
|
||||
Webservices map[string]*webService `json:"webservices"`
|
||||
PcsEnabled bool `json:"pcsEnabled"`
|
||||
TermsUpdateNeeded bool `json:"termsUpdateNeeded"`
|
||||
ConfigBag struct {
|
||||
Urls struct {
|
||||
AccountCreateUI string `json:"accountCreateUI"`
|
||||
AccountLoginUI string `json:"accountLoginUI"`
|
||||
AccountLogin string `json:"accountLogin"`
|
||||
AccountRepairUI string `json:"accountRepairUI"`
|
||||
DownloadICloudTerms string `json:"downloadICloudTerms"`
|
||||
RepairDone string `json:"repairDone"`
|
||||
AccountAuthorizeUI string `json:"accountAuthorizeUI"`
|
||||
VettingURLForEmail string `json:"vettingUrlForEmail"`
|
||||
AccountCreate string `json:"accountCreate"`
|
||||
GetICloudTerms string `json:"getICloudTerms"`
|
||||
VettingURLForPhone string `json:"vettingUrlForPhone"`
|
||||
} `json:"urls"`
|
||||
AccountCreateEnabled bool `json:"accountCreateEnabled"`
|
||||
} `json:"configBag"`
|
||||
HsaTrustedBrowser bool `json:"hsaTrustedBrowser"`
|
||||
AppsOrder []string `json:"appsOrder"`
|
||||
Version int `json:"version"`
|
||||
IsExtendedLogin bool `json:"isExtendedLogin"`
|
||||
PcsServiceIdentitiesIncluded bool `json:"pcsServiceIdentitiesIncluded"`
|
||||
IsRepairNeeded bool `json:"isRepairNeeded"`
|
||||
HsaChallengeRequired bool `json:"hsaChallengeRequired"`
|
||||
RequestInfo struct {
|
||||
Country string `json:"country"`
|
||||
TimeZone string `json:"timeZone"`
|
||||
Region string `json:"region"`
|
||||
} `json:"requestInfo"`
|
||||
PcsDeleted bool `json:"pcsDeleted"`
|
||||
ICloudInfo struct {
|
||||
SafariBookmarksHasMigratedToCloudKit bool `json:"SafariBookmarksHasMigratedToCloudKit"`
|
||||
} `json:"iCloudInfo"`
|
||||
Apps map[string]*ValidateDataApp `json:"apps"`
|
||||
DsInfo *dsInfo `json:"dsInfo"`
|
||||
Webservices map[string]*webService `json:"webservices"`
|
||||
HsaChallengeRequired bool `json:"hsaChallengeRequired"`
|
||||
}
|
||||
|
||||
// ValidateDataDsInfo represents an validation info
|
||||
type ValidateDataDsInfo struct {
|
||||
HsaVersion int `json:"hsaVersion"`
|
||||
LastName string `json:"lastName"`
|
||||
ICDPEnabled bool `json:"iCDPEnabled"`
|
||||
TantorMigrated bool `json:"tantorMigrated"`
|
||||
Dsid string `json:"dsid"`
|
||||
HsaEnabled bool `json:"hsaEnabled"`
|
||||
IsHideMyEmailSubscriptionActive bool `json:"isHideMyEmailSubscriptionActive"`
|
||||
IroncadeMigrated bool `json:"ironcadeMigrated"`
|
||||
Locale string `json:"locale"`
|
||||
BrZoneConsolidated bool `json:"brZoneConsolidated"`
|
||||
ICDRSCapableDeviceList string `json:"ICDRSCapableDeviceList"`
|
||||
IsManagedAppleID bool `json:"isManagedAppleID"`
|
||||
IsCustomDomainsFeatureAvailable bool `json:"isCustomDomainsFeatureAvailable"`
|
||||
IsHideMyEmailFeatureAvailable bool `json:"isHideMyEmailFeatureAvailable"`
|
||||
ContinueOnDeviceEligibleDeviceInfo []string `json:"ContinueOnDeviceEligibleDeviceInfo"`
|
||||
Gilligvited bool `json:"gilligvited"`
|
||||
AppleIDAliases []any `json:"appleIdAliases"`
|
||||
UbiquityEOLEnabled bool `json:"ubiquityEOLEnabled"`
|
||||
IsPaidDeveloper bool `json:"isPaidDeveloper"`
|
||||
CountryCode string `json:"countryCode"`
|
||||
NotificationID string `json:"notificationId"`
|
||||
PrimaryEmailVerified bool `json:"primaryEmailVerified"`
|
||||
ADsID string `json:"aDsID"`
|
||||
Locked bool `json:"locked"`
|
||||
ICDRSCapableDeviceCount int `json:"ICDRSCapableDeviceCount"`
|
||||
HasICloudQualifyingDevice bool `json:"hasICloudQualifyingDevice"`
|
||||
PrimaryEmail string `json:"primaryEmail"`
|
||||
AppleIDEntries []struct {
|
||||
IsPrimary bool `json:"isPrimary"`
|
||||
Type string `json:"type"`
|
||||
Value string `json:"value"`
|
||||
} `json:"appleIdEntries"`
|
||||
GilliganEnabled bool `json:"gilligan-enabled"`
|
||||
IsWebAccessAllowed bool `json:"isWebAccessAllowed"`
|
||||
FullName string `json:"fullName"`
|
||||
MailFlags struct {
|
||||
IsThreadingAvailable bool `json:"isThreadingAvailable"`
|
||||
IsSearchV2Provisioned bool `json:"isSearchV2Provisioned"`
|
||||
SCKMail bool `json:"sCKMail"`
|
||||
IsMppSupportedInCurrentCountry bool `json:"isMppSupportedInCurrentCountry"`
|
||||
} `json:"mailFlags"`
|
||||
LanguageCode string `json:"languageCode"`
|
||||
AppleID string `json:"appleId"`
|
||||
HasUnreleasedOS bool `json:"hasUnreleasedOS"`
|
||||
AnalyticsOptInStatus bool `json:"analyticsOptInStatus"`
|
||||
FirstName string `json:"firstName"`
|
||||
ICloudAppleIDAlias string `json:"iCloudAppleIdAlias"`
|
||||
NotesMigrated bool `json:"notesMigrated"`
|
||||
BeneficiaryInfo struct {
|
||||
IsBeneficiary bool `json:"isBeneficiary"`
|
||||
} `json:"beneficiaryInfo"`
|
||||
HasPaymentInfo bool `json:"hasPaymentInfo"`
|
||||
PcsDelet bool `json:"pcsDelet"`
|
||||
AppleIDAlias string `json:"appleIdAlias"`
|
||||
BrMigrated bool `json:"brMigrated"`
|
||||
StatusCode int `json:"statusCode"`
|
||||
FamilyEligible bool `json:"familyEligible"`
|
||||
// dsInfo holds the account metadata fields we read
|
||||
type dsInfo struct {
|
||||
HsaVersion int `json:"hsaVersion"`
|
||||
}
|
||||
|
||||
// ValidateDataApp represents an app
|
||||
type ValidateDataApp struct {
|
||||
CanLaunchWithOneFactor bool `json:"canLaunchWithOneFactor"`
|
||||
IsQualifiedForBeta bool `json:"isQualifiedForBeta"`
|
||||
}
|
||||
|
||||
// WebService represents a web service
|
||||
// webService represents a web service
|
||||
type webService struct {
|
||||
PcsRequired bool `json:"pcsRequired"`
|
||||
URL string `json:"url"`
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func TestExtractHeadersMergesCookies(t *testing.T) {
|
||||
s := NewSession()
|
||||
s.Cookies = []*http.Cookie{{Name: "existing", Value: "old"}}
|
||||
|
||||
resp := &http.Response{Header: make(http.Header)}
|
||||
resp.Header.Add("Set-Cookie", (&http.Cookie{Name: "existing", Value: "new"}).String())
|
||||
resp.Header.Add("Set-Cookie", (&http.Cookie{Name: "fresh", Value: "value"}).String())
|
||||
resp.Header.Set("X-Apple-Session-Token", "session-token")
|
||||
|
||||
s.extractHeaders(resp)
|
||||
|
||||
require.Len(t, s.Cookies, 2)
|
||||
assert.Equal(t, "new", s.Cookies[0].Value)
|
||||
assert.Equal(t, "fresh", s.Cookies[1].Name)
|
||||
assert.Equal(t, "session-token", s.SessionToken)
|
||||
}
|
||||
|
||||
func TestExtractHeadersDeletesEmptyCookies(t *testing.T) {
|
||||
s := NewSession()
|
||||
s.Cookies = []*http.Cookie{{Name: "X-APPLE-WEBAUTH-HSA-LOGIN", Value: "stale"}, {Name: "keep", Value: "value"}}
|
||||
|
||||
resp := &http.Response{Header: make(http.Header)}
|
||||
resp.Header.Add("Set-Cookie", (&http.Cookie{Name: "X-APPLE-WEBAUTH-HSA-LOGIN", Value: ""}).String())
|
||||
|
||||
s.extractHeaders(resp)
|
||||
|
||||
require.Len(t, s.Cookies, 1)
|
||||
assert.Equal(t, "keep", s.Cookies[0].Name)
|
||||
assert.Equal(t, "keep=value", s.GetCookieString())
|
||||
}
|
||||
@@ -46,10 +46,10 @@ type srpClient struct {
|
||||
}
|
||||
|
||||
// newSRPClient generates a random 32-byte secret and computes the public value A.
|
||||
func newSRPClient() *srpClient {
|
||||
func newSRPClient() (*srpClient, error) {
|
||||
secret := make([]byte, 32)
|
||||
if _, err := rand.Read(secret); err != nil {
|
||||
panic(fmt.Sprintf("srp: rand.Read failed: %v", err))
|
||||
return nil, fmt.Errorf("srp: rand.Read failed: %w", err)
|
||||
}
|
||||
|
||||
a := new(big.Int).SetBytes(secret)
|
||||
@@ -60,7 +60,7 @@ func newSRPClient() *srpClient {
|
||||
a: a,
|
||||
A: A,
|
||||
k: k,
|
||||
}
|
||||
}, nil
|
||||
}
|
||||
|
||||
// getABytes returns the padded public value A.
|
||||
@@ -71,16 +71,21 @@ func (c *srpClient) getABytes() []byte {
|
||||
// processChallenge computes the session key and proof values from the server challenge.
|
||||
// username is the Apple ID, derivedKey is the output of derivePassword, salt and B come
|
||||
// from the server's init response (raw bytes, already base64-decoded).
|
||||
func (c *srpClient) processChallenge(username, derivedKey, salt, serverB []byte) {
|
||||
func (c *srpClient) processChallenge(username, derivedKey, salt, serverB []byte) error {
|
||||
B := new(big.Int).SetBytes(serverB)
|
||||
|
||||
// Validate B
|
||||
// Validate B (RFC 5054)
|
||||
if B.Cmp(big.NewInt(0)) <= 0 || B.Cmp(srpN) >= 0 {
|
||||
panic("srp: invalid server-supplied B, must be 1..N-1")
|
||||
return fmt.Errorf("srp: invalid server-supplied B, must be 1..N-1")
|
||||
}
|
||||
|
||||
x := calculateX(salt, derivedKey)
|
||||
u := calculateU(c.A, B)
|
||||
|
||||
// Validate u (RFC 5054)
|
||||
if u.Sign() == 0 {
|
||||
return fmt.Errorf("srp: calculated u is zero, aborting")
|
||||
}
|
||||
S := calculateS(c.k, x, c.a, B, u)
|
||||
c.K = calculateK(S)
|
||||
|
||||
@@ -88,6 +93,7 @@ func (c *srpClient) processChallenge(username, derivedKey, salt, serverB []byte)
|
||||
bBytes := padToN(B)
|
||||
c.M1 = calculateM1(username, salt, aBytes, bBytes, c.K)
|
||||
c.M2 = calculateM2(aBytes, c.M1, c.K)
|
||||
return nil
|
||||
}
|
||||
|
||||
// derivePassword performs Apple's password key derivation.
|
||||
@@ -98,10 +104,12 @@ func derivePassword(password string, salt []byte, iterations int, protocol strin
|
||||
|
||||
var passInput string
|
||||
switch protocol {
|
||||
case "s2k":
|
||||
passInput = string(passHash[:])
|
||||
case "s2k_fo":
|
||||
passInput = hex.EncodeToString(passHash[:])
|
||||
default: // "s2k"
|
||||
passInput = string(passHash[:])
|
||||
default:
|
||||
return nil, fmt.Errorf("unsupported SRP protocol: %q", protocol)
|
||||
}
|
||||
return pbkdf2.Key(sha256.New, passInput, salt, iterations, 32)
|
||||
}
|
||||
@@ -142,7 +150,7 @@ func getMultiplier() *big.Int {
|
||||
}
|
||||
|
||||
// calculateX computes x = H(salt | H(":" | password))
|
||||
// Apple's variant: NoUserNameInX — username is omitted from the inner hash.
|
||||
// Apple's variant: NoUserNameInX - username is omitted from the inner hash.
|
||||
// The "password" here is actually the derived key from derivePassword.
|
||||
func calculateX(salt, derivedKey []byte) *big.Int {
|
||||
h := srpHashFunc()
|
||||
|
||||
@@ -27,7 +27,7 @@ func TestPadToN(t *testing.T) {
|
||||
})
|
||||
|
||||
t.Run("preserves value that fills N", func(t *testing.T) {
|
||||
// Use N itself — already 256 bytes
|
||||
// Use N itself - already 256 bytes
|
||||
result := padToN(srpN)
|
||||
assert.Equal(t, srpNLenBytes, len(result))
|
||||
assert.Equal(t, srpN.Bytes(), result)
|
||||
@@ -52,7 +52,7 @@ func TestGetMultiplier(t *testing.T) {
|
||||
k2 := getMultiplier()
|
||||
assert.Equal(t, 0, k.Cmp(k2), "multiplier must be deterministic")
|
||||
|
||||
// k = H(N | pad(g)) — verify by manual computation
|
||||
// k = H(N | pad(g)) - verify by manual computation
|
||||
h := srpHashFunc()
|
||||
nBytes := srpN.Bytes()
|
||||
gBytes := srpG.Bytes()
|
||||
@@ -123,6 +123,12 @@ func TestDerivePassword(t *testing.T) {
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, 32, len(key))
|
||||
})
|
||||
|
||||
t.Run("unknown protocol returns error", func(t *testing.T) {
|
||||
_, err := derivePassword(password, salt, 1000, "unknown")
|
||||
assert.Error(t, err)
|
||||
assert.Contains(t, err.Error(), "unsupported SRP protocol")
|
||||
})
|
||||
}
|
||||
|
||||
func TestCalculateX(t *testing.T) {
|
||||
@@ -256,7 +262,8 @@ func TestCalculateM2(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestNewSRPClient(t *testing.T) {
|
||||
client := newSRPClient()
|
||||
client, err := newSRPClient()
|
||||
require.NoError(t, err)
|
||||
|
||||
// A must be non-zero and in range [1, N-1]
|
||||
assert.True(t, client.A.Sign() > 0, "A must be positive")
|
||||
@@ -273,7 +280,8 @@ func TestNewSRPClient(t *testing.T) {
|
||||
assert.Equal(t, 0, client.A.Cmp(recoveredA))
|
||||
|
||||
// Two clients should have different secrets (probabilistic, but 2^256 collision chance)
|
||||
client2 := newSRPClient()
|
||||
client2, err := newSRPClient()
|
||||
require.NoError(t, err)
|
||||
assert.NotEqual(t, 0, client.a.Cmp(client2.a), "two clients must have different secrets")
|
||||
}
|
||||
|
||||
@@ -315,7 +323,8 @@ func TestProcessChallenge(t *testing.T) {
|
||||
|
||||
serverB := padToN(B)
|
||||
|
||||
client.processChallenge(username, derivedKey, salt, serverB)
|
||||
err := client.processChallenge(username, derivedKey, salt, serverB)
|
||||
require.NoError(t, err)
|
||||
|
||||
// M1, M2, K must be populated
|
||||
assert.NotNil(t, client.M1)
|
||||
@@ -328,13 +337,14 @@ func TestProcessChallenge(t *testing.T) {
|
||||
// M1 and M2 must be different
|
||||
assert.NotEqual(t, client.M1, client.M2)
|
||||
|
||||
// Results must be deterministic — run again with same inputs
|
||||
// Results must be deterministic - run again with same inputs
|
||||
client2 := &srpClient{
|
||||
a: new(big.Int).Set(a),
|
||||
A: new(big.Int).Set(A),
|
||||
k: new(big.Int).Set(k),
|
||||
}
|
||||
client2.processChallenge(username, derivedKey, salt, serverB)
|
||||
err = client2.processChallenge(username, derivedKey, salt, serverB)
|
||||
require.NoError(t, err)
|
||||
|
||||
assert.Equal(t, client.M1, client2.M1, "M1 must be deterministic")
|
||||
assert.Equal(t, client.M2, client2.M2, "M2 must be deterministic")
|
||||
@@ -352,6 +362,28 @@ func TestProcessChallenge(t *testing.T) {
|
||||
assert.Equal(t, client.K, kServer, "client and server must derive the same session key")
|
||||
}
|
||||
|
||||
func TestProcessChallenge_InvalidB(t *testing.T) {
|
||||
k := getMultiplier()
|
||||
client := &srpClient{
|
||||
a: big.NewInt(42),
|
||||
A: new(big.Int).Exp(srpG, big.NewInt(42), srpN),
|
||||
k: k,
|
||||
}
|
||||
username := []byte("test@apple.com")
|
||||
derivedKey := make([]byte, 32)
|
||||
salt := []byte("salt")
|
||||
|
||||
t.Run("B=0 rejected", func(t *testing.T) {
|
||||
err := client.processChallenge(username, derivedKey, salt, padToN(big.NewInt(0)))
|
||||
assert.Error(t, err)
|
||||
})
|
||||
|
||||
t.Run("B=N rejected", func(t *testing.T) {
|
||||
err := client.processChallenge(username, derivedKey, salt, padToN(srpN))
|
||||
assert.Error(t, err)
|
||||
})
|
||||
}
|
||||
|
||||
func TestSRPGroupParameters(t *testing.T) {
|
||||
// Verify the 2048-bit prime starts with known hex prefix
|
||||
nHex := hex.EncodeToString(srpN.Bytes())
|
||||
|
||||
@@ -0,0 +1,495 @@
|
||||
//go:build !plan9 && !solaris
|
||||
|
||||
// Package iclouddrive provides access to iCloud Drive and Photos
|
||||
package iclouddrive
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/rclone/rclone/backend/iclouddrive/api"
|
||||
"github.com/rclone/rclone/fs"
|
||||
"github.com/rclone/rclone/fs/config"
|
||||
"github.com/rclone/rclone/fs/config/configmap"
|
||||
"github.com/rclone/rclone/fs/config/configstruct"
|
||||
"github.com/rclone/rclone/fs/config/obscure"
|
||||
"github.com/rclone/rclone/lib/encoder"
|
||||
)
|
||||
|
||||
const configAuthSession = "_auth_session"
|
||||
|
||||
// configAuthState holds session fields that must be preserved across
|
||||
// Config() state machine steps (push 2FA and SMS both need it)
|
||||
type configAuthState struct {
|
||||
Scnt string `json:"s"`
|
||||
SessionID string `json:"i"`
|
||||
AuthAttributes string `json:"a"`
|
||||
FrameID string `json:"f"`
|
||||
SessionToken string `json:"t"`
|
||||
ClientID string `json:"c"`
|
||||
AccountCountry string `json:"ac"`
|
||||
Phones []smsPhone `json:"p,omitempty"`
|
||||
}
|
||||
|
||||
type smsPhone struct {
|
||||
ID int `json:"id"`
|
||||
Num string `json:"n"`
|
||||
Mode string `json:"m"`
|
||||
}
|
||||
|
||||
func saveAuthSession(m configmap.Mapper, s *api.Session, phones []smsPhone) {
|
||||
st := configAuthState{
|
||||
Scnt: s.Scnt,
|
||||
SessionID: s.SessionID,
|
||||
AuthAttributes: s.AuthAttributes,
|
||||
FrameID: s.FrameID,
|
||||
SessionToken: s.SessionToken,
|
||||
ClientID: s.ClientID,
|
||||
AccountCountry: s.AccountCountry,
|
||||
Phones: phones,
|
||||
}
|
||||
data, err := json.Marshal(st)
|
||||
if err != nil {
|
||||
fs.Debugf(nil, "iclouddrive: failed to marshal auth session: %v", err)
|
||||
return
|
||||
}
|
||||
m.Set(configAuthSession, base64.StdEncoding.EncodeToString(data))
|
||||
}
|
||||
|
||||
func loadAuthSession(m configmap.Mapper) (*configAuthState, error) {
|
||||
raw, _ := m.Get(configAuthSession)
|
||||
if raw == "" {
|
||||
return nil, errors.New("auth session state lost, please reconfigure")
|
||||
}
|
||||
data, err := base64.StdEncoding.DecodeString(raw)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("corrupt auth session state: %w", err)
|
||||
}
|
||||
var st configAuthState
|
||||
if err := json.Unmarshal(data, &st); err != nil {
|
||||
return nil, fmt.Errorf("invalid auth session state: %w", err)
|
||||
}
|
||||
return &st, nil
|
||||
}
|
||||
|
||||
func restoreAuthSession(icloud *api.Client, st *configAuthState) {
|
||||
icloud.Session.Scnt = st.Scnt
|
||||
icloud.Session.SessionID = st.SessionID
|
||||
icloud.Session.AuthAttributes = st.AuthAttributes
|
||||
icloud.Session.FrameID = st.FrameID
|
||||
icloud.Session.SessionToken = st.SessionToken
|
||||
icloud.Session.ClientID = st.ClientID
|
||||
icloud.Session.AccountCountry = st.AccountCountry
|
||||
}
|
||||
|
||||
// resumeConfigClient recreates an API client and restores session state saved
|
||||
// from a previous Config() step. Used across 2FA states to avoid re-authenticating
|
||||
func resumeConfigClient(m configmap.Mapper, appleid, password, trustToken, clientID string, cookies []*http.Cookie) (*api.Client, *configAuthState, error) {
|
||||
st, err := loadAuthSession(m)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
icloud, err := api.New(appleid, password, trustToken, clientID, cookies, nil, "_config")
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
restoreAuthSession(icloud, st)
|
||||
return icloud, st, nil
|
||||
}
|
||||
|
||||
// saveAuthCredentials persists trust token, cookies, and clears session state
|
||||
// after successful 2FA validation
|
||||
func saveAuthCredentials(m configmap.Mapper, icloud *api.Client, name string) {
|
||||
m.Set(configTrustToken, icloud.Session.TrustToken)
|
||||
m.Set(configCookies, icloud.Session.GetCookieString())
|
||||
m.Set(configAuthSession, "")
|
||||
api.ClearCacheDir(name)
|
||||
}
|
||||
|
||||
// triggerSMSFlow handles phone selection and SMS triggering
|
||||
// Single phone: triggers SMS immediately, returns code prompt
|
||||
// Multiple phones: saves session + phone list, returns phone picker
|
||||
func triggerSMSFlow(ctx context.Context, icloud *api.Client, phones []api.TrustedPhoneNumber, m configmap.Mapper) (*fs.ConfigOut, error) {
|
||||
smsPhones := make([]smsPhone, len(phones))
|
||||
for i, p := range phones {
|
||||
mode := p.PushMode
|
||||
if mode == "" {
|
||||
mode = "sms"
|
||||
}
|
||||
smsPhones[i] = smsPhone{ID: p.ID, Num: p.NumberWithDialCode, Mode: mode}
|
||||
}
|
||||
|
||||
if len(phones) == 1 {
|
||||
p := smsPhones[0]
|
||||
if err := icloud.Session.RequestSMSCode(ctx, p.ID, p.Mode); err != nil {
|
||||
return nil, fmt.Errorf("failed to send SMS code: %w", err)
|
||||
}
|
||||
saveAuthSession(m, icloud.Session, nil)
|
||||
nextState := fmt.Sprintf("2fa_sms_%d_%s", p.ID, p.Mode)
|
||||
return fs.ConfigInput(nextState, "config_2fa_sms", fmt.Sprintf("Enter the verification code sent to %s", p.Num))
|
||||
}
|
||||
|
||||
// Multiple phones - save session and present picker
|
||||
saveAuthSession(m, icloud.Session, smsPhones)
|
||||
items := make([]fs.OptionExample, len(smsPhones))
|
||||
for i, p := range smsPhones {
|
||||
items[i] = fs.OptionExample{
|
||||
Value: fmt.Sprintf("%d_%s", p.ID, p.Mode),
|
||||
Help: p.Num,
|
||||
}
|
||||
}
|
||||
return fs.ConfigChooseExclusiveFixed("2fa_sms_select", "config_2fa_phone", "Select phone number for SMS verification", items)
|
||||
}
|
||||
|
||||
const (
|
||||
configService = "service"
|
||||
|
||||
// Service types
|
||||
serviceDrive = "drive"
|
||||
servicePhotos = "photos"
|
||||
)
|
||||
|
||||
// ServiceOptions defines the configuration for service selection
|
||||
type ServiceOptions struct {
|
||||
Service string `config:"service"`
|
||||
}
|
||||
|
||||
// Register with rclone
|
||||
func init() {
|
||||
fs.Register(&fs.RegInfo{
|
||||
Name: "iclouddrive",
|
||||
Description: "iCloud Drive and Photos",
|
||||
Config: Config,
|
||||
NewFs: NewServiceFs,
|
||||
MetadataInfo: &fs.MetadataInfo{
|
||||
System: map[string]fs.MetadataHelp{
|
||||
"width": {
|
||||
Help: "Image width in pixels",
|
||||
Type: "int",
|
||||
ReadOnly: true,
|
||||
},
|
||||
"height": {
|
||||
Help: "Image height in pixels",
|
||||
Type: "int",
|
||||
ReadOnly: true,
|
||||
},
|
||||
"added-time": {
|
||||
Help: "Time the item was added to the iCloud library",
|
||||
Type: "RFC 3339",
|
||||
Example: "2006-01-02T15:04:05Z",
|
||||
ReadOnly: true,
|
||||
},
|
||||
"favorite": {
|
||||
Help: "Whether the item is marked as favorite",
|
||||
Type: "bool",
|
||||
ReadOnly: true,
|
||||
},
|
||||
"hidden": {
|
||||
Help: "Whether the item is hidden",
|
||||
Type: "bool",
|
||||
ReadOnly: true,
|
||||
},
|
||||
},
|
||||
Help: "Metadata is read-only and available for the Photos service only.",
|
||||
},
|
||||
Options: []fs.Option{{
|
||||
Name: configService,
|
||||
Help: "iCloud service to use.",
|
||||
Required: true,
|
||||
Default: serviceDrive,
|
||||
Examples: []fs.OptionExample{{
|
||||
Value: serviceDrive,
|
||||
Help: "iCloud Drive",
|
||||
}, {
|
||||
Value: servicePhotos,
|
||||
Help: "iCloud Photos",
|
||||
}},
|
||||
}, {
|
||||
Name: configAppleID,
|
||||
Help: "Apple ID.",
|
||||
Required: true,
|
||||
Sensitive: true,
|
||||
}, {
|
||||
Name: configPassword,
|
||||
Help: "Password.",
|
||||
Required: true,
|
||||
IsPassword: true,
|
||||
Sensitive: true,
|
||||
}, {
|
||||
Name: configTrustToken,
|
||||
Help: "Trust token for session authentication.",
|
||||
IsPassword: false,
|
||||
Required: false,
|
||||
Sensitive: true,
|
||||
Hide: fs.OptionHideBoth,
|
||||
}, {
|
||||
Name: configCookies,
|
||||
Help: "Session cookies.",
|
||||
Required: false,
|
||||
Advanced: false,
|
||||
Sensitive: true,
|
||||
Hide: fs.OptionHideBoth,
|
||||
}, {
|
||||
Name: configClientID,
|
||||
Help: "Client ID for iCloud API access.",
|
||||
Required: false,
|
||||
Advanced: true,
|
||||
Default: "d39ba9916b7251055b22c7f910e2ea796ee65e98b2ddecea8f5dde8d9d1a815d",
|
||||
}, {
|
||||
Name: config.ConfigEncoding,
|
||||
Help: config.ConfigEncodingHelp,
|
||||
Advanced: true,
|
||||
Default: (encoder.Display |
|
||||
encoder.EncodeBackSlash |
|
||||
encoder.EncodeSlash |
|
||||
encoder.EncodeInvalidUtf8),
|
||||
}},
|
||||
})
|
||||
}
|
||||
|
||||
// Config handles the authentication and configuration flow
|
||||
func Config(ctx context.Context, name string, m configmap.Mapper, config fs.ConfigIn) (*fs.ConfigOut, error) {
|
||||
var err error
|
||||
appleid, _ := m.Get(configAppleID)
|
||||
if appleid == "" {
|
||||
return nil, errors.New("an Apple ID is required")
|
||||
}
|
||||
|
||||
password, _ := m.Get(configPassword)
|
||||
if password != "" {
|
||||
password, err = obscure.Reveal(password)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
trustToken, _ := m.Get(configTrustToken)
|
||||
cookieRaw, _ := m.Get(configCookies)
|
||||
clientID, _ := m.Get(configClientID)
|
||||
cookies := ReadCookies(cookieRaw)
|
||||
|
||||
switch {
|
||||
case config.State == "":
|
||||
// Force fresh SRP authentication - ignore stale trust token and cookies
|
||||
// so that reconnect always prompts for 2FA
|
||||
m.Set(configAuthSession, "")
|
||||
icloud, err := api.New(appleid, password, "", clientID, nil, nil, "_config")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := icloud.Authenticate(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
m.Set(configCookies, icloud.Session.GetCookieString())
|
||||
if icloud.Session.Requires2FA() {
|
||||
// Check if user has no trusted devices - auto-trigger SMS if so
|
||||
authState, err := icloud.Session.GetAuthState(ctx)
|
||||
if err == nil && authState.NoTrustedDevices && len(authState.TrustedPhoneNumbers) > 0 {
|
||||
return triggerSMSFlow(ctx, icloud, authState.TrustedPhoneNumbers, m)
|
||||
}
|
||||
// Explicitly request push to trusted devices - required for iOS 26.4+
|
||||
// where the SRP 409 no longer auto-pushes. GET /appleauth/auth above
|
||||
// may also trigger a push (cosmetic double on pre-26.4, harmless)
|
||||
if err := icloud.Session.RequestPushNotification(ctx); err != nil {
|
||||
fs.Debugf(nil, "iclouddrive: push notification request failed (SMS fallback available): %v", err)
|
||||
} else {
|
||||
fs.Debugf(nil, "iclouddrive: push notification requested to trusted devices")
|
||||
}
|
||||
// Save session state so 2fa_do can validate without re-authenticating
|
||||
// Push codes are account-scoped so session reuse is not strictly required,
|
||||
// but it avoids a redundant SRP roundtrip and a second push on pre-26.4
|
||||
saveAuthSession(m, icloud.Session, nil)
|
||||
return fs.ConfigInput("2fa_do", "config_2fa", "Two-factor authentication: enter your 2FA code or type 'sms' for a text message")
|
||||
}
|
||||
// Auth succeeded without 2FA - save updated credentials and clear stale cache
|
||||
m.Set(configTrustToken, icloud.Session.TrustToken)
|
||||
api.ClearCacheDir(name)
|
||||
return nil, nil
|
||||
|
||||
case config.State == "2fa_do":
|
||||
code := config.Result
|
||||
if code == "" {
|
||||
return fs.ConfigError("authenticate", "2FA codes can't be blank")
|
||||
}
|
||||
|
||||
// Restore session from initial sign-in instead of re-authenticating
|
||||
// This avoids a redundant SRP roundtrip and extra push on pre-26.4
|
||||
icloud, _, err := resumeConfigClient(m, appleid, password, trustToken, clientID, cookies)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if strings.EqualFold(code, "sms") {
|
||||
authState, err := icloud.Session.GetAuthState(ctx)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to get trusted phone numbers: %w", err)
|
||||
}
|
||||
if len(authState.TrustedPhoneNumbers) == 0 {
|
||||
return nil, errors.New("no trusted phone numbers on this account")
|
||||
}
|
||||
return triggerSMSFlow(ctx, icloud, authState.TrustedPhoneNumbers, m)
|
||||
}
|
||||
|
||||
if err := icloud.Session.Validate2FACode(ctx, code); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
saveAuthCredentials(m, icloud, name)
|
||||
return nil, nil
|
||||
|
||||
case config.State == "2fa_sms_select":
|
||||
// User selected a phone from ConfigChooseExclusiveFixed (value: "ID_mode")
|
||||
idStr, mode, _ := strings.Cut(config.Result, "_")
|
||||
phoneID, err := strconv.Atoi(idStr)
|
||||
if err != nil {
|
||||
m.Set(configAuthSession, "")
|
||||
return nil, fmt.Errorf("invalid phone selection %q", config.Result)
|
||||
}
|
||||
if mode == "" {
|
||||
mode = "sms"
|
||||
}
|
||||
icloud, smsState, err := resumeConfigClient(m, appleid, password, trustToken, clientID, cookies)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// Find the selected phone's display number
|
||||
var phoneNum string
|
||||
for _, p := range smsState.Phones {
|
||||
if p.ID == phoneID {
|
||||
phoneNum = p.Num
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
if err := icloud.Session.RequestSMSCode(ctx, phoneID, mode); err != nil {
|
||||
m.Set(configAuthSession, "")
|
||||
return nil, fmt.Errorf("failed to send SMS code: %w", err)
|
||||
}
|
||||
saveAuthSession(m, icloud.Session, nil)
|
||||
nextState := fmt.Sprintf("2fa_sms_%d_%s", phoneID, mode)
|
||||
return fs.ConfigInput(nextState, "config_2fa_sms", fmt.Sprintf("Enter the verification code sent to %s", phoneNum))
|
||||
|
||||
case strings.HasPrefix(config.State, "2fa_sms_"):
|
||||
code := config.Result
|
||||
if code == "" {
|
||||
return fs.ConfigError("authenticate", "SMS code can't be blank")
|
||||
}
|
||||
// State encodes phone ID and mode: "2fa_sms_<ID>_<mode>"
|
||||
suffix := strings.TrimPrefix(config.State, "2fa_sms_")
|
||||
idStr, mode, _ := strings.Cut(suffix, "_")
|
||||
phoneID, err := strconv.Atoi(idStr)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("invalid phone ID in state %q: %w", config.State, err)
|
||||
}
|
||||
if mode == "" {
|
||||
mode = "sms"
|
||||
}
|
||||
|
||||
icloud, _, err := resumeConfigClient(m, appleid, password, trustToken, clientID, cookies)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if err := icloud.Session.ValidateSMSCode(ctx, code, phoneID, mode); err != nil {
|
||||
m.Set(configAuthSession, "")
|
||||
return nil, err
|
||||
}
|
||||
saveAuthCredentials(m, icloud, name)
|
||||
return nil, nil
|
||||
|
||||
default:
|
||||
return nil, fmt.Errorf("unknown state %q", config.State)
|
||||
}
|
||||
}
|
||||
|
||||
// newICloudClient parses options, authenticates, and returns a ready client
|
||||
// Shared between NewFs (Drive) and NewFsPhotos (Photos) to avoid duplication
|
||||
func newICloudClient(ctx context.Context, name string, m configmap.Mapper) (*api.Client, *Options, error) {
|
||||
opt := new(Options)
|
||||
err := configstruct.Set(m, opt)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
|
||||
if opt.Password != "" {
|
||||
opt.Password, err = obscure.Reveal(opt.Password)
|
||||
if err != nil {
|
||||
return nil, nil, fmt.Errorf("couldn't decrypt user password: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
if opt.TrustToken == "" {
|
||||
return nil, nil, fmt.Errorf("missing icloud trust token: try refreshing it with \"rclone config reconnect %s:\"", name)
|
||||
}
|
||||
|
||||
cookies := ReadCookies(opt.Cookies)
|
||||
|
||||
callback := func(session *api.Session) {
|
||||
m.Set(configCookies, session.GetCookieString())
|
||||
}
|
||||
|
||||
icloud, err := api.New(
|
||||
opt.AppleID,
|
||||
opt.Password,
|
||||
opt.TrustToken,
|
||||
opt.ClientID,
|
||||
cookies,
|
||||
callback,
|
||||
name,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
|
||||
if err := icloud.Authenticate(ctx); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
|
||||
if icloud.Session.Requires2FA() {
|
||||
return nil, nil, errors.New("trust token expired, please reauth")
|
||||
}
|
||||
|
||||
return icloud, opt, nil
|
||||
}
|
||||
|
||||
// disconnectClient clears authentication state and removes disk caches
|
||||
// Shared between Drive Fs and Photos Fs Disconnect() implementations
|
||||
func disconnectClient(m configmap.Mapper, icloud *api.Client) error {
|
||||
m.Set(configTrustToken, "")
|
||||
m.Set(configCookies, "")
|
||||
m.Set(configAuthSession, "")
|
||||
return os.RemoveAll(icloud.CacheDir())
|
||||
}
|
||||
|
||||
// NewServiceFs creates a filesystem instance for the selected service
|
||||
func NewServiceFs(ctx context.Context, name, root string, m configmap.Mapper) (fs.Fs, error) {
|
||||
// Parse the service selection
|
||||
opt := new(ServiceOptions)
|
||||
err := configstruct.Set(m, opt)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Set default service if not specified
|
||||
if opt.Service == "" {
|
||||
opt.Service = serviceDrive
|
||||
}
|
||||
|
||||
// Route to the appropriate backend
|
||||
switch opt.Service {
|
||||
case serviceDrive:
|
||||
// Create Drive filesystem
|
||||
return NewFs(ctx, name, root, m)
|
||||
case servicePhotos:
|
||||
// Create Photos filesystem
|
||||
return NewFsPhotos(ctx, name, root, m)
|
||||
default:
|
||||
return nil, fmt.Errorf("invalid service selection: %s (must be 'drive' or 'photos')", opt.Service)
|
||||
}
|
||||
}
|
||||
@@ -9,17 +9,13 @@ import (
|
||||
"path"
|
||||
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/rclone/rclone/fs"
|
||||
"github.com/rclone/rclone/fs/config"
|
||||
"github.com/rclone/rclone/fs/config/configmap"
|
||||
"github.com/rclone/rclone/fs/config/configstruct"
|
||||
"github.com/rclone/rclone/fs/config/obscure"
|
||||
"github.com/rclone/rclone/fs/fserrors"
|
||||
|
||||
"github.com/rclone/rclone/backend/iclouddrive/api"
|
||||
@@ -51,61 +47,10 @@ const (
|
||||
decayConstant = 2
|
||||
)
|
||||
|
||||
// Register with Fs
|
||||
func init() {
|
||||
fs.Register(&fs.RegInfo{
|
||||
Name: "iclouddrive",
|
||||
Description: "iCloud Drive",
|
||||
Config: Config,
|
||||
NewFs: NewFs,
|
||||
Options: []fs.Option{{
|
||||
Name: configAppleID,
|
||||
Help: "Apple ID.",
|
||||
Required: true,
|
||||
Sensitive: true,
|
||||
}, {
|
||||
Name: configPassword,
|
||||
Help: "Password.",
|
||||
Required: true,
|
||||
IsPassword: true,
|
||||
Sensitive: true,
|
||||
}, {
|
||||
Name: configTrustToken,
|
||||
Help: "Trust token (internal use)",
|
||||
IsPassword: false,
|
||||
Required: false,
|
||||
Sensitive: true,
|
||||
Hide: fs.OptionHideBoth,
|
||||
}, {
|
||||
Name: configCookies,
|
||||
Help: "cookies (internal use only)",
|
||||
Required: false,
|
||||
Advanced: false,
|
||||
Sensitive: true,
|
||||
Hide: fs.OptionHideBoth,
|
||||
}, {
|
||||
Name: configClientID,
|
||||
Help: "Client id",
|
||||
Required: false,
|
||||
Advanced: true,
|
||||
Default: "d39ba9916b7251055b22c7f910e2ea796ee65e98b2ddecea8f5dde8d9d1a815d",
|
||||
}, {
|
||||
Name: config.ConfigEncoding,
|
||||
Help: config.ConfigEncodingHelp,
|
||||
Advanced: true,
|
||||
Default: (encoder.Display |
|
||||
//encoder.EncodeDot |
|
||||
encoder.EncodeBackSlash |
|
||||
encoder.EncodeInvalidUtf8),
|
||||
}},
|
||||
})
|
||||
}
|
||||
|
||||
// Options defines the configuration for this backend
|
||||
type Options struct {
|
||||
AppleID string `config:"apple_id"`
|
||||
Password string `config:"password"`
|
||||
Photos bool `config:"photos"`
|
||||
TrustToken string `config:"trust_token"`
|
||||
Cookies string `config:"cookies"`
|
||||
ClientID string `config:"client_id"`
|
||||
@@ -118,6 +63,7 @@ type Fs struct {
|
||||
root string // the path we are working on.
|
||||
rootID string
|
||||
opt Options // parsed config options
|
||||
m configmap.Mapper // config map for persisting auth state
|
||||
features *fs.Features // optional features
|
||||
dirCache *dircache.DirCache // Map of directory path to directory id
|
||||
icloud *api.Client
|
||||
@@ -139,72 +85,6 @@ type Object struct {
|
||||
downloadURL string
|
||||
}
|
||||
|
||||
// Config configures the iCloud remote.
|
||||
func Config(ctx context.Context, name string, m configmap.Mapper, config fs.ConfigIn) (*fs.ConfigOut, error) {
|
||||
var err error
|
||||
appleid, _ := m.Get(configAppleID)
|
||||
if appleid == "" {
|
||||
return nil, errors.New("a apple ID is required")
|
||||
}
|
||||
|
||||
password, _ := m.Get(configPassword)
|
||||
if password != "" {
|
||||
password, err = obscure.Reveal(password)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
trustToken, _ := m.Get(configTrustToken)
|
||||
cookieRaw, _ := m.Get(configCookies)
|
||||
clientID, _ := m.Get(configClientID)
|
||||
cookies := ReadCookies(cookieRaw)
|
||||
|
||||
switch config.State {
|
||||
case "":
|
||||
icloud, err := api.New(appleid, password, trustToken, clientID, cookies, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := icloud.Authenticate(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
m.Set(configCookies, icloud.Session.GetCookieString())
|
||||
if icloud.Session.Requires2FA() {
|
||||
return fs.ConfigInput("2fa_do", "config_2fa", "Two-factor authentication: please enter your 2FA code")
|
||||
}
|
||||
return nil, nil
|
||||
case "2fa_do":
|
||||
code := config.Result
|
||||
if code == "" {
|
||||
return fs.ConfigError("authenticate", "2FA codes can't be blank")
|
||||
}
|
||||
|
||||
icloud, err := api.New(appleid, password, trustToken, clientID, cookies, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := icloud.SignIn(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if err := icloud.Session.Validate2FACode(ctx, code); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
m.Set(configTrustToken, icloud.Session.TrustToken)
|
||||
m.Set(configCookies, icloud.Session.GetCookieString())
|
||||
return nil, nil
|
||||
|
||||
case "2fa_error":
|
||||
if config.Result == "true" {
|
||||
return fs.ConfigGoto("2fa")
|
||||
}
|
||||
return nil, errors.New("2fa authentication failed")
|
||||
}
|
||||
return nil, fmt.Errorf("unknown state %q", config.State)
|
||||
}
|
||||
|
||||
// find item by path. Will not return any children for the item
|
||||
func (f *Fs) findItem(ctx context.Context, dir string) (item *api.DriveItem, found bool, err error) {
|
||||
var resp *http.Response
|
||||
@@ -797,51 +677,11 @@ func retryResultUnknown(ctx context.Context, resp *http.Response, err error) (bo
|
||||
|
||||
// NewFs constructs an Fs from the path, container:path
|
||||
func NewFs(ctx context.Context, name, root string, m configmap.Mapper) (fs.Fs, error) {
|
||||
// Parse config into Options struct
|
||||
opt := new(Options)
|
||||
err := configstruct.Set(m, opt)
|
||||
icloud, opt, err := newICloudClient(ctx, name, m)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if opt.Password != "" {
|
||||
var err error
|
||||
opt.Password, err = obscure.Reveal(opt.Password)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("couldn't decrypt user password: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
if opt.TrustToken == "" {
|
||||
return nil, fmt.Errorf("missing icloud trust token: try refreshing it with \"rclone config reconnect %s:\"", name)
|
||||
}
|
||||
|
||||
cookies := ReadCookies(opt.Cookies)
|
||||
|
||||
callback := func(session *api.Session) {
|
||||
m.Set(configCookies, session.GetCookieString())
|
||||
}
|
||||
|
||||
icloud, err := api.New(
|
||||
opt.AppleID,
|
||||
opt.Password,
|
||||
opt.TrustToken,
|
||||
opt.ClientID,
|
||||
cookies,
|
||||
callback,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if err := icloud.Authenticate(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if icloud.Session.Requires2FA() {
|
||||
return nil, errors.New("trust token expired, please reauth")
|
||||
}
|
||||
|
||||
root = strings.Trim(root, "/")
|
||||
|
||||
f := &Fs{
|
||||
@@ -850,6 +690,7 @@ func NewFs(ctx context.Context, name, root string, m configmap.Mapper) (fs.Fs, e
|
||||
icloud: icloud,
|
||||
rootID: "FOLDER::com.apple.CloudDocs::root",
|
||||
opt: *opt,
|
||||
m: m,
|
||||
pacer: fs.NewPacer(ctx, pacer.NewDefault(pacer.MinSleep(minSleep), pacer.MaxSleep(maxSleep), pacer.DecayConstant(decayConstant))),
|
||||
}
|
||||
f.features = (&fs.Features{
|
||||
@@ -1164,9 +1005,15 @@ func (o *Object) Update(ctx context.Context, in io.Reader, src fs.ObjectInfo, op
|
||||
return nil
|
||||
}
|
||||
|
||||
// Disconnect clears authentication state and removes disk caches
|
||||
func (f *Fs) Disconnect(ctx context.Context) error {
|
||||
return disconnectClient(f.m, f.icloud)
|
||||
}
|
||||
|
||||
// Check interfaces are satisfied
|
||||
var (
|
||||
_ fs.Fs = &Fs{}
|
||||
_ fs.Disconnecter = (*Fs)(nil)
|
||||
_ fs.Mover = (*Fs)(nil)
|
||||
_ fs.Purger = (*Fs)(nil)
|
||||
_ fs.DirMover = (*Fs)(nil)
|
||||
|
||||
@@ -0,0 +1,892 @@
|
||||
//go:build !plan9 && !solaris
|
||||
|
||||
package iclouddrive
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"path"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/rclone/rclone/backend/iclouddrive/api"
|
||||
"github.com/rclone/rclone/fs"
|
||||
"github.com/rclone/rclone/fs/config/configmap"
|
||||
"github.com/rclone/rclone/fs/fshttp"
|
||||
"github.com/rclone/rclone/fs/hash"
|
||||
"github.com/rclone/rclone/fs/list"
|
||||
"github.com/rclone/rclone/lib/dircache"
|
||||
"github.com/rclone/rclone/lib/pacer"
|
||||
)
|
||||
|
||||
const rootID = "photos-root"
|
||||
|
||||
// PhotosFs represents a remote iCloud Photos server
|
||||
type PhotosFs struct {
|
||||
name string
|
||||
root string
|
||||
opt Options
|
||||
features *fs.Features
|
||||
icloud *api.Client
|
||||
m configmap.Mapper
|
||||
pacer *fs.Pacer
|
||||
dirCache *dircache.DirCache
|
||||
httpClient *http.Client
|
||||
startTime time.Time
|
||||
mu sync.Mutex
|
||||
photos *api.PhotosService
|
||||
}
|
||||
|
||||
// PhotosObject describes an iCloud Photos object
|
||||
type PhotosObject struct {
|
||||
fs *PhotosFs
|
||||
remote string
|
||||
size int64
|
||||
modTime time.Time
|
||||
masterID string // CloudKit recordName for fresh URL lookup (master or asset depending on resource)
|
||||
zone string // CloudKit zone (e.g. "PrimarySync")
|
||||
resourceKey string // CloudKit resource field (resOriginalRes or resOriginalVidComplRes)
|
||||
width int
|
||||
height int
|
||||
addedDate int64
|
||||
isFavorite bool
|
||||
isHidden bool
|
||||
}
|
||||
|
||||
// NewFsPhotos constructs an Fs for Photos from the path, container:path
|
||||
func NewFsPhotos(ctx context.Context, name, root string, m configmap.Mapper) (fs.Fs, error) {
|
||||
icloud, opt, err := newICloudClient(ctx, name, m)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
root = strings.Trim(root, "/")
|
||||
|
||||
f := &PhotosFs{
|
||||
name: name,
|
||||
root: root,
|
||||
opt: *opt,
|
||||
icloud: icloud,
|
||||
m: m,
|
||||
pacer: fs.NewPacer(ctx, pacer.NewDefault(pacer.MinSleep(minSleep), pacer.MaxSleep(maxSleep), pacer.DecayConstant(decayConstant))),
|
||||
httpClient: fshttp.NewClient(ctx),
|
||||
startTime: time.Now(),
|
||||
}
|
||||
|
||||
f.features = (&fs.Features{
|
||||
CanHaveEmptyDirectories: true,
|
||||
PartialUploads: false,
|
||||
ReadMimeType: false,
|
||||
ReadMetadata: true,
|
||||
}).Fill(ctx, f)
|
||||
|
||||
f.dirCache = dircache.New(root, rootID, f)
|
||||
|
||||
err = f.dirCache.FindRoot(ctx, false)
|
||||
if err != nil {
|
||||
// Check if root points to a file (e.g. PrimarySync/Videos/file.mp4)
|
||||
newRoot, remote := dircache.SplitPath(root)
|
||||
tempF := &PhotosFs{
|
||||
name: f.name,
|
||||
root: newRoot,
|
||||
opt: f.opt,
|
||||
features: f.features,
|
||||
icloud: f.icloud,
|
||||
pacer: f.pacer,
|
||||
httpClient: f.httpClient,
|
||||
startTime: f.startTime,
|
||||
photos: f.photos, // nil here; photosService() creates lazily on first use
|
||||
}
|
||||
tempF.dirCache = dircache.New(newRoot, rootID, tempF)
|
||||
if err2 := tempF.dirCache.FindRoot(ctx, false); err2 != nil {
|
||||
return f, nil
|
||||
}
|
||||
_, err2 := tempF.NewObject(ctx, remote)
|
||||
if err2 != nil {
|
||||
return f, nil
|
||||
}
|
||||
// Root is a file - adjust f to parent dir, signal ErrorIsFile
|
||||
f.root = newRoot
|
||||
f.dirCache = tempF.dirCache
|
||||
return f, fs.ErrorIsFile
|
||||
}
|
||||
|
||||
return f, nil
|
||||
}
|
||||
|
||||
// photosService returns the PhotosService, creating it on first call
|
||||
func (f *PhotosFs) photosService(ctx context.Context) (*api.PhotosService, error) {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
if f.photos == nil {
|
||||
var err error
|
||||
f.photos, err = api.NewPhotosService(ctx, f.icloud, f.pacer, shouldRetry)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
return f.photos, nil
|
||||
}
|
||||
|
||||
// Name of the remote (as passed into NewFs)
|
||||
func (f *PhotosFs) Name() string {
|
||||
return f.name
|
||||
}
|
||||
|
||||
// Root of the remote (as passed into NewFs)
|
||||
func (f *PhotosFs) Root() string {
|
||||
return f.opt.Enc.ToStandardPath(f.root)
|
||||
}
|
||||
|
||||
// String converts this Fs to a string
|
||||
func (f *PhotosFs) String() string {
|
||||
return fmt.Sprintf("iCloud Photos root '%s'", f.root)
|
||||
}
|
||||
|
||||
// Precision of the object storage system
|
||||
func (f *PhotosFs) Precision() time.Duration {
|
||||
return time.Second
|
||||
}
|
||||
|
||||
// Hashes returns the supported hash sets
|
||||
func (f *PhotosFs) Hashes() hash.Set {
|
||||
return hash.Set(hash.None)
|
||||
}
|
||||
|
||||
// Features returns the optional features of this Fs
|
||||
func (f *PhotosFs) Features() *fs.Features {
|
||||
return f.features
|
||||
}
|
||||
|
||||
// List the objects and directories in dir into entries
|
||||
func (f *PhotosFs) List(ctx context.Context, dir string) (entries fs.DirEntries, err error) {
|
||||
photosService, err := f.photosService(ctx)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to get photos service: %w", err)
|
||||
}
|
||||
|
||||
dirID, err := f.dirCache.FindDir(ctx, dir, false)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
switch {
|
||||
case dirID == rootID:
|
||||
// List libraries
|
||||
libraries, err := photosService.GetLibraries(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
albumCounts, err := photosService.GetLibraryAlbumCounts(ctx)
|
||||
if err != nil {
|
||||
fs.Debugf(f, "Failed to get library album counts: %v", err)
|
||||
}
|
||||
|
||||
for libraryName := range libraries {
|
||||
count := int64(-1)
|
||||
if albumCounts != nil {
|
||||
if c, ok := albumCounts[libraryName]; ok {
|
||||
count = c
|
||||
}
|
||||
}
|
||||
d := fs.NewDir(path.Join(dir, f.opt.Enc.ToStandardName(libraryName)), f.startTime).SetItems(count)
|
||||
entries = append(entries, d)
|
||||
}
|
||||
|
||||
case strings.HasPrefix(dirID, "lib:"):
|
||||
// List albums in a library
|
||||
libraryName := strings.TrimPrefix(dirID, "lib:")
|
||||
libraries, err := photosService.GetLibraries(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
library, exists := libraries[libraryName]
|
||||
if !exists {
|
||||
return nil, fs.ErrorDirNotFound
|
||||
}
|
||||
|
||||
albums, err := library.GetAlbums(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
counts, err := library.GetAlbumCounts(ctx)
|
||||
if err != nil {
|
||||
fs.Debugf(f, "Failed to get album counts for library %q: %v", libraryName, err)
|
||||
}
|
||||
|
||||
for albumName := range albums {
|
||||
count := int64(-1)
|
||||
if counts != nil {
|
||||
if c, ok := counts[albumName]; ok {
|
||||
count = c
|
||||
}
|
||||
}
|
||||
d := fs.NewDir(path.Join(dir, f.opt.Enc.ToStandardName(albumName)), f.startTime).SetItems(count)
|
||||
entries = append(entries, d)
|
||||
}
|
||||
|
||||
case strings.HasPrefix(dirID, "album:"):
|
||||
// List contents of an album or folder
|
||||
libraryName, albumPath, ok := parseAlbumDirID(dirID)
|
||||
if !ok {
|
||||
return nil, fs.ErrorDirNotFound
|
||||
}
|
||||
|
||||
album, err := f.resolveAlbum(ctx, photosService, libraryName, albumPath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Folders list their child albums as subdirectories
|
||||
if album.IsFolder {
|
||||
for childName, child := range album.Children {
|
||||
d := fs.NewDir(path.Join(dir, f.opt.Enc.ToStandardName(childName)), f.startTime)
|
||||
if child.IsFolder {
|
||||
d.SetItems(int64(len(child.Children)))
|
||||
}
|
||||
entries = append(entries, d)
|
||||
}
|
||||
return entries, nil
|
||||
}
|
||||
|
||||
photos, err := album.GetPhotos(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
for _, photo := range photos {
|
||||
if photo.Filename == "" {
|
||||
continue
|
||||
}
|
||||
encodedName := f.opt.Enc.ToStandardName(photo.Filename)
|
||||
remotePath := encodedName
|
||||
if dir != "" {
|
||||
remotePath = path.Join(dir, encodedName)
|
||||
}
|
||||
o := f.newPhotosObject(remotePath, photo, libraryName)
|
||||
entries = append(entries, o)
|
||||
}
|
||||
|
||||
default:
|
||||
return nil, fs.ErrorDirNotFound
|
||||
}
|
||||
|
||||
return entries, nil
|
||||
}
|
||||
|
||||
// NewObject finds the Object at remote. If it can't be found
|
||||
// it returns the error fs.ErrorObjectNotFound
|
||||
func (f *PhotosFs) NewObject(ctx context.Context, remote string) (fs.Object, error) {
|
||||
dir, leaf := dircache.SplitPath(remote)
|
||||
filename := f.opt.Enc.FromStandardName(leaf)
|
||||
|
||||
dirID, err := f.dirCache.FindDir(ctx, dir, false)
|
||||
if err != nil {
|
||||
fs.Debugf(f, "NewObject(%s): FindDir: %v", remote, err)
|
||||
return nil, fs.ErrorObjectNotFound
|
||||
}
|
||||
|
||||
if !strings.HasPrefix(dirID, "album:") {
|
||||
return nil, fs.ErrorObjectNotFound
|
||||
}
|
||||
|
||||
libraryName, albumPath, ok := parseAlbumDirID(dirID)
|
||||
if !ok {
|
||||
return nil, fs.ErrorObjectNotFound
|
||||
}
|
||||
|
||||
photosService, err := f.photosService(ctx)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to get photos service: %w", err)
|
||||
}
|
||||
|
||||
album, err := f.resolveAlbum(ctx, photosService, libraryName, albumPath)
|
||||
if err != nil || album.IsFolder {
|
||||
fs.Debugf(f, "NewObject(%s): album resolve failed or is folder", remote)
|
||||
return nil, fs.ErrorObjectNotFound
|
||||
}
|
||||
|
||||
photo, err := album.GetPhotoByName(ctx, filename)
|
||||
if err != nil {
|
||||
fs.Debugf(f, "NewObject(%s): %v", remote, err)
|
||||
return nil, fs.ErrorObjectNotFound
|
||||
}
|
||||
|
||||
return f.newPhotosObject(remote, photo, libraryName), nil
|
||||
}
|
||||
|
||||
// newPhotosObject creates a PhotosObject from a Photo and zone
|
||||
func (f *PhotosFs) newPhotosObject(remote string, photo *api.Photo, zone string) *PhotosObject {
|
||||
return &PhotosObject{
|
||||
fs: f,
|
||||
remote: remote,
|
||||
size: photo.Size,
|
||||
modTime: time.UnixMilli(photo.AssetDate),
|
||||
masterID: photo.ID,
|
||||
zone: zone,
|
||||
resourceKey: photo.ResourceKey,
|
||||
width: photo.Width,
|
||||
height: photo.Height,
|
||||
addedDate: photo.AddedDate,
|
||||
isFavorite: photo.IsFavorite,
|
||||
isHidden: photo.IsHidden,
|
||||
}
|
||||
}
|
||||
|
||||
// Put is not supported for this read-only backend
|
||||
func (f *PhotosFs) Put(ctx context.Context, in io.Reader, src fs.ObjectInfo, options ...fs.OpenOption) (fs.Object, error) {
|
||||
return nil, fs.ErrorNotImplemented
|
||||
}
|
||||
|
||||
// Mkdir is not supported for this read-only backend
|
||||
func (f *PhotosFs) Mkdir(ctx context.Context, dir string) error {
|
||||
return fs.ErrorNotImplemented
|
||||
}
|
||||
|
||||
// Rmdir is not supported for this read-only backend
|
||||
func (f *PhotosFs) Rmdir(ctx context.Context, dir string) error {
|
||||
return fs.ErrorNotImplemented
|
||||
}
|
||||
|
||||
// FindLeaf finds a directory of name leaf in the folder with ID pathID
|
||||
func (f *PhotosFs) FindLeaf(ctx context.Context, pathID, leaf string) (pathIDOut string, found bool, err error) {
|
||||
decodedLeaf := f.opt.Enc.FromStandardName(leaf)
|
||||
|
||||
photosService, err := f.photosService(ctx)
|
||||
if err != nil {
|
||||
return "", false, fmt.Errorf("failed to get photos service: %w", err)
|
||||
}
|
||||
|
||||
if pathID == rootID {
|
||||
libraries, err := photosService.GetLibraries(ctx)
|
||||
if err != nil {
|
||||
return "", false, err
|
||||
}
|
||||
|
||||
if _, exists := libraries[decodedLeaf]; exists {
|
||||
return "lib:" + decodedLeaf, true, nil
|
||||
}
|
||||
return "", false, nil
|
||||
}
|
||||
|
||||
if strings.HasPrefix(pathID, "lib:") {
|
||||
libraryName := strings.TrimPrefix(pathID, "lib:")
|
||||
libraries, err := photosService.GetLibraries(ctx)
|
||||
if err != nil {
|
||||
return "", false, err
|
||||
}
|
||||
|
||||
library, exists := libraries[libraryName]
|
||||
if !exists {
|
||||
return "", false, fs.ErrorDirNotFound
|
||||
}
|
||||
|
||||
albums, err := library.GetAlbums(ctx)
|
||||
if err != nil {
|
||||
return "", false, err
|
||||
}
|
||||
|
||||
if _, exists := albums[decodedLeaf]; exists {
|
||||
albumID := "album:" + libraryName + ":" + decodedLeaf
|
||||
return albumID, true, nil
|
||||
}
|
||||
return "", false, nil
|
||||
}
|
||||
|
||||
if strings.HasPrefix(pathID, "album:") {
|
||||
// Check if this is a folder containing child albums
|
||||
libraryName, albumPath, ok := parseAlbumDirID(pathID)
|
||||
if !ok {
|
||||
return "", false, nil
|
||||
}
|
||||
|
||||
album, err := f.resolveAlbum(ctx, photosService, libraryName, albumPath)
|
||||
if err != nil {
|
||||
return "", false, nil
|
||||
}
|
||||
|
||||
if album.IsFolder {
|
||||
if _, exists := album.Children[decodedLeaf]; exists {
|
||||
childID := "album:" + libraryName + ":" + albumPath + "/" + decodedLeaf
|
||||
return childID, true, nil
|
||||
}
|
||||
}
|
||||
return "", false, nil
|
||||
}
|
||||
|
||||
return "", false, nil
|
||||
}
|
||||
|
||||
// CreateDir makes a directory with pathID as parent and name leaf
|
||||
func (f *PhotosFs) CreateDir(ctx context.Context, pathID, leaf string) (newID string, err error) {
|
||||
return "", fs.ErrorNotImplemented
|
||||
}
|
||||
|
||||
// Fs returns the parent Fs
|
||||
func (o *PhotosObject) Fs() fs.Info {
|
||||
return o.fs
|
||||
}
|
||||
|
||||
// Return a string version
|
||||
func (o *PhotosObject) String() string {
|
||||
if o == nil {
|
||||
return "<nil>"
|
||||
}
|
||||
return o.remote
|
||||
}
|
||||
|
||||
// Remote returns the remote path
|
||||
func (o *PhotosObject) Remote() string {
|
||||
return o.remote
|
||||
}
|
||||
|
||||
// ModTime returns the modification time of the object
|
||||
func (o *PhotosObject) ModTime(ctx context.Context) time.Time {
|
||||
return o.modTime
|
||||
}
|
||||
|
||||
// Size returns the size of an object in bytes
|
||||
func (o *PhotosObject) Size() int64 {
|
||||
return o.size
|
||||
}
|
||||
|
||||
// Storable returns a boolean as to whether this object is storable
|
||||
func (o *PhotosObject) Storable() bool {
|
||||
return true
|
||||
}
|
||||
|
||||
// Hash returns the hash of an object returning a lowercase hex string
|
||||
func (o *PhotosObject) Hash(ctx context.Context, ty hash.Type) (string, error) {
|
||||
return "", hash.ErrUnsupported
|
||||
}
|
||||
|
||||
// SetModTime sets the modification time of the object
|
||||
func (o *PhotosObject) SetModTime(ctx context.Context, modTime time.Time) error {
|
||||
return fs.ErrorCantSetModTime
|
||||
}
|
||||
|
||||
// Metadata returns metadata for the photo object
|
||||
func (o *PhotosObject) Metadata(ctx context.Context) (fs.Metadata, error) {
|
||||
metadata := make(fs.Metadata, 5)
|
||||
if o.width > 0 {
|
||||
metadata["width"] = strconv.Itoa(o.width)
|
||||
}
|
||||
if o.height > 0 {
|
||||
metadata["height"] = strconv.Itoa(o.height)
|
||||
}
|
||||
if o.addedDate > 0 {
|
||||
metadata["added-time"] = time.UnixMilli(o.addedDate).UTC().Format(time.RFC3339)
|
||||
}
|
||||
metadata["favorite"] = strconv.FormatBool(o.isFavorite)
|
||||
metadata["hidden"] = strconv.FormatBool(o.isHidden)
|
||||
return metadata, nil
|
||||
}
|
||||
|
||||
// Open an object for read, fetching a fresh download URL via records/lookup
|
||||
func (o *PhotosObject) Open(ctx context.Context, options ...fs.OpenOption) (io.ReadCloser, error) {
|
||||
photosService, err := o.fs.photosService(ctx)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to get photos service: %w", err)
|
||||
}
|
||||
|
||||
downloadURL, err := photosService.LookupDownloadURL(ctx, o.masterID, o.zone, o.resourceKey)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to get download URL for %q: %w", o.remote, err)
|
||||
}
|
||||
|
||||
fs.FixRangeOption(options, o.size)
|
||||
|
||||
var resp *http.Response
|
||||
err = o.fs.pacer.Call(func() (bool, error) {
|
||||
if resp != nil {
|
||||
_ = resp.Body.Close()
|
||||
resp = nil
|
||||
}
|
||||
req, err := http.NewRequestWithContext(ctx, "GET", downloadURL, nil)
|
||||
if err != nil {
|
||||
return false, fmt.Errorf("failed to create request: %w", err)
|
||||
}
|
||||
fs.OpenOptionAddHTTPHeaders(req.Header, options)
|
||||
|
||||
resp, err = o.fs.httpClient.Do(req)
|
||||
return shouldRetry(ctx, resp, err)
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if resp.StatusCode != http.StatusOK && resp.StatusCode != http.StatusPartialContent {
|
||||
_ = resp.Body.Close()
|
||||
return nil, fmt.Errorf("download %q failed: %s", o.remote, resp.Status)
|
||||
}
|
||||
|
||||
return resp.Body, nil
|
||||
}
|
||||
|
||||
// Update is not supported for this read-only backend
|
||||
func (o *PhotosObject) Update(ctx context.Context, in io.Reader, src fs.ObjectInfo, options ...fs.OpenOption) error {
|
||||
return fs.ErrorNotImplemented
|
||||
}
|
||||
|
||||
// Remove is not supported for this read-only backend
|
||||
func (o *PhotosObject) Remove(ctx context.Context) error {
|
||||
return fs.ErrorNotImplemented
|
||||
}
|
||||
|
||||
// resolveAlbum finds an album by path, traversing folder hierarchy
|
||||
// albumPath can be "AlbumName" or "FolderName/AlbumName" for nested albums
|
||||
func (f *PhotosFs) resolveAlbum(ctx context.Context, ps *api.PhotosService, libraryName, albumPath string) (*api.Album, error) {
|
||||
libraries, err := ps.GetLibraries(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
library, exists := libraries[libraryName]
|
||||
if !exists {
|
||||
return nil, fs.ErrorDirNotFound
|
||||
}
|
||||
|
||||
albums, err := library.GetAlbums(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return resolveAlbumPath(albums, albumPath)
|
||||
}
|
||||
|
||||
// resolveAlbumPath finds an album inside an album tree by slash-separated path
|
||||
func resolveAlbumPath(albums map[string]*api.Album, albumPath string) (*api.Album, error) {
|
||||
parts := strings.Split(albumPath, "/")
|
||||
current := albums
|
||||
for i, part := range parts {
|
||||
album, exists := current[part]
|
||||
if !exists {
|
||||
return nil, fs.ErrorDirNotFound
|
||||
}
|
||||
if i == len(parts)-1 {
|
||||
return album, nil
|
||||
}
|
||||
if !album.IsFolder || album.Children == nil {
|
||||
return nil, fs.ErrorDirNotFound
|
||||
}
|
||||
current = album.Children
|
||||
}
|
||||
return nil, fs.ErrorDirNotFound
|
||||
}
|
||||
|
||||
// parseAlbumDirID extracts library and album names from "album:lib:album" dirID
|
||||
func parseAlbumDirID(dirID string) (libraryName, albumName string, ok bool) {
|
||||
parts := strings.SplitN(strings.TrimPrefix(dirID, "album:"), ":", 2)
|
||||
if len(parts) != 2 {
|
||||
return "", "", false
|
||||
}
|
||||
return parts[0], parts[1], true
|
||||
}
|
||||
|
||||
// DirCacheFlush resets the directory cache - used in testing as an
|
||||
// optional interface
|
||||
func (f *PhotosFs) DirCacheFlush() {
|
||||
f.dirCache.ResetRoot()
|
||||
// Also flush API-layer caches so albums/photos are re-fetched
|
||||
// DirCacheFlusher interface has no ctx param; use a bounded timeout
|
||||
// to avoid hanging indefinitely if photosService triggers first init
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
if ps, err := f.photosService(ctx); err == nil {
|
||||
ps.FlushCaches()
|
||||
}
|
||||
}
|
||||
|
||||
// listRAlbumJob represents an album to be listed by a ListR worker
|
||||
type listRAlbumJob struct {
|
||||
album *api.Album
|
||||
zone string
|
||||
dirPath string // path relative to Fs root (e.g. "Videos" when f.root="PrimarySync")
|
||||
}
|
||||
|
||||
// ListR lists the objects and directories of the Fs starting from dir
|
||||
// recursively into out, calling callback for each batch of entries
|
||||
// Albums are listed in parallel using a goroutine pool
|
||||
func (f *PhotosFs) ListR(ctx context.Context, dir string, callback fs.ListRCallback) error {
|
||||
ps, err := f.photosService(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
helper := list.NewHelper(callback)
|
||||
var mu sync.Mutex // protects helper (not thread-safe)
|
||||
|
||||
addEntry := func(entry fs.DirEntry) error {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
return helper.Add(entry)
|
||||
}
|
||||
|
||||
// Resolve dir through dirCache - this respects f.root
|
||||
dirID, err := f.dirCache.FindDir(ctx, dir, false)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Collect album jobs, scoped to the resolved directory
|
||||
var jobs []listRAlbumJob
|
||||
|
||||
// collectAlbumJobs emits directory entries for albums/folders under basePath
|
||||
// and collects leaf albums as jobs for parallel photo listing
|
||||
// Recurses into folders to handle arbitrary nesting depth
|
||||
var collectAlbumJobs func(albums map[string]*api.Album, zone, basePath string) error
|
||||
collectAlbumJobs = func(albums map[string]*api.Album, zone, basePath string) error {
|
||||
for albumName, album := range albums {
|
||||
albumPath := path.Join(basePath, f.opt.Enc.ToStandardName(albumName))
|
||||
if err := addEntry(fs.NewDir(albumPath, f.startTime)); err != nil {
|
||||
return err
|
||||
}
|
||||
if album.IsFolder {
|
||||
if err := collectAlbumJobs(album.Children, zone, albumPath); err != nil {
|
||||
return err
|
||||
}
|
||||
} else {
|
||||
jobs = append(jobs, listRAlbumJob{album: album, zone: zone, dirPath: albumPath})
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
switch {
|
||||
case dirID == rootID:
|
||||
// Root level - emit libraries, then all albums
|
||||
libraries, err := ps.GetLibraries(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for libName, lib := range libraries {
|
||||
libPath := path.Join(dir, f.opt.Enc.ToStandardName(libName))
|
||||
if err := addEntry(fs.NewDir(libPath, f.startTime)); err != nil {
|
||||
return err
|
||||
}
|
||||
albums, err := lib.GetAlbums(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := collectAlbumJobs(albums, libName, libPath); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
case strings.HasPrefix(dirID, "lib:"):
|
||||
// Library level - emit and collect all albums
|
||||
libraryName := strings.TrimPrefix(dirID, "lib:")
|
||||
libraries, err := ps.GetLibraries(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
library, exists := libraries[libraryName]
|
||||
if !exists {
|
||||
return fs.ErrorDirNotFound
|
||||
}
|
||||
albums, err := library.GetAlbums(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := collectAlbumJobs(albums, libraryName, dir); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
case strings.HasPrefix(dirID, "album:"):
|
||||
// Album or folder level
|
||||
libraryName, albumPath, ok := parseAlbumDirID(dirID)
|
||||
if !ok {
|
||||
return fs.ErrorDirNotFound
|
||||
}
|
||||
album, err := f.resolveAlbum(ctx, ps, libraryName, albumPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if album.IsFolder {
|
||||
if err := collectAlbumJobs(album.Children, libraryName, dir); err != nil {
|
||||
return err
|
||||
}
|
||||
} else {
|
||||
// Single album - just list its photos
|
||||
jobs = append(jobs, listRAlbumJob{album: album, zone: libraryName, dirPath: dir})
|
||||
}
|
||||
|
||||
default:
|
||||
return fs.ErrorDirNotFound
|
||||
}
|
||||
|
||||
if len(jobs) == 0 {
|
||||
return helper.Flush()
|
||||
}
|
||||
|
||||
// Fan out album photo listing across goroutines
|
||||
ctx, cancel := context.WithCancel(ctx)
|
||||
defer cancel()
|
||||
|
||||
jobCh := make(chan listRAlbumJob, len(jobs))
|
||||
for _, job := range jobs {
|
||||
jobCh <- job
|
||||
}
|
||||
close(jobCh)
|
||||
|
||||
workers := fs.GetConfig(ctx).Checkers
|
||||
if len(jobs) < workers {
|
||||
workers = len(jobs)
|
||||
}
|
||||
errs := make(chan error, workers)
|
||||
|
||||
for range workers {
|
||||
go func() {
|
||||
for job := range jobCh {
|
||||
photos, err := job.album.GetPhotos(ctx)
|
||||
if err != nil {
|
||||
errs <- err
|
||||
cancel()
|
||||
return
|
||||
}
|
||||
for _, photo := range photos {
|
||||
if photo.Filename == "" {
|
||||
continue
|
||||
}
|
||||
remotePath := path.Join(job.dirPath, f.opt.Enc.ToStandardName(photo.Filename))
|
||||
o := f.newPhotosObject(remotePath, photo, job.zone)
|
||||
if err := addEntry(o); err != nil {
|
||||
errs <- err
|
||||
cancel()
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
errs <- nil
|
||||
}()
|
||||
}
|
||||
|
||||
for range workers {
|
||||
if e := <-errs; e != nil && err == nil {
|
||||
err = e
|
||||
}
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return helper.Flush()
|
||||
}
|
||||
|
||||
func notifyAlbumTree(albums map[string]*api.Album, base string, notifyFunc func(string, fs.EntryType)) {
|
||||
for name, album := range albums {
|
||||
albumPath := name
|
||||
if base != "" {
|
||||
albumPath = path.Join(base, name)
|
||||
}
|
||||
notifyFunc(albumPath, fs.EntryDirectory)
|
||||
if album.IsFolder {
|
||||
notifyAlbumTree(album.Children, albumPath, notifyFunc)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (f *PhotosFs) notifyZoneChange(ctx context.Context, ps *api.PhotosService, zone string, notifyFunc func(string, fs.EntryType)) {
|
||||
if f.root == "" {
|
||||
notifyFunc(zone, fs.EntryDirectory)
|
||||
return
|
||||
}
|
||||
|
||||
libraryName, albumPath, _ := strings.Cut(f.root, "/")
|
||||
if libraryName != zone {
|
||||
return
|
||||
}
|
||||
|
||||
// Always invalidate the mounted root when its backing zone changes
|
||||
notifyFunc("", fs.EntryDirectory)
|
||||
|
||||
libraries, err := ps.GetLibraries(ctx)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
library, ok := libraries[zone]
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
albums, err := library.GetAlbums(ctx)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
if albumPath == "" {
|
||||
notifyAlbumTree(albums, "", notifyFunc)
|
||||
return
|
||||
}
|
||||
|
||||
album, err := resolveAlbumPath(albums, albumPath)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
if album.IsFolder {
|
||||
notifyAlbumTree(album.Children, "", notifyFunc)
|
||||
}
|
||||
}
|
||||
|
||||
// ChangeNotify polls for changes and notifies the VFS when directories are modified
|
||||
func (f *PhotosFs) ChangeNotify(ctx context.Context, notifyFunc func(string, fs.EntryType), pollIntervalChan <-chan time.Duration) {
|
||||
go func() {
|
||||
var ticker *time.Ticker
|
||||
var tickerC <-chan time.Time
|
||||
for {
|
||||
select {
|
||||
case pollInterval, ok := <-pollIntervalChan:
|
||||
if !ok {
|
||||
if ticker != nil {
|
||||
ticker.Stop()
|
||||
}
|
||||
return
|
||||
}
|
||||
if ticker != nil {
|
||||
ticker.Stop()
|
||||
tickerC = nil
|
||||
}
|
||||
if pollInterval > 0 {
|
||||
ticker = time.NewTicker(pollInterval)
|
||||
tickerC = ticker.C
|
||||
}
|
||||
case <-tickerC:
|
||||
ps, err := f.photosService(ctx)
|
||||
if err != nil {
|
||||
fs.Debugf(f, "ChangeNotify: failed to get photos service: %v", err)
|
||||
continue
|
||||
}
|
||||
changedZones := ps.PollForChanges(ctx)
|
||||
for _, zone := range changedZones {
|
||||
f.notifyZoneChange(ctx, ps, zone, notifyFunc)
|
||||
}
|
||||
case <-ctx.Done():
|
||||
if ticker != nil {
|
||||
ticker.Stop()
|
||||
}
|
||||
return
|
||||
}
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
// Disconnect clears authentication state and removes disk caches
|
||||
func (f *PhotosFs) Disconnect(ctx context.Context) error {
|
||||
return disconnectClient(f.m, f.icloud)
|
||||
}
|
||||
|
||||
// Check interfaces are satisfied
|
||||
var (
|
||||
_ fs.Fs = (*PhotosFs)(nil)
|
||||
_ fs.Disconnecter = (*PhotosFs)(nil)
|
||||
_ fs.ListRer = (*PhotosFs)(nil)
|
||||
_ fs.ChangeNotifier = (*PhotosFs)(nil)
|
||||
_ fs.DirCacheFlusher = (*PhotosFs)(nil)
|
||||
_ fs.Object = (*PhotosObject)(nil)
|
||||
_ fs.Metadataer = (*PhotosObject)(nil)
|
||||
)
|
||||
@@ -0,0 +1,516 @@
|
||||
//go:build !plan9 && !solaris
|
||||
|
||||
package iclouddrive
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sort"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/rclone/rclone/backend/iclouddrive/api"
|
||||
"github.com/rclone/rclone/fs"
|
||||
"github.com/rclone/rclone/lib/dircache"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func newTestPhotosFs() *PhotosFs {
|
||||
return &PhotosFs{
|
||||
name: "test-icp",
|
||||
root: "",
|
||||
opt: Options{},
|
||||
}
|
||||
}
|
||||
|
||||
func TestSmartAlbumDefinitions(t *testing.T) {
|
||||
expected := map[string]struct {
|
||||
ObjectType string
|
||||
ListType string
|
||||
}{
|
||||
"All Photos": {"CPLAssetByAssetDateWithoutHiddenOrDeleted", "CPLAssetAndMasterByAssetDateWithoutHiddenOrDeleted"},
|
||||
"Time-lapse": {"CPLAssetInSmartAlbumByAssetDate:Timelapse", "CPLAssetAndMasterInSmartAlbumByAssetDate"},
|
||||
"Videos": {"CPLAssetInSmartAlbumByAssetDate:Video", "CPLAssetAndMasterInSmartAlbumByAssetDate"},
|
||||
"Slo-mo": {"CPLAssetInSmartAlbumByAssetDate:Slomo", "CPLAssetAndMasterInSmartAlbumByAssetDate"},
|
||||
"Bursts": {"CPLAssetBurstStackAssetByAssetDate", "CPLBurstStackAssetAndMasterByAssetDate"},
|
||||
"Favorites": {"CPLAssetInSmartAlbumByAssetDate:Favorite", "CPLAssetAndMasterInSmartAlbumByAssetDate"},
|
||||
"Panoramas": {"CPLAssetInSmartAlbumByAssetDate:Panorama", "CPLAssetAndMasterInSmartAlbumByAssetDate"},
|
||||
"Screenshots": {"CPLAssetInSmartAlbumByAssetDate:Screenshot", "CPLAssetAndMasterInSmartAlbumByAssetDate"},
|
||||
"Live": {"CPLAssetInSmartAlbumByAssetDate:Live", "CPLAssetAndMasterInSmartAlbumByAssetDate"},
|
||||
"Recently Deleted": {"CPLAssetDeletedByExpungedDate", "CPLAssetAndMasterDeletedByExpungedDate"},
|
||||
"Hidden": {"CPLAssetHiddenByAssetDate", "CPLAssetAndMasterHiddenByAssetDate"},
|
||||
"Portrait": {"CPLAssetInSmartAlbumByAssetDate:Depth", "CPLAssetAndMasterInSmartAlbumByAssetDate"},
|
||||
"Long Exposure": {"CPLAssetInSmartAlbumByAssetDate:Exposure", "CPLAssetAndMasterInSmartAlbumByAssetDate"},
|
||||
"Animated": {"CPLAssetInSmartAlbumByAssetDate:Animated", "CPLAssetAndMasterInSmartAlbumByAssetDate"},
|
||||
}
|
||||
|
||||
for name, exp := range expected {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
album, exists := api.SmartAlbums[name]
|
||||
require.True(t, exists, "smart album %q must be defined", name)
|
||||
assert.Equal(t, exp.ObjectType, album.ObjectType)
|
||||
assert.Equal(t, exp.ListType, album.ListType)
|
||||
})
|
||||
}
|
||||
|
||||
assert.Equal(t, len(expected), len(api.SmartAlbums))
|
||||
}
|
||||
|
||||
func TestSmartAlbumFilters(t *testing.T) {
|
||||
filtered := map[string]string{
|
||||
"Time-lapse": "TIMELAPSE",
|
||||
"Videos": "VIDEO",
|
||||
"Slo-mo": "SLOMO",
|
||||
"Favorites": "FAVORITE",
|
||||
"Panoramas": "PANORAMA",
|
||||
"Screenshots": "SCREENSHOT",
|
||||
"Live": "LIVE",
|
||||
"Portrait": "DEPTH",
|
||||
"Long Exposure": "EXPOSURE",
|
||||
"Animated": "ANIMATED",
|
||||
}
|
||||
|
||||
for name, filterVal := range filtered {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
album := api.SmartAlbums[name]
|
||||
require.NotEmpty(t, album.Filters, "album %q must have filters", name)
|
||||
assert.Equal(t, "smartAlbum", album.Filters[0].FieldName)
|
||||
assert.Equal(t, "EQUALS", album.Filters[0].Comparator)
|
||||
fv, ok := album.Filters[0].FieldValue.(map[string]string)
|
||||
require.True(t, ok, "filter value must be map[string]string")
|
||||
assert.Equal(t, filterVal, fv["value"])
|
||||
})
|
||||
}
|
||||
|
||||
unfiltered := []string{"All Photos", "Bursts", "Recently Deleted", "Hidden"}
|
||||
for _, name := range unfiltered {
|
||||
t.Run(name+"_no_filter", func(t *testing.T) {
|
||||
album := api.SmartAlbums[name]
|
||||
assert.Empty(t, album.Filters, "album %q must not have filters", name)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseAlbumDirID(t *testing.T) {
|
||||
lib, album, ok := parseAlbumDirID("album:PrimarySync:Favorites")
|
||||
assert.True(t, ok)
|
||||
assert.Equal(t, "PrimarySync", lib)
|
||||
assert.Equal(t, "Favorites", album)
|
||||
|
||||
// Colon in album name
|
||||
lib, album, ok = parseAlbumDirID("album:PrimarySync:My:Album")
|
||||
assert.True(t, ok)
|
||||
assert.Equal(t, "PrimarySync", lib)
|
||||
assert.Equal(t, "My:Album", album)
|
||||
|
||||
// Missing album
|
||||
_, _, ok = parseAlbumDirID("album:PrimarySync")
|
||||
assert.False(t, ok)
|
||||
|
||||
// Empty
|
||||
_, _, ok = parseAlbumDirID("album:")
|
||||
assert.False(t, ok)
|
||||
}
|
||||
|
||||
func TestPhotosObject_Metadata(t *testing.T) {
|
||||
o := &PhotosObject{
|
||||
fs: &PhotosFs{name: "test"},
|
||||
remote: "PrimarySync/All Photos/IMG_0001.HEIC",
|
||||
size: 4200000,
|
||||
modTime: time.Date(2025, 6, 15, 14, 30, 0, 0, time.UTC),
|
||||
masterID: "master-001",
|
||||
zone: "PrimarySync",
|
||||
width: 4032,
|
||||
height: 3024,
|
||||
addedDate: 1718459400000, // 2024-06-15T13:50:00Z in millis
|
||||
isFavorite: true,
|
||||
isHidden: false,
|
||||
}
|
||||
|
||||
metadata, err := o.Metadata(context.Background())
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "4032", metadata["width"])
|
||||
assert.Equal(t, "3024", metadata["height"])
|
||||
assert.Equal(t, "2024-06-15T13:50:00Z", metadata["added-time"])
|
||||
assert.Equal(t, "true", metadata["favorite"])
|
||||
assert.Equal(t, "false", metadata["hidden"])
|
||||
}
|
||||
|
||||
func TestPhotosObject_MetadataZeroDimensions(t *testing.T) {
|
||||
// Live Photo .MOV companion - no width/height, not favorite
|
||||
o := &PhotosObject{
|
||||
fs: &PhotosFs{name: "test"},
|
||||
remote: "PrimarySync/Live/IMG_3031.MOV",
|
||||
}
|
||||
|
||||
metadata, err := o.Metadata(context.Background())
|
||||
require.NoError(t, err)
|
||||
_, hasWidth := metadata["width"]
|
||||
_, hasHeight := metadata["height"]
|
||||
assert.False(t, hasWidth, "zero width should be omitted")
|
||||
assert.False(t, hasHeight, "zero height should be omitted")
|
||||
assert.Equal(t, "false", metadata["favorite"])
|
||||
assert.Equal(t, "false", metadata["hidden"])
|
||||
}
|
||||
|
||||
// newTestPhotosService builds a PhotosService with pre-populated libraries and albums
|
||||
// for testing resolveAlbum and FindLeaf without HTTP calls
|
||||
func newTestPhotosService() *api.PhotosService {
|
||||
return api.NewTestPhotosService(map[string]map[string]*api.Album{
|
||||
"PrimarySync": {
|
||||
"All Photos": {Name: "All Photos", ObjectType: "CPLAssetByAssetDateWithoutHiddenOrDeleted"},
|
||||
"Videos": {Name: "Videos", ObjectType: "CPLAssetInSmartAlbumByAssetDate:Video"},
|
||||
"UserAlbum": {Name: "UserAlbum", ObjectType: "CPLContainerRelationNotDeletedByAssetDate:rec1", RecordName: "rec1"},
|
||||
"Folder": {
|
||||
Name: "Folder", RecordName: "folder1", IsFolder: true,
|
||||
Children: map[string]*api.Album{
|
||||
"ChildAlbum": {Name: "ChildAlbum", ObjectType: "CPLContainerRelationNotDeletedByAssetDate:rec2", RecordName: "rec2"},
|
||||
"NestedFolder": {
|
||||
Name: "NestedFolder", RecordName: "folder2", IsFolder: true,
|
||||
Children: map[string]*api.Album{
|
||||
"LeafAlbum": {Name: "LeafAlbum", ObjectType: "CPLContainerRelationNotDeletedByAssetDate:rec3", RecordName: "rec3"},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
func TestResolveAlbum(t *testing.T) {
|
||||
f := newTestPhotosFs()
|
||||
f.photos = newTestPhotosService()
|
||||
ctx := context.Background()
|
||||
|
||||
t.Run("simple album", func(t *testing.T) {
|
||||
album, err := f.resolveAlbum(ctx, f.photos, "PrimarySync", "Videos")
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "Videos", album.Name)
|
||||
})
|
||||
|
||||
t.Run("nested folder child", func(t *testing.T) {
|
||||
album, err := f.resolveAlbum(ctx, f.photos, "PrimarySync", "Folder/ChildAlbum")
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "ChildAlbum", album.Name)
|
||||
})
|
||||
|
||||
t.Run("two-level nesting", func(t *testing.T) {
|
||||
album, err := f.resolveAlbum(ctx, f.photos, "PrimarySync", "Folder/NestedFolder/LeafAlbum")
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "LeafAlbum", album.Name)
|
||||
})
|
||||
|
||||
t.Run("folder itself", func(t *testing.T) {
|
||||
album, err := f.resolveAlbum(ctx, f.photos, "PrimarySync", "Folder")
|
||||
require.NoError(t, err)
|
||||
assert.True(t, album.IsFolder)
|
||||
})
|
||||
|
||||
t.Run("missing album", func(t *testing.T) {
|
||||
_, err := f.resolveAlbum(ctx, f.photos, "PrimarySync", "NoSuchAlbum")
|
||||
assert.ErrorIs(t, err, fs.ErrorDirNotFound)
|
||||
})
|
||||
|
||||
t.Run("missing library", func(t *testing.T) {
|
||||
_, err := f.resolveAlbum(ctx, f.photos, "NoSuchLib", "Videos")
|
||||
assert.ErrorIs(t, err, fs.ErrorDirNotFound)
|
||||
})
|
||||
|
||||
t.Run("traverse into non-folder", func(t *testing.T) {
|
||||
_, err := f.resolveAlbum(ctx, f.photos, "PrimarySync", "Videos/SubAlbum")
|
||||
assert.ErrorIs(t, err, fs.ErrorDirNotFound)
|
||||
})
|
||||
}
|
||||
|
||||
func TestFindLeaf(t *testing.T) {
|
||||
f := newTestPhotosFs()
|
||||
f.photos = newTestPhotosService()
|
||||
ctx := context.Background()
|
||||
|
||||
t.Run("root to library", func(t *testing.T) {
|
||||
id, found, err := f.FindLeaf(ctx, rootID, "PrimarySync")
|
||||
require.NoError(t, err)
|
||||
assert.True(t, found)
|
||||
assert.Equal(t, "lib:PrimarySync", id)
|
||||
})
|
||||
|
||||
t.Run("root to missing library", func(t *testing.T) {
|
||||
_, found, err := f.FindLeaf(ctx, rootID, "NoSuchLib")
|
||||
require.NoError(t, err)
|
||||
assert.False(t, found)
|
||||
})
|
||||
|
||||
t.Run("library to album", func(t *testing.T) {
|
||||
id, found, err := f.FindLeaf(ctx, "lib:PrimarySync", "Videos")
|
||||
require.NoError(t, err)
|
||||
assert.True(t, found)
|
||||
assert.Equal(t, "album:PrimarySync:Videos", id)
|
||||
})
|
||||
|
||||
t.Run("library to missing album", func(t *testing.T) {
|
||||
_, found, err := f.FindLeaf(ctx, "lib:PrimarySync", "NoSuchAlbum")
|
||||
require.NoError(t, err)
|
||||
assert.False(t, found)
|
||||
})
|
||||
|
||||
t.Run("missing library returns ErrorDirNotFound", func(t *testing.T) {
|
||||
_, _, err := f.FindLeaf(ctx, "lib:NoSuchLib", "Videos")
|
||||
assert.ErrorIs(t, err, fs.ErrorDirNotFound)
|
||||
})
|
||||
|
||||
t.Run("folder to child album", func(t *testing.T) {
|
||||
id, found, err := f.FindLeaf(ctx, "album:PrimarySync:Folder", "ChildAlbum")
|
||||
require.NoError(t, err)
|
||||
assert.True(t, found)
|
||||
assert.Equal(t, "album:PrimarySync:Folder/ChildAlbum", id)
|
||||
})
|
||||
|
||||
t.Run("non-folder album returns not found", func(t *testing.T) {
|
||||
_, found, err := f.FindLeaf(ctx, "album:PrimarySync:Videos", "SubAlbum")
|
||||
require.NoError(t, err)
|
||||
assert.False(t, found)
|
||||
})
|
||||
|
||||
t.Run("unknown prefix returns not found", func(t *testing.T) {
|
||||
_, found, err := f.FindLeaf(ctx, "unknown:prefix", "leaf")
|
||||
require.NoError(t, err)
|
||||
assert.False(t, found)
|
||||
})
|
||||
}
|
||||
|
||||
func TestResolveAlbum_EmptyPath(t *testing.T) {
|
||||
f := newTestPhotosFs()
|
||||
f.photos = newTestPhotosService()
|
||||
ctx := context.Background()
|
||||
|
||||
// parseAlbumDirID("album:PrimarySync:") returns ("PrimarySync", "", true)
|
||||
// This empty album path feeds into resolveAlbum
|
||||
_, err := f.resolveAlbum(ctx, f.photos, "PrimarySync", "")
|
||||
assert.ErrorIs(t, err, fs.ErrorDirNotFound, "empty album path should return ErrorDirNotFound")
|
||||
}
|
||||
|
||||
func TestNewObject_ErrorPaths(t *testing.T) {
|
||||
f := newTestPhotosFs()
|
||||
f.photos = newTestPhotosService()
|
||||
ctx := context.Background()
|
||||
|
||||
// Pre-populate photoCache on the "Videos" album so GetPhotoByName works
|
||||
// without HTTP. Access via the test PhotosService's internal structure
|
||||
libs, _ := f.photos.GetLibraries(ctx)
|
||||
albums, _ := libs["PrimarySync"].GetAlbums(ctx)
|
||||
videos := albums["Videos"]
|
||||
videos.SetTestPhotoCache(map[string]*api.Photo{
|
||||
"existing.mp4": {ID: "m1", Filename: "existing.mp4", Size: 1024, ResourceKey: "resOriginalRes"},
|
||||
})
|
||||
|
||||
// NewObject depends on dircache.FindDir which requires FindRoot + HTTP
|
||||
// Test the reachable paths through resolveAlbum and GetPhotoByName
|
||||
|
||||
// Test resolveAlbum → folder → GetPhotoByName on folder (should fail)
|
||||
album, err := f.resolveAlbum(ctx, f.photos, "PrimarySync", "Folder")
|
||||
require.NoError(t, err)
|
||||
assert.True(t, album.IsFolder, "Folder should be a folder")
|
||||
// NewObject would return ErrorObjectNotFound for a folder path
|
||||
|
||||
// Test GetPhotoByName on album with populated cache
|
||||
photo, err := videos.GetPhotoByName(ctx, "existing.mp4")
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "m1", photo.ID)
|
||||
|
||||
// Test GetPhotoByName cache miss
|
||||
_, err = videos.GetPhotoByName(ctx, "nonexistent.mp4")
|
||||
assert.Error(t, err)
|
||||
assert.Contains(t, err.Error(), "nonexistent.mp4")
|
||||
}
|
||||
|
||||
func TestParseAlbumDirID_Exhaustive(t *testing.T) {
|
||||
tests := []struct {
|
||||
input string
|
||||
lib, album string
|
||||
ok bool
|
||||
}{
|
||||
{"album:PrimarySync:Videos", "PrimarySync", "Videos", true},
|
||||
{"album:PrimarySync:Folder/Child", "PrimarySync", "Folder/Child", true},
|
||||
{"album:PrimarySync:Name:With:Colons", "PrimarySync", "Name:With:Colons", true},
|
||||
{"album:PrimarySync:", "PrimarySync", "", true},
|
||||
{"album:", "", "", false},
|
||||
{"notalbum:foo:bar", "notalbum", "foo:bar", true}, // strips "album:" prefix literally, "notalbum:" stays
|
||||
{"", "", "", false},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
lib, album, ok := parseAlbumDirID(tt.input)
|
||||
if tt.ok {
|
||||
assert.True(t, ok, "input=%q", tt.input)
|
||||
assert.Equal(t, tt.lib, lib, "input=%q", tt.input)
|
||||
assert.Equal(t, tt.album, album, "input=%q", tt.input)
|
||||
} else {
|
||||
assert.False(t, ok, "input=%q", tt.input)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// setEmptyPhotoCaches recursively sets empty photo caches on all leaf albums
|
||||
// so GetPhotos returns without HTTP (service=nil test fast path)
|
||||
func setEmptyPhotoCaches(albums map[string]*api.Album) {
|
||||
for _, album := range albums {
|
||||
if album.IsFolder {
|
||||
setEmptyPhotoCaches(album.Children)
|
||||
} else {
|
||||
album.SetTestPhotoCache(map[string]*api.Photo{})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestListR_NestedFolderRecursion(t *testing.T) {
|
||||
f := newTestPhotosFs()
|
||||
f.photos = newTestPhotosService()
|
||||
f.dirCache = dircache.New("", rootID, f)
|
||||
f.features = (&fs.Features{}).Fill(context.Background(), f)
|
||||
f.startTime = time.Now()
|
||||
ctx := context.Background()
|
||||
|
||||
err := f.dirCache.FindRoot(ctx, false)
|
||||
require.NoError(t, err)
|
||||
|
||||
// Pre-populate empty photo caches on all leaf albums
|
||||
libs, err := f.photos.GetLibraries(ctx)
|
||||
require.NoError(t, err)
|
||||
for _, lib := range libs {
|
||||
albums, err := lib.GetAlbums(ctx)
|
||||
require.NoError(t, err)
|
||||
setEmptyPhotoCaches(albums)
|
||||
}
|
||||
|
||||
var dirs []string
|
||||
err = f.ListR(ctx, "", func(entries fs.DirEntries) error {
|
||||
for _, entry := range entries {
|
||||
if _, ok := entry.(fs.Directory); ok {
|
||||
dirs = append(dirs, entry.Remote())
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
require.NoError(t, err)
|
||||
sort.Strings(dirs)
|
||||
|
||||
// All directories must be present, including deeply nested ones
|
||||
assert.Contains(t, dirs, "PrimarySync")
|
||||
assert.Contains(t, dirs, "PrimarySync/All Photos")
|
||||
assert.Contains(t, dirs, "PrimarySync/Videos")
|
||||
assert.Contains(t, dirs, "PrimarySync/Folder")
|
||||
assert.Contains(t, dirs, "PrimarySync/Folder/ChildAlbum")
|
||||
assert.Contains(t, dirs, "PrimarySync/Folder/NestedFolder")
|
||||
assert.Contains(t, dirs, "PrimarySync/Folder/NestedFolder/LeafAlbum")
|
||||
}
|
||||
|
||||
func TestListR_FolderLevel(t *testing.T) {
|
||||
f := &PhotosFs{
|
||||
name: "test",
|
||||
root: "PrimarySync/Folder",
|
||||
opt: Options{},
|
||||
startTime: time.Now(),
|
||||
}
|
||||
f.features = (&fs.Features{}).Fill(context.Background(), f)
|
||||
f.photos = newTestPhotosService()
|
||||
f.dirCache = dircache.New("PrimarySync/Folder", rootID, f)
|
||||
ctx := context.Background()
|
||||
|
||||
err := f.dirCache.FindRoot(ctx, false)
|
||||
require.NoError(t, err)
|
||||
|
||||
libs, _ := f.photos.GetLibraries(ctx)
|
||||
for _, lib := range libs {
|
||||
albums, _ := lib.GetAlbums(ctx)
|
||||
setEmptyPhotoCaches(albums)
|
||||
}
|
||||
|
||||
var dirs []string
|
||||
err = f.ListR(ctx, "", func(entries fs.DirEntries) error {
|
||||
for _, entry := range entries {
|
||||
if _, ok := entry.(fs.Directory); ok {
|
||||
dirs = append(dirs, entry.Remote())
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
require.NoError(t, err)
|
||||
|
||||
// When rooted at a folder, should see children recursively
|
||||
assert.Contains(t, dirs, "ChildAlbum")
|
||||
assert.Contains(t, dirs, "NestedFolder")
|
||||
assert.Contains(t, dirs, "NestedFolder/LeafAlbum")
|
||||
}
|
||||
|
||||
func TestNotifyZoneChange(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
ps := newTestPhotosService()
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
root string
|
||||
zone string
|
||||
want []string
|
||||
}{
|
||||
{
|
||||
name: "top level root",
|
||||
root: "",
|
||||
zone: "PrimarySync",
|
||||
want: []string{"PrimarySync"},
|
||||
},
|
||||
{
|
||||
name: "library root",
|
||||
root: "PrimarySync",
|
||||
zone: "PrimarySync",
|
||||
want: []string{"", "All Photos", "Folder", "Folder/ChildAlbum", "Folder/NestedFolder", "Folder/NestedFolder/LeafAlbum", "UserAlbum", "Videos"},
|
||||
},
|
||||
{
|
||||
name: "folder root",
|
||||
root: "PrimarySync/Folder",
|
||||
zone: "PrimarySync",
|
||||
want: []string{"", "ChildAlbum", "NestedFolder", "NestedFolder/LeafAlbum"},
|
||||
},
|
||||
{
|
||||
name: "nested folder root",
|
||||
root: "PrimarySync/Folder/NestedFolder",
|
||||
zone: "PrimarySync",
|
||||
want: []string{"", "LeafAlbum"},
|
||||
},
|
||||
{
|
||||
name: "leaf album root",
|
||||
root: "PrimarySync/Folder/NestedFolder/LeafAlbum",
|
||||
zone: "PrimarySync",
|
||||
want: []string{""},
|
||||
},
|
||||
{
|
||||
name: "missing nested album still invalidates root",
|
||||
root: "PrimarySync/Folder/NoSuchAlbum",
|
||||
zone: "PrimarySync",
|
||||
want: []string{""},
|
||||
},
|
||||
{
|
||||
name: "different zone ignored",
|
||||
root: "PrimarySync/Folder/NestedFolder",
|
||||
zone: "SharedSync",
|
||||
want: nil,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
f := newTestPhotosFs()
|
||||
f.root = tt.root
|
||||
|
||||
var got []string
|
||||
f.notifyZoneChange(ctx, ps, tt.zone, func(remote string, entryType fs.EntryType) {
|
||||
assert.Equal(t, fs.EntryDirectory, entryType)
|
||||
got = append(got, remote)
|
||||
})
|
||||
|
||||
sort.Strings(got)
|
||||
want := append([]string(nil), tt.want...)
|
||||
sort.Strings(want)
|
||||
assert.Equal(t, want, got)
|
||||
})
|
||||
}
|
||||
}
|
||||
+135
-24
@@ -1,20 +1,25 @@
|
||||
---
|
||||
title: "iCloud Drive"
|
||||
description: "Rclone docs for iCloud Drive"
|
||||
title: "iCloud Drive and Photos"
|
||||
description: "Rclone docs for iCloud Drive and Photos"
|
||||
versionIntroduced: "v1.69"
|
||||
---
|
||||
|
||||
# iCloud Drive
|
||||
# {{< icon "fa fa-cloud" >}} iCloud Drive and Photos
|
||||
|
||||
## Configuration
|
||||
|
||||
The initial setup for an iCloud Drive backend involves getting a trust token/session.
|
||||
This can be done by simply using the regular iCloud password, and accepting the code
|
||||
prompt on another iCloud connected device.
|
||||
The initial setup for an iCloud backend involves getting a trust token/session.
|
||||
This uses your regular Apple ID password plus 2FA, either from a trusted
|
||||
device prompt or an SMS code sent to a trusted phone number.
|
||||
|
||||
**IMPORTANT**: App-specific passwords are not accepted. Only use your
|
||||
regular Apple ID password and 2FA.
|
||||
|
||||
This backend serves two Apple services:
|
||||
|
||||
- `drive` - iCloud Drive (default)
|
||||
- `photos` - iCloud Photos
|
||||
|
||||
`rclone config` walks you through the token creation. The trust token is valid
|
||||
for 30 days. After which you will have to reauthenticate with `rclone reconnect`
|
||||
or `rclone config`.
|
||||
@@ -30,10 +35,12 @@ The authentication flow is:
|
||||
|
||||
1. rclone initiates a session with Apple's identity service
|
||||
2. An SRP key exchange takes place (your password is used locally to derive a key)
|
||||
3. Apple sends a 2FA prompt to your trusted devices
|
||||
3. Apple sends a 2FA prompt to your trusted devices, or lets you request an
|
||||
SMS code
|
||||
4. After you enter the 2FA code, rclone receives a trust token for future sessions
|
||||
|
||||
Here is an example of how to make a remote called `iclouddrive`. First run:
|
||||
Here is an example of how to make a Photos remote called `icloudphotos`.
|
||||
For iCloud Drive, leave `service` at its default `drive` value. First run:
|
||||
|
||||
```console
|
||||
rclone config
|
||||
@@ -47,7 +54,7 @@ n) New remote
|
||||
s) Set configuration password
|
||||
q) Quit config
|
||||
n/s/q> n
|
||||
name> iclouddrive
|
||||
name> icloudphotos
|
||||
Option Storage.
|
||||
Type of storage to configure.
|
||||
Choose a number from below, or type in your own value.
|
||||
@@ -56,6 +63,15 @@ XX / iCloud Drive
|
||||
\ (iclouddrive)
|
||||
[snip]
|
||||
Storage> iclouddrive
|
||||
Option service.
|
||||
iCloud service to use.
|
||||
Choose a number from below, or type in your own value of type string.
|
||||
Press Enter for the default (drive).
|
||||
1 / iCloud Drive
|
||||
\ (drive)
|
||||
2 / iCloud Photos
|
||||
\ (photos)
|
||||
service> 2
|
||||
Option apple_id.
|
||||
Apple ID.
|
||||
Enter a value.
|
||||
@@ -75,13 +91,14 @@ y) Yes
|
||||
n) No (default)
|
||||
y/n> n
|
||||
Option config_2fa.
|
||||
Two-factor authentication: please enter your 2FA code
|
||||
Two-factor authentication: enter your 2FA code or type 'sms' for a text message
|
||||
Enter a value.
|
||||
config_2fa> 2FACODE
|
||||
Remote config
|
||||
--------------------
|
||||
[iclouddrive]
|
||||
[icloudphotos]
|
||||
- type: iclouddrive
|
||||
- service: photos
|
||||
- apple_id: APPLEID
|
||||
- password: *** ENCRYPTED ***
|
||||
- cookies: ****************************
|
||||
@@ -93,30 +110,124 @@ d) Delete this remote
|
||||
y/e/d> y
|
||||
```
|
||||
|
||||
## iCloud Photos
|
||||
|
||||
The iCloud Drive backend also supports accessing iCloud Photos by setting the
|
||||
`service` option to `photos`:
|
||||
|
||||
```console
|
||||
rclone lsd iclouddrive: --iclouddrive-service photos
|
||||
```
|
||||
|
||||
This presents a read-only hierarchy rooted at photo libraries:
|
||||
|
||||
- **Level 1**: Photo libraries — your personal library (`PrimarySync`) and
|
||||
any Shared Photo Library (`SharedSync-XXXX`)
|
||||
- **Level 2+**: Albums and folders within a library, nested recursively as in
|
||||
Apple Photos
|
||||
- **Leaf level**: Photos/videos within an album, including Live Photo `.MOV`
|
||||
companions
|
||||
|
||||
Examples:
|
||||
|
||||
```console
|
||||
# List libraries
|
||||
rclone lsd iclouddrive: --iclouddrive-service photos
|
||||
|
||||
# List albums in your primary library
|
||||
rclone lsd iclouddrive:PrimarySync/ --iclouddrive-service photos
|
||||
|
||||
# List photos in an album
|
||||
rclone ls iclouddrive:PrimarySync/All\ Photos/ --iclouddrive-service photos
|
||||
|
||||
# Download a photo
|
||||
rclone copy iclouddrive:PrimarySync/Favorites/IMG_0001.HEIC /tmp/ --iclouddrive-service photos
|
||||
```
|
||||
|
||||
You can either:
|
||||
|
||||
- set `service = photos` in `rclone config` for a dedicated Photos remote
|
||||
- keep `service = drive` and pass `--iclouddrive-service photos` when needed
|
||||
|
||||
### Metadata
|
||||
|
||||
With `--metadata`, Photos entries expose these read-only metadata keys:
|
||||
|
||||
- `width`
|
||||
- `height`
|
||||
- `added-time`
|
||||
- `favorite`
|
||||
- `hidden`
|
||||
|
||||
These metadata keys are only available when `service = photos`.
|
||||
|
||||
### Caching
|
||||
|
||||
iCloud Photos caches album listings to disk for fast subsequent access.
|
||||
On the first run, listing a large album uses parallel `startRank`
|
||||
partitions to fetch pages concurrently. After that, a lightweight change
|
||||
check (~200ms) determines whether the cache is still valid.
|
||||
|
||||
Cache location: `~/.cache/rclone/iclouddrive-photos/<remote>/<zone>/`
|
||||
|
||||
To clear the cache: delete that directory or run `rclone config reconnect`.
|
||||
|
||||
### FUSE mounts
|
||||
|
||||
For mounting iCloud Photos via `rclone mount`, the following flags
|
||||
are recommended:
|
||||
|
||||
rclone mount remote: /mnt/photos \
|
||||
--iclouddrive-service photos \
|
||||
--vfs-refresh \
|
||||
--dir-cache-time 1h \
|
||||
--vfs-cache-mode full \
|
||||
--attr-timeout 1m \
|
||||
--read-only
|
||||
|
||||
- `--vfs-refresh` pre-warms directory caches in the background on mount
|
||||
start so that albums are ready when you browse them
|
||||
- `--dir-cache-time 1h` extends the in-memory cache lifetime beyond the
|
||||
default 5 minutes (change detection is fast, so this is safe)
|
||||
- `--vfs-cache-mode full` caches downloaded photos and videos to local
|
||||
disk for fast repeated access
|
||||
- `--attr-timeout 1m` reduces kernel attribute lookups (safe because
|
||||
the backend is read-only)
|
||||
- `--read-only` prevents confusing write errors
|
||||
|
||||
The first listing of a very large album (e.g. 75,000 items in "All
|
||||
Photos") can take several minutes due to API pagination limits. This
|
||||
happens once — subsequent listings use the disk cache.
|
||||
|
||||
### Limitations
|
||||
|
||||
iCloud Photos is read-only. Upload, delete, rename, and move operations
|
||||
are not supported.
|
||||
|
||||
## Advanced Data Protection
|
||||
|
||||
ADP is currently unsupported and need to be disabled
|
||||
Advanced Data Protection is supported.
|
||||
|
||||
On iPhone, Settings `>` Apple Account `>` iCloud `>` 'Access iCloud Data on the Web'
|
||||
must be ON, and 'Advanced Data Protection' OFF.
|
||||
must be ON.
|
||||
|
||||
If ADP is enabled on your account, rclone requests PCS cookies after 2FA.
|
||||
Apple may require approval on a trusted device before those cookies are issued.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Missing PCS cookies from the request
|
||||
### PCS cookie errors with ADP
|
||||
|
||||
This means you have Advanced Data Protection (ADP) turned on. This is not supported
|
||||
at the moment. If you want to use rclone you will have to turn it off. See above
|
||||
for how to turn it off.
|
||||
If you see `Missing PCS cookies from the request` or a `requestPCS:` error,
|
||||
the ADP approval flow did not complete successfully.
|
||||
|
||||
You will need to clear the `cookies` and the `trust_token` fields in the config.
|
||||
Or you can delete the remote config and start again.
|
||||
Check that 'Access iCloud Data on the Web' is enabled and approve any prompt
|
||||
on your trusted device.
|
||||
|
||||
You should then run `rclone reconnect remote:`.
|
||||
Then run `rclone reconnect remote:`.
|
||||
|
||||
Note that changing the ADP setting may not take effect immediately - you may
|
||||
need to wait a few hours or a day before you can get rclone to work - keep
|
||||
clearing the config entry and running `rclone reconnect remote:` until rclone
|
||||
functions properly.
|
||||
If the remote still has stale auth state, clear the `cookies` and
|
||||
`trust_token` fields in the config, or delete and recreate the remote.
|
||||
|
||||
<!-- autogenerated options start - DO NOT EDIT - instead edit fs.RegInfo in backend/iclouddrive/iclouddrive.go and run make backenddocs to verify --> <!-- markdownlint-disable-line line-length -->
|
||||
### Standard options
|
||||
|
||||
Reference in New Issue
Block a user