# Mass migrating Data to Shade via Rclone ### Handling Data Migration and Transfers Efficiently This is a guide for migrating large amounts of data into or out of your shade drive using rclone, a powerful open-source tool which supports many different sources. ### Downloading rclone The Shade backend is now available natively as part of Rclone. You can download rclone from the [official rclone downloads page](https://rclone.org/downloads/). ### About rclone [rclone](https://rclone.org/) is a powerful command-line program to manage files on cloud storage. It's often described as "rsync for cloud storage" and supports over 70 different cloud storage providers. Common use cases for rclone include: * **Syncing and backing up** files between your local system and cloud storage * **Migrating data** from one cloud provider to another * **Automating workflows** with scheduled syncs and backups * **Managing large file transfers** with resume capability and bandwidth controls * **Batch operations** for copying, moving, or deleting files across cloud services With Shade's rclone backend, you can leverage all of rclone's powerful features to interact with your Shade drives directly from the command line. ### Configuration Here is an example of making a Shade configuration. First, [create a free account](https://app.shade.inc/) and choose a plan. You will need to log in and get the `API Key` and `Drive ID` for your account from the settings section of your account and created drive respectively. Now open a command prompt or terminal and navigate to where you downloaded that file, and run `./rclone config` \ \ (If you are on a mac you will need to tell your computer that this file can be executed, so run `chmod +x rclone` ) Then follow this interactive process: ``` e) Edit existing remote n) New remote d) Delete remote r) Rename remote c) Copy remote s) Set configuration password q) Quit config e/n/d/r/c/s/q> n Enter name for new remote. name> Shade Option Storage. Type of storage to configure. Choose a number from below, or type in your own value. (Scroll along the list and find the Shade option, as of 11/5/25 it is 52) [OTHER OPTIONS] xx / Shade FS \ (shade) [OTHER OPTIONS] Storage> xx Option drive_id. The ID of your drive, see this in the drive settings. Individual rclone configs must be made per drive. Enter a value. drive_id> [YOUR_ID] Option api_key. An API key for your account. Enter a value. api_key> [YOUR_API_KEY] Edit advanced config? y) Yes n) No (default) y/n> n Configuration complete. Options: - type: shade - drive_id: [YOUR_ID] - api_key: [YOUR_API_KEY] Keep this "Shade" remote? y) Yes this is OK (default) e) Edit this remote d) Delete this remote y/e/d> y ``` ### Example Workflow: Copying Files to Shade Once you've configured your Shade remote, you can easily copy files from your local filesystem to Shade. Here's a common workflow: #### Copying a Local Folder to Shade To copy a folder from your local system to your Shade drive: ```bash ./rclone copy /path/to/local/folder Shade:/destination/folder ``` For example, to copy your "Projects" folder to Shade: ```bash ./rclone copy ~/Projects Shade:/Projects ``` This command will: * Copy all files from your local Projects folder to Shade * Preserve the folder structure * Skip files that already exist and haven't changed * Show progress during the transfer #### Additional Useful Commands **Sync a folder** (makes destination identical to source, including deletions): ```bash ./rclone sync ~/Projects Shade:/Projects ``` **Check what would be copied** without actually copying: ```bash ./rclone copy ~/Projects Shade:/Projects --dry-run ``` **Copy with progress and statistics**: ```bash ./rclone copy ~/Projects Shade:/Projects --progress --stats 1s ``` **List files in your Shade drive**: ```bash ./rclone ls Shade:/ ``` ### Standard Options Here are the Standard options specific to shade (Shade FS). **Note:** For most users, you only need to configure the `drive_id` and `api_key`. The advanced options below are for specific use cases and can be left at their defaults. #### --shade-drive-id The ID of your drive, see this in the drive settings. Individual rclone configs must be made per drive. Properties: * Config: drive\_id * Env Var: RCLONE\_SHADE\_DRIVE\_ID * Type: string * Required: true #### --shade-api-key An API key for your account. You can find this under Settings > API Keys Properties: * Config: api\_key * Env Var: RCLONE\_SHADE\_API\_KEY * Type: string * Required: true ### Advanced Options Here are the Advanced options specific to shade (Shade FS). #### --shade-endpoint Endpoint for the service. Leave blank normally. Properties: * Config: endpoint * Env Var: RCLONE\_SHADE\_ENDPOINT * Type: string * Required: false #### --shade-chunk-size Chunk size to use for uploading. Any files larger than this will be uploaded in chunks of this size. Note that this is stored in memory per transfer, so increasing it will increase memory usage. Minimum is 5MB, maximum is 5GB. Properties: * Config: chunk\_size * Env Var: RCLONE\_SHADE\_CHUNK\_SIZE * Type: SizeSuffix * Default: 64Mi #### --shade-encoding The encoding for the backend. See the [encoding section in the overview](https://github.com/shade-labs/docs/blob/main/overview/README.md#encoding) for more info. Properties: * Config: encoding * Env Var: RCLONE\_SHADE\_ENCODING * Type: Encoding * Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot #### --shade-description Description of the remote. Properties: * Config: description * Env Var: RCLONE\_SHADE\_DESCRIPTION * Type: string * Required: false ### API Routes Used by Rclone Integration The following routes are used by the rclone integration to interface directly with the Shade filesystem. If you want to use these routes directly in your own applications, you'll need to first obtain a user token by calling the appropriate route from the 'Users' section in the 'Using the API' section of our documentation. Once you have the user token, pass it as a bearer token in your requests instead of your API key. **Note:** The rclone backend handles authentication automatically using your API key - you only need to worry about user tokens if you're building custom integrations outside of rclone. ## GET /{drive-id}/fs/listdir > List directory contents for a given drive path ```json {"openapi":"3.1.1","info":{"title":"Shade Rclone Routes","version":"0.0.0"},"paths":{"/{drive-id}/fs/listdir":{"get":{"tags":["drives"],"parameters":[{"name":"drive","in":"path","required":true,"schema":{"type":"string"}},{"name":"path","in":"query","required":true,"schema":{"type":"string"}},{"name":"authorization","in":"header","required":true,"schema":{"type":"string"}}],"operationId":"getByDriveFsListdir","summary":"List directory contents for a given drive path"}}}} ``` ## GET /{drive-id}/fs/attr > Get file or directory attributes/metadata ```json {"openapi":"3.1.1","info":{"title":"Shade Rclone Routes","version":"0.0.0"},"paths":{"/{drive-id}/fs/attr":{"get":{"tags":["drives"],"parameters":[{"name":"drive","in":"path","required":true,"schema":{"type":"string"}},{"name":"path","in":"query","required":true,"schema":{"type":"string"}},{"name":"authorization","in":"header","required":true,"schema":{"type":"string"}}],"operationId":"getByDriveFsAttr","summary":"Get file or directory attributes/metadata"}}}} ``` ## GET /{drive-id}/fs/download > Download a file with optional content disposition ```json {"openapi":"3.1.1","info":{"title":"Shade Rclone Routes","version":"0.0.0"},"paths":{"/{drive-id}/fs/download":{"get":{"tags":["drives"],"parameters":[{"name":"drive","in":"path","required":true,"schema":{"type":"string"}},{"name":"path","in":"query","required":true,"schema":{"type":"string"}},{"name":"disposition_type","in":"query","required":true,"schema":{"default":"attachment","type":"string"}},{"name":"authorization","in":"header","required":true,"schema":{"type":"string"}}],"operationId":"getByDriveFsDownload","summary":"Download a file with optional content disposition"}}}} ``` ## POST /{drive-id}/fs/mkdir > Create a directory at the specified path ```json {"openapi":"3.1.1","info":{"title":"Shade Rclone Routes","version":"0.0.0"},"paths":{"/{drive-id}/fs/mkdir":{"post":{"tags":["drives"],"parameters":[{"name":"drive","in":"path","required":true,"schema":{"type":"string"}},{"name":"path","in":"query","required":true,"schema":{"type":"string"}},{"name":"authorization","in":"header","required":true,"schema":{"type":"string"}}],"operationId":"postByDriveFsMkdir","summary":"Create a directory at the specified path"}}}} ``` ## POST /{drive-id}/fs/mkdir/batch > Create multiple directories in a single request ```json {"openapi":"3.1.1","info":{"title":"Shade Rclone Routes","version":"0.0.0"},"paths":{"/{drive-id}/fs/mkdir/batch":{"post":{"tags":["drives"],"parameters":[{"name":"drive","in":"path","required":true,"schema":{"type":"string"}},{"name":"authorization","in":"header","required":true,"schema":{"type":"string"}}],"requestBody":{"content":{},"required":true},"operationId":"postByDriveFsMkdirBatch","summary":"Create multiple directories in a single request"}}}} ``` ## POST /{drive-id}/fs/copy-file > Copy a file from one path to another ```json {"openapi":"3.1.1","info":{"title":"Shade Rclone Routes","version":"0.0.0"},"paths":{"/{drive-id}/fs/copy-file":{"post":{"tags":["drives"],"parameters":[{"name":"drive","in":"path","required":true,"schema":{"type":"string"}},{"name":"from","in":"query","required":true,"schema":{"type":"string"}},{"name":"to","in":"query","required":true,"schema":{"type":"string"}},{"name":"authorization","in":"header","required":true,"schema":{"type":"string"}}],"operationId":"postByDriveFsCopy-file","summary":"Copy a file from one path to another"}}}} ``` ## POST /{drive-id}/fs/move > Move or rename a file or directory ```json {"openapi":"3.1.1","info":{"title":"Shade Rclone Routes","version":"0.0.0"},"paths":{"/{drive-id}/fs/move":{"post":{"tags":["drives"],"parameters":[{"name":"drive","in":"path","required":true,"schema":{"type":"string"}},{"name":"from","in":"query","required":true,"schema":{"type":"string"}},{"name":"to","in":"query","required":true,"schema":{"type":"string"}},{"name":"authorization","in":"header","required":true,"schema":{"type":"string"}}],"operationId":"postByDriveFsMove","summary":"Move or rename a file or directory"}}}} ``` ## POST /{drive-id}/fs/delete > Delete a file or directory at the specified path ```json {"openapi":"3.1.1","info":{"title":"Shade Rclone Routes","version":"0.0.0"},"paths":{"/{drive-id}/fs/delete":{"post":{"tags":["drives"],"parameters":[{"name":"drive","in":"path","required":true,"schema":{"type":"string"}},{"name":"path","in":"query","required":true,"schema":{"type":"string"}},{"name":"authorization","in":"header","required":true,"schema":{"type":"string"}}],"operationId":"postByDriveFsDelete","summary":"Delete a file or directory at the specified path"}}}} ``` ## GET /{drive-id}/info/storage > Fetch overall storage usage and limits ```json {"openapi":"3.1.1","info":{"title":"Shade Rclone Routes","version":"0.0.0"},"paths":{"/{drive-id}/info/storage":{"get":{"tags":["drives"],"parameters":[{"name":"drive","in":"path","required":true,"schema":{"type":"string"}},{"name":"authorization","in":"header","required":true,"schema":{"type":"string"}}],"operationId":"getByDriveInfoStorage","summary":"Fetch overall storage usage and limits"}}}} ``` ## GET /{drive-id}/info/upload > Fetch upload configuration and constraints ```json {"openapi":"3.1.1","info":{"title":"Shade Rclone Routes","version":"0.0.0"},"paths":{"/{drive-id}/info/upload":{"get":{"tags":["drives"],"parameters":[{"name":"drive","in":"path","required":true,"schema":{"type":"string"}},{"name":"authorization","in":"header","required":true,"schema":{"type":"string"}}],"operationId":"getByDriveInfoUpload","summary":"Fetch upload configuration and constraints"}}}} ``` ## POST /{drive-id}/upload/multipart/part/{partNumber} > Upload a specific part of a multipart upload ```json {"openapi":"3.1.1","info":{"title":"Shade Rclone Routes","version":"0.0.0"},"paths":{"/{drive-id}/upload/multipart/part/{partNumber}":{"post":{"tags":["drives"],"parameters":[{"name":"drive","in":"path","required":true,"schema":{"type":"string"}},{"name":"partNumber","in":"path","required":true,"schema":{"type":"number"}},{"name":"token","in":"query","required":true,"schema":{"type":"string"}},{"name":"signAllEndpoints","in":"query","required":false,"schema":{"type":"string"}}],"operationId":"postByDriveUploadMultipartPartByPartNumber","summary":"Upload a specific part of a multipart upload"}}}} ``` ## POST /{drive-id}/upload/multipart > Initiate a multipart upload session ```json {"openapi":"3.1.1","info":{"title":"Shade Rclone Routes","version":"0.0.0"},"paths":{"/{drive-id}/upload/multipart":{"post":{"tags":["drives"],"parameters":[{"name":"drive","in":"path","required":true,"schema":{"type":"string"}},{"name":"authorization","in":"header","required":true,"schema":{"type":"string"}}],"requestBody":{"content":{},"required":true},"operationId":"postByDriveUploadMultipart","summary":"Initiate a multipart upload session"}}}} ``` ## POST /{drive-id}/upload/multipart/complete > Complete a multipart upload by finalizing parts ```json {"openapi":"3.1.1","info":{"title":"Shade Rclone Routes","version":"0.0.0"},"paths":{"/{drive-id}/upload/multipart/complete":{"post":{"tags":["drives"],"parameters":[{"name":"drive","in":"path","required":true,"schema":{"type":"string"}},{"name":"token","in":"query","required":true,"schema":{"type":"string"}},{"name":"authorization","in":"header","required":true,"schema":{"type":"string"}}],"requestBody":{"content":{},"required":true},"operationId":"postByDriveUploadMultipartComplete","summary":"Complete a multipart upload by finalizing parts"}}}} ``` --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://academy.shade.inc/guides/mass-migrating-data-to-shade-via-rclone.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.