For the complete documentation index, see llms.txt. This page is also available as Markdown.

Uploading to Shade

Choose the right way to get files into Shade from scripts, servers, pipelines, and your own apps, and know what happens after the upload.

There are several ways to get files into a Shade drive. Pick the one that matches where the files are coming from.

Use case
Best method
Why

Migrating an existing NAS, bucket, or cloud drive

Resumable, parallel, and checks what's already copied

A script, server, render farm, or media pipeline

Works with any S3 SDK or CLI, multipart included

Uploads inside your own app, with progress and maximum speed

The same multipart flow the Shade apps use

Backups from a Synology NAS

Scheduled, versioned backups

People working on their desktop

Drag and drop, or save straight into a mounted drive

Every method needs an account with edit access to the destination drive.

Upload from a script or server with the S3 API

Each drive is an S3 bucket named with its drive ID. Any S3 client works once you point it at https://s3.shade.inc, set a region, and turn on path-style addressing. Create an S3 key under Settings → Integrate → S3 API keys first.

Python (boto3)

import os

import boto3
from boto3.s3.transfer import TransferConfig
from botocore.config import Config

s3 = boto3.client(
    "s3",
    endpoint_url="https://s3.shade.inc",
    region_name="us-east-1",
    aws_access_key_id=os.environ["SHADE_S3_ACCESS_KEY_ID"],
    aws_secret_access_key=os.environ["SHADE_S3_SECRET_ACCESS_KEY"],
    config=Config(
        s3={"addressing_style": "path"},
        request_checksum_calculation="when_required",
        response_checksum_validation="when_required",
    ),
)

drive_id = os.environ["SHADE_DRIVE_ID"]

s3.upload_file(
    "renders/ep01_v3.mov",
    drive_id,
    "Deliverables/Ep 01/ep01_v3.mov",
    Config=TransferConfig(multipart_chunksize=64 * 1024 * 1024),
)

upload_file switches to multipart automatically for large files and uploads parts in parallel. The two checksum settings stop recent boto3 versions from sending extra checksum headers that S3-compatible services don't always accept.

AWS CLI

See Using Shade's S3 API for the CLI config, supported actions, and troubleshooting.

Upload from your own app with the file system API

If you're building an app on top of Shade and want progress reporting, pause and resume, and the fastest transfers, use the same multipart flow the Shade web and desktop apps use:

  1. Exchange your API key for a ShadeFS token with GET https://api.shade.inc/workspaces/drives/{drive_id}/shade-fs-token.

  2. Create the destination folders with POST https://fs.shade.inc/{drive_id}/fs/mkdir.

  3. Start a multipart upload with POST https://fs.shade.inc/{drive_id}/upload/multipart.

  4. For each part, get a presigned URL, then PUT the bytes to it and keep the ETag.

  5. Finish with POST https://fs.shade.inc/{drive_id}/upload/multipart/complete, or abort on failure.

Parts of 64 to 128 MB work well for most connections. Upload several parts at once for more speed. The full walkthrough with code is in Writing your own uploader.

ShadeFS tokens expire after one hour. For long uploads, check the token's exp claim before every request and fetch a new one when it's close to expiring.

Common upload workflows

Deliver renders from a render farm or encoder

Have the render job's final step call aws s3 cp or a small boto3 script with a dedicated S3 key. Upload to a fixed folder such as /Deliverables/<project>/, and let a Shade automation or webhook handle notifications and sharing from there.

Ingest from camera cards or a watch folder

Run aws s3 sync or rclone copy on a schedule against the watch folder. Both skip files that are already in the drive, so you can re-run them safely after an interrupted transfer.

Upload from a web app or internal tool

Never ship an API key to a browser or a client you don't control. Have your backend hold the key, fetch ShadeFS tokens, and run the multipart upload, or hand the client a short-lived ShadeFS token for a single drive instead of the key itself.

Accept files from people outside your workspace

You don't need code for this. Create a share link on a folder and turn on its Upload permission so recipients can add files to it. See Published Links.

What happens after an upload

When an upload completes, Shade indexes the file and starts background processing: previews, video proxies, transcription, and AI indexing for search and metadata. Each step finishes on its own schedule.

To act on new files, subscribe to webhooks:

Event
Fires when

asset.uploaded

The file is fully uploaded and indexed. resource.id is the asset ID.

preview.completed

Preview images are ready

proxy.completed

The video proxy is ready for playback

transcription.completed

The transcript is ready

From asset.uploaded you can set metadata, add the asset to a collection, or create a share link with the REST API. For a full example, see Integrating with Shade.

Troubleshooting

  • 403 Forbidden: the key's owner doesn't have edit access to the drive, or the key has expired.

  • Requests go to <drive-id>.s3.shade.inc, or signatures don't match: turn on path-style addressing and set a non-empty region.

  • Uploads fail partway through with the file system API: the ShadeFS token expired. Refresh it before each part.

  • A file doesn't show up in search yet: it's still processing. Wait for asset.uploaded.

Last updated

Was this helpful?