Architecture

Decoupled, Vendor-Agnostic Design

FetchStream is architected with a strict separation of concerns. Unlike conventional video downloaders that lock you into proprietary servers or commercial subscriptions, FetchStream separates the client browser extension from the execution backend.

Client Extension

Inspects network media streams (HLS/M3U8), embedded web PDFs, disk images, and direct media files. Downloads directly to disk with parallel workers, or offloads to a private runner.

Remote Runner (Optional)

An independent HTTP server that accepts stream download jobs, assembles them with FFmpeg, and can upload to configured storage. The browser still communicates with the runner.

Privacy Note: FetchStream does not provide a centralized download-tracking service. Local workflows run in the extension; Server Upload sends data to the runner you configure.

Execution Modes: Local vs Server

Every detected media item gives you complete control over how to process it:

Action Button Where It Executes Bandwidth Used Setup Required
Download Your computer's browser engine (OPFS) Local download only No Setup
Local Upload Browser downloads file & streams directly to cloud host API Local download + Local upload No Setup
Server Upload Remote cloud runner (VPS / Docker / CI/CD Runner) 0MB Local Data Configure Server URL in Settings
API Reference

The Universal Remote Runner Protocol

To build or host a compatible backend, your server must implement two straightforward REST endpoints: POST /api/server-relay and GET /api/server-status.

Developer Disclaimer

The code blocks, JSON schemas, and architectural blueprints provided in this documentation and in our repositories are mere examples meant to demonstrate intended functionality. They are provided as-is and may be prone to error depending on your production environment, dependencies, or configuration. Please review, test, and adapt them thoroughly before deploying to production.

POST /api/server-relay (or /api/tasks)

Dispatches a stream or media file download and upload job to the server.

Request Headers
Content-Type: application/json
Authorization: Bearer <YOUR_API_KEY>
X-API-Key: <YOUR_API_KEY>
Request Body Schema (JSON)
{
  "jobId": "job_1725900000000_abc12",
  "mediaUrl": "https://cdn.site.com/hls.m3u8",
  "fileName": "lecture_video.mp4",
  "type": "hls",
  "format": "mp4",
  "service": "gofile.io",
  "threads": 8,
  "headers": {
    "User-Agent": "Mozilla/5.0 ...",
    "Referer": "https://streaming-site.com/"
  },
  "credentials": {
    "caption": "My Optional Description",
    "gofile": {
      "token": "YOUR_GOFILE_API_TOKEN",
      "folderId": "OPTIONAL_FOLDER_ID"
    },
    "telegram": {
      "chatId": "-100123456789",
      "botToken": "1234:ABCDEF",
      "caption": "My Optional Description"
    },
    "s3_compatible": {
      "endpoint": "https://s3.us-east-1.amazonaws.com",
      "bucket": "my-bucket",
      "region": "us-east-1",
      "accessKeyId": "YOUR_KEY",
      "secretAccessKey": "YOUR_SECRET",
      "prefix": "optional/folder/path/"
    },
    "pixeldrain.com": {
      "apiKey": "YOUR_PIXELDRAIN_API_KEY"
    }
  },
  "useFallbackProxy": true
}
Response (200 OK)
{
  "jobId": "job_1725900000000_abc12",
  "status": "QUEUED",
  "message": "Job received and queued successfully."
}

GET /api/server-status?jobId={jobId} (or /api/tasks/{jobId})

Returns real-time download and upload progress for the specified job.

Response States

The status field must be one of: "QUEUED", "RUNNING", "COMPLETED", or "FAILED".

Response: In Progress
{
  "jobId": "job_1725900000000_abc12",
  "status": "RUNNING",
  "stage": "DOWNLOADING",
  "progress": 55.4,
  "speed": "12.8 MB/s",
  "completedSegments": 554,
  "totalSegments": 1000,
  "fileName": "lecture_video.mp4",
  "fileSize": 157286400
}
Response: Completed
{
  "jobId": "job_1725900000000_abc12",
  "status": "COMPLETED",
  "stage": "COMPLETED",
  "progress": 100,
  "speed": "Complete",
  "fileName": "lecture_video.mp4",
  "fileSize": 284190120,
  "url": "https://gofile.io/d/abc123",
  "links": [
    {
      "service": "gofile.io",
      "name": "GoFile",
      "url": "https://gofile.io/d/abc123",
      "directUrl": "https://srv.gofile.io/download/web/abc123/video.mp4"
    }
  ]
}

POST /api/server-callback Runner Webhook

Internal webhook called by runner scripts (server_uploader.py) to stream fine-grained download progress, conversion status, and resulting upload links to the edge/server store.

{
  "jobId": "job_1725900000000_abc12",
  "status": "RUNNING",
  "stage": "UPLOADING",
  "progress": 82.5,
  "speed": "Telegram • 45.2/60.0MB (75%)",
  "fileName": "lecture_video.mp4"
}

POST /api/server-cancel (or /api/tasks/{jobId}/cancel)
Request Body Schema (JSON)
{
  "jobId": "job_1725900000000_abc12"
}
Response (200 OK)
{
  "ok": true,
  "jobId": "job_1725900000000_abc12",
  "status": "CANCELLED"
}

POST /api/server-pause and /api/server-resume

Optional endpoints. If your custom backend supports pausing and resuming active streams, implementing these endpoints will allow the Extension UI's Pause/Resume buttons to control the server jobs. If unimplemented, the extension will gracefully notify the user that the server lacks pause capabilities.

Request Body Schema (JSON)
{
  "jobId": "job_1725900000000_abc12"
}

GET /health Liveness Check

Returns real-time worker load, version, and auth status of the runner.

{
  "status": "healthy",
  "service": "FetchStream Remote Runner",
  "version": "1.0.0",
  "active_jobs": 0,
  "total_jobs": 12,
  "auth_enabled": true
}

GET /api/stream-proxy?url={target_segment_url} Edge Proxy

Streams media playlists and segments through Edge API workers. When an origin CDN blocks datacenter IP ranges (returning HTTP 403 Forbidden to CI/CD or VPS runners), the runner automatically retries chunk downloads via this edge proxy using residential/edge IP routing.


Authentication & Security

To prevent unauthorized users from abusing your server bandwidth, you can set an API_KEY environment variable on your backend.

In the FetchStream extension, open Settings -> Server Upload and paste your secret key into Server API Key / Token (Optional). The extension automatically transmits your key in the Authorization: Bearer <key> header.

Stream Session Headers & Cookie Forwarding

Many authorized streaming platforms authenticate video segments (.ts, .m4s, .m3u8) using session tokens, signed query parameters, HTTP cookies (Cookie header), or authorization bearer tokens (Authorization header).

Privacy & Security Disclosure: When you initiate a Server Upload, FetchStream forwards the captured video URL along with relevant request headers (including active session cookies, authorization headers, User-Agent, and Referer) to your configured Remote Runner. This is strictly required so that the remote FFmpeg process can authenticate and fetch segments without encountering HTTP 403 Forbidden errors.

Security Recommendation: Session headers and cookies can represent your authenticated state on streaming websites. Configure Remote Runner URLs only for infrastructure you own, operate, or have explicitly vetted, and do not use unknown public runners.

Disclaimer: Reference Implementations Only

The Docker and Serverless backend blueprints below are reference implementations. Review, test, and secure them for your deployment. Operators are responsible for runner security, configuration, maintenance, and lawful use.

Self-Hosted

Example Blueprint A: Standalone Docker / VPS Server

Suitable for a Linux VPS, Raspberry Pi, home server, or local PC. The example includes FFmpeg and Python 3.11 for server-side stream processing.

📁 Reference Source Code & Dockerfile: server-examples/docker/

1. Launch with Docker Compose (1 Command)
git clone https://github.com/Abhishek-banal/fetchstream.git
cd fetchstream/server-examples/docker

docker compose up -d --build
Standalone Container Deployments: For deployment to managed container platforms, refer to the Standalone Deployment Guide in the README to properly configure the repository structure and Dockerfile paths.
2. Verify Server Health
curl http://localhost:8000/health
3. Connect to the Extension
  1. Open the FetchStream extension and click the Settings (⚙️) icon.
  2. In Remote Server / Cloud Runner URL, enter http://localhost:8000 (or your VPS domain, e.g. https://downloader.yourdomain.com).
  3. Click outside the panel to save. Server Uploads will now run on your private server.
open-source

Example Blueprint B: Serverless Edge and CI/CD Pipelines

Do not own a server? You can run the cloud downloader using an Edge Worker or CI/CD pipeline with temporary runner VMs.

📁 Reference Blueprints & Workflows: server-examples/serverless/

  1. Create Runner Repository: Create a private repository on your CI/CD provider and copy the blueprints from server-examples/serverless/ (workflows and functions) into your repository.
  2. Enable Pipeline Permissions: In your private runner repository, ensure your CI/CD pipelines have read and write permissions to execute jobs.
  3. Generate Provider Token: Generate a Personal Access Token or API Token in your CI/CD provider with pipeline execution scopes.
  4. Deploy to an Edge Worker: Deploy the functions/api scripts to an Edge provider. For example, using Edge CDN Workers connected to your private runner repo:
    • Build output directory: public
    • Environment variables:
      • CI_TOKEN: Your copied CI/CD token
      • CI_REPO: your-username/your-runner-repo-name
  5. Connect Extension: Copy your Edge Worker URL (e.g. https://my-fetchstream-runner.pages.dev) and paste it into the FetchStream extension settings under Remote Server URL.
DIY

Option 3: Building a Custom Backend

Because FetchStream communicates via a standard JSON contract, you can implement a custom backend in any programming language using minimal code.

Minimal Python (FastAPI) Sample
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
import uuid

app = FastAPI()
app.add_middleware(CORSMiddleware, allow_origins=["*"], allow_methods=["*"], allow_headers=["*"])

jobs = {}

@app.post("/api/server-relay")
async def dispatch_task(data: dict):
    job_id = data.get("jobId") or str(uuid.uuid4())
    jobs[job_id] = {"status": "RUNNING", "progress": 0, "url": data.get("mediaUrl")}
    return {"jobId": job_id, "status": "QUEUED"}

@app.get("/api/server-status")
async def task_status(jobId: str):
    return jobs.get(jobId, {"status": "FAILED", "error": "Job not found"})

Supported Cloud Upload Targets

FetchStream includes upload adapters for several third-party storage hosts and generic object storage endpoints:

GoFile.io
File hosting with provider-managed limits
Buzzheavier.com
High-speed temporary hosting up to 50 GB
Pixeldrain.com
Fast cloud offloading up to 10 GB
FFast
Multi-chunk streaming up to 5 GB
Storage.to
Direct API cloud hosting up to 5 GB
Catbox / Litterbox
Permanent or temporary hosting; provider limits apply
Generic S3 Storage
S3-compatible Storage, Edge Object Storage, MinIO, Backblaze B2
Native Cloud Provider SigV4 storage into HF Buckets
Telegram Bot & Channel
Bot uploads; limits depend on Telegram and the configured runner
Multi-Destination & Telegram Relay
Parallel upload to S3 Object Storage with automated Telegram video previews & links
Provider Sizing & Capability Matrix
Target Provider Local Upload Cap Server Upload Cap Auth / Credentials Features & Retention
GoFile.io Provider-defined Provider-defined None (Optional API Token) High-speed streaming & direct download links
Buzzheavier.com 50 GB 50 GB None Ultra-fast temporary cloud transfer
Pixeldrain.com 10 GB 10 GB None (Optional API Key) Direct file streaming with fast delivery
FFast 5 GB 5 GB None Parallel chunked streaming pipeline
Storage.to 5 GB 5 GB None Instant direct download hosting
Catbox / Litterbox 200MB 200MB None (Optional Userhash) Retention depends on the selected service
Telegram 50MB (Bot API) Telegram and runner limits Bot Token + Chat ID Native streaming video cards in your private channel
Generic S3 / R2 5 TB (Multi-part) 5 TB Access Key + Secret Key + Bucket Private cloud bucket (Cloud Provider, Edge Object Storage, MinIO, B2)
S3 + Telegram Relay Composite Composite Bucket Keys + Telegram Credentials Stores asset in S3 and relays playable preview to Telegram
Object Storage

S3-Compatible Storage & CORS Configuration

FetchStream supports direct client-side uploads to any S3-compliant object storage bucket (such as Edge Object Storage, S3-compatible Object Storage, Backblaze B2, or MinIO) using authentic Cloud Provider Signature Version 4 (SigV4) generated directly in your browser.

Mandatory Browser Requirement: Bucket CORS Policy

Because transfers execute directly from the browser runtime to your storage bucket endpoint via HTTP PUT requests, your storage provider must permit cross-origin requests. Without a valid CORS policy, browsers will abort uploads with a Cross-Origin Request Blocked (CORS) error.

Copy and paste this standard CORS JSON policy into your bucket's CORS configuration:

[
  {
    "AllowedOrigins": ["*"],
    "AllowedMethods": ["GET", "PUT", "HEAD"],
    "AllowedHeaders": ["*"],
    "ExposeHeaders": ["ETag"]
  }
]
Provider Parameter Specifications
Provider Endpoint URL Format Region Credentials & Where to Configure CORS
Edge Object Storage
Recommended (10GB Free)
https://<ACCOUNT_ID>.r2.Edge Providerstorage.com auto Create an R2 API Token with Object Read & Write permissions. Configure CORS in Bucket Settings ➔ CORS Policy.
Major Cloud Providers (S3-compatible Storage) https://s3.<region>.amazonaws.com
(e.g. https://s3.us-east-1.amazonaws.com)
Target Region (e.g. us-east-1) IAM User Access Key & Secret Key with s3:PutObject and s3:GetObject permissions. Configure CORS in Bucket ➔ Permissions ➔ CORS.
Backblaze B2 https://s3.<region>.backblazeb2.com Target Region (e.g. us-west-004) Application Key ID and Application Key with Read and Write access. Configure CORS via B2 CLI or B2 web console.
Self-Hosted MinIO https://minio.yourdomain.com (or http://IP:9000) us-east-1 (default) MinIO Access Key & Secret Key. Set CORS via MinIO Console or mc anonymous set.
Optional Path Prefix

You can organize uploads into a subfolder by entering a prefix (e.g. recordings/ or archives/) in Settings. FetchStream will automatically place all uploads under that bucket key prefix.

Pre-Signed URL Generation

Upon upload completion, FetchStream automatically generates a SigV4 signed download URL valid for 7 days so you can immediately retrieve and share the file without making your bucket publicly accessible.

Integration

Telegram Bot & Channel Setup Guide

FetchStream can upload downloaded streams, videos, and files directly to a private Telegram Channel, Group, or Chat using Telegram's official Bot API. Follow this step-by-step guide to create your bot and find your numeric Channel ID.

Step 1: Create a Bot via @BotFather

1. In Telegram, search for the official @BotFather and click Start.
2. Send the command: /newbot
3. Choose a friendly name for your bot (e.g. My Storage Bot).
4. Choose a unique username ending in bot (e.g. my_media_vault_bot).
5. BotFather will provide your HTTP API Token (e.g. 7123456789:AAFn73hK...). Copy this token.

Step 2: Create a Private Channel & Add Bot as Admin

1. In Telegram, create a new Channel (e.g. My Cloud Archive) and set it to Private.
2. Open Channel Info ➔ Administrators ➔ Add Administrator.
3. Search for your bot username and select it.
4. Ensure the permission "Post Messages" is enabled so the bot can upload media. Click Done.

Step 3: Extract Your Numeric Chat ID

Telegram channel IDs are negative numbers prefixed with -100 (e.g. -1001234567890). To easily find yours:
1. Post any sample message (e.g. "test") in your newly created channel.
2. Forward that message to @JsonDumpBot or @userinfobot.
3. The helper bot will return the raw Telegram update JSON. Locate the forward_from_chat section:

"forward_from_chat": {
  "id": -1001234567890,
  "title": "My Cloud Archive",
  "type": "channel"
}

4. Copy the entire id number, including the -100 prefix: -1001234567890.

Step 4: Configure in FetchStream

1. Open the FetchStream extension and click Settings (⚙️).
2. Under Telegram Bot Settings, paste your Bot Token and Chat ID.
3. Click outside to save. Whenever you upload to Telegram, your bot will post the files directly into your private channel.

Large Files
Telegram Upload Limits and Optional Runner Support

Telegram Bot API limits and upload methods depend on Telegram's current rules and the configured backend. Server Upload can pass a job to a runner that implements additional Telegram upload methods.

Optional runner-specific Telegram client

Some runner deployments may include a Telegram client that supports methods beyond the Bot API. Refer to that runner's documentation for supported credentials, limits, and transfer methods.

Custom credentials: If your selected runner requires Telegram application credentials, configure them only in that runner's documented environment.
2. Local Bot API Server (--local)

If running the official Telegram Bot API server locally on port 8081, FetchStream auto-detects http://127.0.0.1:8081 and utilizes Telegram's local filesystem upload method (bypassing HTTP payload limits).

Video metadata: Runner support for duration, dimensions, and thumbnails depends on the selected implementation.
Legal & Compliance

Third-Party Non-Affiliation & Terms Acceptance

FetchStream is an open-source, client-side utility and offloading bridge. Please review the following terms regarding third-party cloud hosting providers:

  • Independent Services & Non-Affiliation: FetchStream is an independent open-source project and is not affiliated with, endorsed by, sponsored by, or associated with GoFile.io, Buzzheavier.com, Pixeldrain.com, Catbox.moe, Telegram, Major Cloud Providers, Edge Provider, Backblaze, or any other third-party hosting service mentioned.
  • No Content Storage or File Control: FetchStream does not operate, manage, host, cache, or maintain any file storage servers. All files uploaded using FetchStream are transmitted directly from your client machine (Local Upload) or your configured private remote runner (Server Upload) directly to the target third-party service.
  • Terms of Service Acceptance: By uploading any file to a third-party host, you agree to comply with that specific provider's Terms of Service, Acceptable Use Policy, and Privacy Policy. It is your sole responsibility to ensure that you hold the legal right to upload, store, and distribute any media or files.
  • File Retention & Availability: File retention windows, bandwidth limits, geographic access restrictions, and account requirements are determined solely by each third-party provider and may change at any time without notice. FetchStream makes no warranties or guarantees regarding file availability, download speed, or persistence.
  • User Liability: Users assume full legal liability and responsibility for all content downloaded, processed, or uploaded using FetchStream.