Download Engine Architecture

Technical architecture details of FlowPick's download engine โ€” segment concurrent download, AES-128 decryption, three-tier write strategy, memory safety management, and progress tracking.

FlowPick's download engine is the core of the entire system, responsible for downloading streaming segments, decrypting, merging them, and writing to disk. This document is for developers and advanced users who want to deeply understand the internal mechanisms.


Architecture Overview

Download engine consists of three core modules, with data flowing through each module in pipeline fashion:

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                   Download Engine                 โ”‚
โ”‚                                                  โ”‚
โ”‚  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”  โ”‚
โ”‚  โ”‚ Segment   โ”‚โ†’โ”‚ Segment   โ”‚โ†’โ”‚ File Writing    โ”‚  โ”‚
โ”‚  โ”‚ Download  โ”‚  โ”‚ Process  โ”‚  โ”‚ Module          โ”‚  โ”‚
โ”‚  โ”‚ Module    โ”‚  โ”‚ Module   โ”‚  โ”‚                 โ”‚  โ”‚
โ”‚  โ”‚          โ”‚  โ”‚          โ”‚  โ”‚                 โ”‚  โ”‚
โ”‚  โ”‚ ยท Concur- โ”‚  โ”‚ ยท Decryptโ”‚  โ”‚ ยท FSA Streamingโ”‚  โ”‚
โ”‚  โ”‚   rency  โ”‚  โ”‚ ยท Concat โ”‚  โ”‚ ยท StreamSaver   โ”‚  โ”‚
โ”‚  โ”‚ ยท Retry  โ”‚  โ”‚ ยท Remux  โ”‚  โ”‚ ยท Blob Fallback โ”‚  โ”‚
โ”‚  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜  โ”‚
โ”‚                                                  โ”‚
โ”‚  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚
โ”‚  โ”‚           Memory Safety Manager              โ”‚ โ”‚
โ”‚  โ”‚  ยท Size Estimation ยท Threshold Check        โ”‚ โ”‚
โ”‚  โ”‚  ยท Strategy Selection                       โ”‚ โ”‚
โ”‚  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

Full Data Flow

Complete chain from user clicking download to file written to disk:

User clicks download
    โ”‚
    โ–ผ
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”     โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”     โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ 1. Manifest   โ”‚ โ”€โ”€โ†’ โ”‚ 2. Segment    โ”‚ โ”€โ”€โ†’ โ”‚ 3. Segment    โ”‚
โ”‚    Parsing    โ”‚     โ”‚    Download   โ”‚     โ”‚    Processing โ”‚
โ”‚              โ”‚     โ”‚               โ”‚     โ”‚               โ”‚
โ”‚ ยท Download   โ”‚     โ”‚ ยท Worker Pool โ”‚     โ”‚ ยท AES Decrypt โ”‚
โ”‚   M3U8       โ”‚     โ”‚ ยท Concurrency โ”‚     โ”‚ ยท TS Concat   โ”‚
โ”‚ ยท Parse seg- โ”‚     โ”‚ ยท Exponential โ”‚     โ”‚ ยท FFmpeg Remuxโ”‚
โ”‚   ment list  โ”‚     โ”‚   backoff     โ”‚     โ”‚ ยท A/V Merge   โ”‚
โ”‚ ยท Extract    โ”‚     โ”‚ ยท Progress    โ”‚     โ”‚               โ”‚
โ”‚   key        โ”‚     โ”‚   reporting   โ”‚     โ”‚               โ”‚
โ”‚ ยท Estimate   โ”‚     โ”‚               โ”‚     โ”‚               โ”‚
โ”‚   size       โ”‚     โ”‚               โ”‚     โ”‚               โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜     โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜     โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                                                  โ”‚
                                                  โ–ผ
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”     โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”     โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ 6. Completion โ”‚ โ†โ”€โ”€ โ”‚ 5. Cleanup   โ”‚ โ†โ”€โ”€ โ”‚ 4. File Write โ”‚
โ”‚ Notification โ”‚     โ”‚ & Recycling   โ”‚     โ”‚               โ”‚
โ”‚              โ”‚     โ”‚               โ”‚     โ”‚               โ”‚
โ”‚ ยท Desktop    โ”‚     โ”‚ ยท Release     โ”‚     โ”‚ ยท FSA Stream  โ”‚
โ”‚   notifi-    โ”‚     โ”‚   memory      โ”‚     โ”‚ ยท StreamSaver โ”‚
โ”‚   cation    โ”‚     โ”‚ ยท Clean temp  โ”‚     โ”‚ ยท Blob Fall-  โ”‚
โ”‚ ยท Copy path  โ”‚     โ”‚   files       โ”‚     โ”‚   back        โ”‚
โ”‚ ยท Queue adv- โ”‚     โ”‚ ยท Reset state โ”‚     โ”‚               โ”‚
โ”‚   ance       โ”‚     โ”‚               โ”‚     โ”‚               โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜     โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜     โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
For specific implementation of manifest parsing phase (Master Playlist vs Media Playlist, encryption detection), see Video Sniffing โ€” HLS Streams. For FFmpeg remuxing details in segment processing, see Format Conversion. For engine's position in overall system, see Project Architecture.

Engine Lifecycle

A complete run of download engine goes through following state transitions:

  [Idle]
    โ”‚ User triggers download
    โ–ผ
  [Initializing] โ”€โ”€โ”€ Load WASM, check API availability, select write strategy
    โ”‚
    โ–ผ
  [Manifest Parsing] โ”€โ”€ Download and parse M3U8/MPD, extract segment list and keys
    โ”‚
    โ–ผ
  [Segment Download] โ”€โ”€ Worker Pool concurrent download, real-time progress reporting
    โ”‚
    โ–ผ
  [Segment Processing] โ”€โ”€ Decrypt โ†’ Concat/Remux โ†’ Write to disk
    โ”‚
    โ”œโ”€โ”€ Success โ†’ [Cleanup] โ†’ [Complete] โ†’ [Idle]
    โ”‚
    โ”œโ”€โ”€ Cancel โ†’ [Cleanup] โ†’ [Idle]
    โ”‚
    โ””โ”€โ”€ Failure โ†’ [Cleanup] โ†’ [Error] โ†’ [Idle]

Each state transition triggers corresponding lifecycle hooks; UI layer updates interface state by listening to these hooks. For UI-engine interaction details, see Project Architecture โ€” Data Flow.


Segment Download Module

Concurrent Downloader

Segment download uses Worker Pool pattern to implement concurrency control:

const downloadSegmentsConcurrently = async (
  segments: SegmentInfo[],
  onSegmentDownloaded: (buffer: ArrayBuffer, index: number) => void
) => {
  const concurrency = Math.min(config.concurrency, 8)
  let nextIndex = 0

  const worker = async () => {
    while (nextIndex < segments.length) {
      const index = nextIndex++
      const buffer = await downloadWithRetry(segments[index]!)
      onSegmentDownloaded(buffer, index)
    }
  }

  const workers = Array.from(
    { length: Math.min(concurrency, segments.length) },
    () => worker()
  )
  await Promise.all(workers)
}

Design Points:

  • Use shared nextIndex counter for task assignment, avoiding load imbalance from pre-splitting segments
  • Worker count doesn't exceed total segment count, avoiding idle workers
  • Each worker runs independently; single segment failure doesn't affect other workers

Worker Pool Deep Dive

Core advantage of Worker Pool pattern is dynamic load balancing. Unlike pre-splitting segment array into N equal parts, shared counter ensures:

Pre-splitting (Not recommended):
Worker 1: [Segment 0-49]   โ† If these segments are larger, Worker 1 becomes bottleneck
Worker 2: [Segment 50-99]  โ† May finish early, then wait idle

Shared Counter (FlowPick's approach):
Worker 1: Segment 0 โ†’ Segment 3 โ†’ Segment 5 โ†’ ...
Worker 2: Segment 1 โ†’ Segment 4 โ†’ Segment 7 โ†’ ...
Worker 3: Segment 2 โ†’ Segment 6 โ†’ Segment 8 โ†’ ...

Each worker immediately grabs next unassigned segment after completing current one, until all segments processed. This pattern naturally adapts to scenarios with uneven segment sizes (e.g., HLS streams' first/last segments usually smaller, middle segments larger).

Concurrency controlled by config.concurrency, capped at 8. Users can adjust default value in Configuration Reference or modify in real-time on download interface. For queue scheduling in batch download scenarios, see Batch Download โ€” Concurrency Selection Guide.

Async Generator Pattern

For very large files (segment count > 500), engine uses AsyncGenerator pattern to avoid loading all segment data into memory at once:

async function* segmentGenerator(
  segments: SegmentInfo[],
  signal?: AbortSignal
): AsyncGenerator<ArrayBuffer> {
  for (const segment of segments) {
    if (signal?.aborted) break
    const buffer = await downloadWithRetry(segment)
    yield buffer
  }
}

Difference from Array Pattern:

DimensionArray PatternGenerator Pattern
Memory peakAll segments in memory simultaneouslyOnly currently processing segment in memory
Applicable scenarioSegment count < 500Segment count > 500
Progress trackingKnown total, precise percentageKnown total, precise percentage
Cancel responseMust wait for current batch to completeImmediate response

Engine automatically selects pattern based on segment count without user intervention. For actual scenarios of very large file downloads, see Live Replay Saving.

Retry Mechanism

When segment download fails, uses exponential backoff retry:

const downloadWithRetry = async (
  segment: SegmentInfo,
  maxRetries = 3
): Promise<ArrayBuffer> => {
  let lastError: unknown
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      return await downloadAndDecryptSegment(segment)
    } catch (e) {
      lastError = e
      if (attempt < maxRetries - 1) {
        // Exponential backoff: 400ms, 800ms, 1600ms
        await new Promise(r => setTimeout(r, Math.pow(2, attempt) * 400))
      }
    }
  }
  throw lastError
}

Retry Strategy:

Attempt #DelayCumulative Wait
1st failure400ms400ms
2nd failure800ms1200ms
3rd failure1600ms2800ms

Cases that don't trigger retry:

  • HTTP 4xx client errors (403, 404, etc.) โ€” retry meaningless
  • CORS errors โ€” policy issue, retry won't change result
  • Unsupported encryption error โ€” cannot be resolved via retry
Retry mechanism works together with error classification table in Online Tools โ€” Error Handling. HTTP 4xx and CORS errors thrown directly, displayed as corresponding user prompts by upper UI layer. For troubleshooting mid-download failures, see Common Issues Troubleshooting โ€” Mid-Download Failure.

Error Classification

class FetchError extends Error {
  constructor(
    message: string,
    public status: number,
    public url: string
  ) {
    super(message)
    this.name = 'FetchError'
  }
}

Errors classified and handled by HTTP status code:

Status CodeError TypeUser Prompt
403Auth/authorization failure"Access denied, please check if URL is valid"
404Resource not found"Segment not found, stream may have expired"
502/503/504Server error"Server temporarily unavailable, please try again later"
CORSCross-origin restriction"Cross-origin request blocked, recommend using extension version"
NetworkNetwork interruption"Network connection failed, please check network"
When encountering CORS errors, recommend installing browser extension, which can bypass same-origin policy via webRequest permissions. See Common Issues Troubleshooting for more troubleshooting methods. For CORS technical principles and limitations, see Known Limitations โ€” Browser Limitations.

Segment Processing Module

AES-128 Decryption

For encrypted HLS streams, FlowPick uses Web Crypto API to decrypt in browser:

const decryptAES128 = async (
  encryptedData: ArrayBuffer,
  key: ArrayBuffer,
  iv?: Uint8Array
): Promise<ArrayBuffer> => {
  const keyBytes = await crypto.subtle.importKey(
    'raw', key,
    { name: 'AES-CBC' },
    false,
    ['decrypt']
  )

  const ivBuffer = iv ? new Uint8Array(iv) : new Uint8Array(16)

  const decrypted = await crypto.subtle.decrypt(
    { name: 'AES-CBC', iv: ivBuffer },
    keyBytes,
    encryptedData
  )

  return decrypted
}

Decryption Flow:

  1. Extract key URI and IV from M3U8's #EXT-X-KEY tag
  2. Download key file (usually 16-byte binary file)
  3. Decrypt each segment using AES-128-CBC mode
  4. If IV not specified, use segment sequence number as IV (HLS spec default behavior)

Performance Considerations:

  • Web Crypto API uses hardware acceleration, decryption speed usually not bottleneck
  • Each segment decrypted independently, can parallelize with download
  • Key only needs to be downloaded once, cached in memory
All decryption operations completed locally in browser; keys and segment data not uploaded to any server. For more details on encryption detection, see Video Sniffing โ€” Encrypted Streams. For privacy protection strategy, see Privacy & Security. For unsupported encryption types (Widevine, PlayReady etc.), see Known Limitations โ€” DRM Protected Content.

TS Segment Concatenation

For TS output format, segments directly concatenated in binary:

// Pseudocode
const merged = new Uint8Array(totalSize)
let offset = 0
for (const segment of segments) {
  merged.set(new Uint8Array(segment), offset)
  offset += segment.byteLength
}

This approach has zero CPU overhead, speed limited only by memory copy speed.

FFmpeg WASM Remuxing

For MP4 output format, use FFmpeg WASM for container conversion. See Format Conversion document for details.

If you only need TS format, selecting TS output can completely bypass FFmpeg, significantly improving merge speed. TS files can be played directly with VLC, PotPlayer etc. For output format selection recommendations, see Format Conversion โ€” Output Format Selection.

DASH Stream Special Processing

DASH streams differ significantly from HLS streams in segment processing:

HLS stream processing:
  Segment 0 โ†’ Segment 1 โ†’ Segment 2 โ†’ ... โ†’ Concat โ†’ Output

DASH stream processing (audio-video separated):
  Init segment โ†’ Video Seg 0 โ†’ Video Seg 1 โ†’ ... โ”€โ”
                                              โ”œโ†’ FFmpeg Merge โ†’ Output
  Init segment โ†’ Audio Seg 0 โ†’ Audio Seg 1 โ†’ ... โ”€โ”˜

DASH's FMP4 (Fragmented MP4) format requires special handling:

  1. Init segment (ftyp + moov box) must be placed at beginning of file
  2. Media segments (moof + mdat box) appended sequentially
  3. When audio-video separated, need to download video and audio tracks separately, finally merge via FFmpeg
For DASH stream manifest parsing details, see Video Sniffing โ€” DASH Streams. For specific FMP4 reassembly implementation, see Format Conversion โ€” FMP4 Reassembly. For known issues with DASH audio-video separation, see Known Limitations โ€” Separated Audio-Video Streams.

File Writing Module

Writing module selects strategy by priority to ensure working across as many browsers as possible.

Strategy 1: File System Access API (Optimal)

async function createFSAStream(
  filename: string,
  dirHandle?: FileSystemDirectoryHandle | null
): Promise<WritableStream<Uint8Array>> {
  let handle: FileSystemFileHandle

  if (dirHandle) {
    // Already have directory permission, directly create file
    handle = await dirHandle.getFileHandle(filename, { create: true })
  } else {
    // Popup for user to choose save location
    handle = await window.showSaveFilePicker!({
      suggestedName: filename,
      types: [{
        description: 'Video',
        accept: { 'video/mp4': ['.mp4'] }
      }]
    })
  }

  const writable = await handle.createWritable()

  return new WritableStream<Uint8Array>({
    async write(chunk) { await writable.write(chunk) },
    async close() { await writable.close() },
    async abort(reason) { await writable.abort(reason) }
  }, {
    highWaterMark: 16 * 1024 * 1024 // 16MB buffer
  })
}

Advantages:

  • Streaming write, constant memory usage (only 16MB buffer)
  • Supports arbitrary file sizes
  • After directory persistence, no repeated popups needed

Limitations:

  • Only supported on Chrome 86+ and Edge 86+
  • Requires user gesture trigger (showDirectoryPicker must be called within user click event)

Strategy 2: StreamSaver.js (Alternative)

async function createStreamSaverStream(
  filename: string,
  fileSize?: number
): Promise<WritableStream<Uint8Array>> {
  const ss = await getStreamSaver()
  const fileStream = ss.createWriteStream(filename, { size: fileSize })
  return fileStream
}

Advantages:

  • Streaming write, low memory usage
  • Better compatibility than FSA API

Limitations:

  • Requires Service Worker support
  • Requires mitm.html and streamsaver-sw.js properly deployed
  • Some enterprise network environments may block Service Worker

Strategy 3: Blob Fallback

// Pseudocode
const blob = new Blob([mergedData], { type: 'video/mp4' })
const url = URL.createObjectURL(blob)
const a = document.createElement('a')
a.href = url
a.download = filename
a.click()
URL.revokeObjectURL(url)

Limitations:

  • Entire file loaded into memory
  • Hard limit 1.5GB (MEMORY_CONFIG.maxBlobSize)
  • Console warning when exceeding 800MB (MEMORY_CONFIG.warnBlobSize)

Strategy Comparison Summary

DimensionFSA APIStreamSaver.jsBlob
Memory usage16MB (constant)Low (streaming)Equal to file size
File size limitUnlimitedUnlimited1.5GB
Browser requirementChrome/Edge 86+Service Worker supportAll browsers
Directory persistenceSupportedNot supportedNot supported
Download experienceBestGoodAverage
For browser support status of these three strategies, see Browser Compatibility โ€” Feature Fallback Strategies. For write strategy usage in online tools, see Online Tools โ€” Write Strategy Priority. For Blob mode size limits and solutions, see Known Limitations โ€” File Size Limits.

Memory Safety Management

Size Estimation

Before starting download, engine samples to estimate total file size:

async function estimateSegmentsSize(
  segments: ArrayBuffer[] | AsyncGenerator<ArrayBuffer>,
  totalCount: number
): Promise<{ estimatedSize: number; isEstimate: boolean }> {
  // Sample count: min(5, max(1, totalCount * 0.1))
  const sampleCount = Math.min(5, Math.max(1, Math.floor(totalCount * 0.1)))

  // Download first few segments and calculate average size
  let sampledTotal = 0
  for (let i = 0; i < sampleCount; i++) {
    sampledTotal += sampleSegment.byteLength
  }

  const avgSample = sampledTotal / sampleCount
  return {
    estimatedSize: Math.round(avgSample * totalCount),
    isEstimate: true
  }
}

Sampling Strategy Explanation:

  • Sample count takes min(5, max(1, total segments ร— 10%)), ensuring small streams sample at least 1, large streams at most 5
  • Sampling result marked isEstimate: true, UI layer displays "~XX MB" instead of exact value based on this
  • For HLS streams, if Master Playlist declares BANDWIDTH attribute, engine prioritizes using that value as estimation reference

Strategy Selection Logic

Estimated size < 800MB  โ†’ Any strategy acceptable
Estimated size 800MB-1.5GB โ†’ Prioritize streaming write, Blob mode shows warning
Estimated size > 1.5GB โ†’ Force streaming write, Blob mode refused

Memory Configuration Constants

const MEMORY_CONFIG = {
  maxBlobSize: 1500 * 1024 * 1024,    // 1.5GB hard limit
  warnBlobSize: 800 * 1024 * 1024,    // 800MB warning threshold
}

Runtime Memory Monitoring

In addition to pre-download estimation, engine continuously monitors memory pressure during runtime:

const checkMemoryPressure = (): 'normal' | 'warning' | 'critical' => {
  // Check current total allocated segment buffer size
  const allocatedMB = allocatedBuffers.reduce((sum, buf) => sum + buf.byteLength, 0) / 1048576

  if (allocatedMB > 1200) return 'critical'
  if (allocatedMB > 600) return 'warning'
  return 'normal'
}

When memory pressure reaches critical level, engine will:

  • Pause new segment download requests
  • Prioritize writing already downloaded segments to disk
  • Release buffers for segments already written
Memory safety is one of core design considerations for FlowPick's download engine. For very large files (e.g., 20GB+ live replays), engine forces streaming write strategy to ensure browser doesn't crash due to insufficient memory. For actual scenarios of large file downloads, see Live Replay Saving. For troubleshooting large file download failures, see Common Issues Troubleshooting โ€” Large File Download Failure.

Progress Tracking

Speed Calculation

Engine uses sliding window algorithm to calculate real-time download speed:

class SpeedCalculator {
  private samples: Array<{ bytes: number; timestamp: number }> = []
  private readonly WINDOW_SIZE = 5000 // 5 second window

  record(bytesDownloaded: number): void {
    this.samples.push({ bytes: bytesDownloaded, timestamp: Date.now() })
    // Remove samples older than window
    const cutoff = Date.now() - this.WINDOW_SIZE
    this.samples = this.samples.filter(s => s.timestamp >= cutoff)
  }

  getSpeed(): number {
    if (this.samples.length < 2) return 0
    
    const oldest = this.samples[0]!
    const newest = this.samples[this.samples.length - 1]!
    
    const timeDiff = (newest.timestamp - oldest.timestamp) / 1000 // seconds
    const byteDiff = newest.bytes - oldest.bytes
    
    return timeDiff > 0 ? byteDiff / timeDiff : 0
  }
}

Algorithm Characteristics:

  • Sliding window: Only considers data from recent 5 seconds, avoids historical data interference
  • Responsive: Speed update reflects current network conditions in near real-time
  • Smooth: Averages out momentary fluctuations, providing stable display value

Display Format:

Speed RangeDisplay UnitExample
< 1 MB/sKB/s512 KB/s
1-1024 MB/sMB/s5.2 MB/s
> 1024 MB/sGB/s1.3 GB/s

ETA Calculation

Estimated Time of Arrival (ETA) calculated based on current speed and remaining work:

function calculateETA(
  downloadedBytes: number,
  totalBytes: number,
  currentSpeed: number
): string {
  if (currentSpeed <= 0 || downloadedBytes >= totalBytes) return '--'
  
  const remainingBytes = totalBytes - downloadedBytes
  const remainingSeconds = remainingBytes / currentSpeed
  
  return formatDuration(remainingSeconds)
}

function formatDuration(seconds: number): string {
  if (seconds < 60) return `${Math.round(seconds)}s`
  if (seconds < 3600) return `${Math.floor(seconds / 60)}m ${Math.round(seconds % 60)}s`
  return `${Math.floor(seconds / 3600)}h ${Math.floor((seconds % 3600) / 60)}m`
}

Calculation Logic:

  • Based on sliding window average speed, not instantaneous speed
  • Dynamically adjusts to network condition changes
  • Shows "--" when speed is zero or download complete

Progress Event Structure

Engine emits structured progress events for UI layer consumption:

interface DownloadProgress {
  // Basic info
  stage: 'parsing' | 'downloading' | 'processing' | 'writing' | 'completed'
  
  // Overall progress
  percent: number                    // 0-100
  downloadedBytes: number            // Total downloaded bytes
  totalBytes: number                // Estimated or exact total
  
  // Speed info
  speed: number                     // Bytes per second
  eta: number                       // Estimated remaining seconds
  
  // Segment-level detail
  completedSegments: number         // Completed segment count
  totalSegments: number             // Total segment count
  
  // Error info (if any)
  error?: {
    type: string
    message: string
    segmentIndex?: number
    recoverable: boolean
  }
}

Event Throttling:

To avoid excessive UI updates, progress events throttled to maximum 10 times per second:

const THROTTLE_INTERVAL = 100 // 100ms minimum interval

let lastEmitTime = 0
function emitProgress(progress: DownloadProgress) {
  const now = Date.now()
  if (now - lastEmitTime < THROTTLE_INTERVAL) return
  
  lastEmitTime = now
  eventEmitter.emit('progress', progress)
}

Cancellation Mechanism

AbortController Integration

Download engine fully supports cancellation via standard AbortController API:

async function startDownload(
  url: string,
  options: DownloadOptions,
  signal: AbortSignal
): Promise<void> {
  // Pass signal to each module
  await parseManifest(url, { signal })
  await downloadSegments(segments, { signal })
  await processSegments(segments, { signal })
  await writeFile(outputFile, { signal })
}

Cancellation Propagation Chain:

User clicks cancel button
    โ”‚
    โ–ผ
UI layer calls controller.abort()
    โ”‚
    โ–ผ
1. Set signal.aborted = true
    โ”‚
    โ”œโ”€โ†’ 2. Stop accepting new download tasks
    โ”‚      โ””โ”€โ”€ Worker Pool stops grabbing new segments from queue
    โ”‚
    โ”œโ”€โ†’ 3. Interrupt ongoing fetch requests
    โ”‚      โ””โ”€โ”€ fetch() throws AbortError, caught by retry mechanism
    โ”‚
    โ”œโ”€โ†’ 4. Stop Worker Pool
    โ”‚      โ””โ”€โ”€ All Workers detect signal.aborted, exit loop
    โ”‚
    โ”œโ”€โ†’ 5. Clean up FFmpeg virtual filesystem
    โ”‚      โ””โ”€โ”€ Delete temp files: filelist.txt, segment_*.ts, output.*
    โ”‚
    โ”œโ”€โ†’ 6. Release allocated memory buffers
    โ”‚      โ””โ”€โ”€ Set all ArrayBuffer references to null, wait for GC
    โ”‚
    โ”œโ”€โ†’ 7. Discard incomplete file
    โ”‚      โ””โ”€โ”€ FSA mode: call writable.abort()
    โ”‚      โ””โ”€โ”€ StreamSaver mode: call writable.abort()
    โ”‚      โ””โ”€โ”€ Blob mode: Don't trigger download, release Blob directly
    โ”‚
    โ””โ”€โ†’ 8. Reset engine state
           โ””โ”€โ”€ Clear progress data, speed stats, error messages

Cancellation Propagation Mechanism

AbortSignal propagates layer-by-layer through entire download chain:

UI Layer AbortController
    โ”‚
    โ”œโ”€โ”€โ†’ Segment Download Worker Pool (interrupt fetch)
    โ”œโ”€โ”€โ†’ Segment Processing Pipeline (skip remaining segments)
    โ”œโ”€โ”€โ†’ FFmpeg WASM (terminate remuxing)
    โ””โ”€โ”€โ†’ File Write Stream (abort write)

Each layer independently checks signal.aborted, ensuring cancel operation responds quickly. Even when cancelling during FFmpeg remuxing, engine waits for current FFmpeg operation to complete one atomic step before terminating, avoiding virtual filesystem corruption.

Cancel operation especially important in Batch Download scenarios โ€” users can cancel a specific download in queue without affecting other ongoing tasks. For queue behavior after cancellation, see Batch Download โ€” Queue Management.

Performance Benchmarks

Following data tested under Chrome 126 + 100Mbps network environment, for reference only:

ScenarioFile SizeConcurrencyDownload TimeMerge TimeTotal Time
Short video (TS output)~50MB (30 segments)4~8s<1s~9s
Short video (MP4 output)~50MB (30 segments)4~8s~3s~11s
Long video (TS output)~500MB (200 segments)6~45s~2s~47s
Long video (MP4 output)~500MB (200 segments)6~45s~15s~60s
Very large file (TS output)~2GB (800 segments)8~3min~8s~3min
DASH audio-video separate~300MB video + ~30MB audio4~30s~12s~42s

Influencing Factors:

FactorImpact LevelDescription
CDN speedHighUpper limit of segment download speed
Concurrent threadsMedium4-6 threads usually optimal range
Output formatMediumTS output skips FFmerge, faster merging
SharedArrayBufferMediumMulti-threaded FFmpeg 40-60% faster than single-threaded
Encrypted streamsLowWeb Crypto API hardware accelerated, minimal decryption overhead

Relationship Between Concurrency and Performance

Concurrency 1: โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ Slow, low bandwidth utilization
Concurrency 2: โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ Faster, basically usable
Concurrency 4: โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ Fast, recommended default
Concurrency 6: โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ Very fast, near optimal
Concurrency 8: โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ About same as 6, diminishing returns
Concurrency 12+: โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ May trigger CDN throttling,ๅ่€Œ slower
Selecting TS output format can skip FFmpeg merge phase, saving 10-20% total time for very large files. TS files can be played directly with VLC, PotPlayer, IINA etc. For more troubleshooting methods on slow download speeds, see Common Issues Troubleshooting โ€” Slow Download Speed.

Error Boundaries & Exception Propagation

Download engine adopts layered error handling architecture, ensuring exceptions at each level are properly caught and converted to user-friendly prompts:

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                   UI Layer                  โ”‚
โ”‚  ยท Display user prompts (Toast/popup)       โ”‚
โ”‚  ยท Update download status to "failed"       โ”‚
โ”‚  ยท Provide retry/feedback entry points      โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                   โ”‚ Catch all exceptions
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚              Engine Facade Layer             โ”‚
โ”‚  ยท Unified exception format (StreamMergeError) โ”‚
โ”‚  ยท Attach context info (URL, segment index, stage) โ”‚
โ”‚  ยท Decide whether retryable                 โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                   โ”‚
    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
    โ–ผ              โ–ผ              โ–ผ
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚Downloadโ”‚  โ”‚Process   โ”‚  โ”‚Write     โ”‚
โ”‚ Module โ”‚  โ”‚Module    โ”‚  โ”‚Module    โ”‚
โ”‚        โ”‚  โ”‚          โ”‚  โ”‚          โ”‚
โ”‚FetchErrโ”‚  โ”‚CryptoErr โ”‚  โ”‚WriteErr  โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

Error Type Mapping:

Underlying ErrorEngine Error TypeUser PromptRetryable
FetchError(403)AuthError"Access denied"No
FetchError(404)NotFoundError"Resource expired"No
FetchError(5xx)ServerError"Server error"Yes
TypeError: Failed to fetchNetworkError"Network connection failed"Yes
DOMException: AbortErrorCancelledErrorNo prompt (silent)โ€”
CryptoErrorDecryptError"Decryption failed"No
FFmpegErrorMergeError"Merge failed"No
QuotaExceededErrorStorageError"Insufficient storage space"No
For detailed troubleshooting steps for various errors, see Common Issues Troubleshooting. For engine's known limitations and edge cases, see Known Issues. For how to contribute error handling improvements to engine, see Contributing Guide.