woody
/
legco_ai_assistant
1
1
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

revert: remove Phase 3 YouTube proxy — all 7 sub-phases 

Reverts commits 284028b through b4096d6. Phase 4 (System Audio Capture)
will replace the YouTube use case with a more versatile getDisplayMedia approach.

Removed: YouTube router, HLS proxy, YouTubeService, YouTubeInput,
YouTubeVideoPlayer, useYouTubeASR hook, all Phase 3 tests, hls.js dep,
YouTube config fields, YouTube README/plan sections.

Modified files restored to pre-Phase-3 state: LTTPage (no source toggle),
api.ts (no YouTube extract), types (no YouTube types), config.py (no
youtube fields), main.py (no YouTube router), requirements.txt (no yt-dlp),
.env.example (no YouTube vars), package.json (no hls.js).

Relevant Phase 2 code preserved: ws_asr.py (unchanged), useVideoASR,
VideoPlayer, VideoUpload, QueryInput, Full Transcript.
This commit is contained in:
Woody 2026-05-09 21:07:21 +08:00
parent b4096d6afc
commit b05c361fbd
30 changed files with 187 additions and 3690 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

406
.plans/phase3_youtube_proxy_plan.md
View File

@ -1,15 +1,15 @@
# Phase 3: YouTube Live Stream Proxy → ASR → RAG — Implementation Plan # Phase 3: YouTube Live Stream Proxy → ASR → RAG — Implementation Plan
**Created:** 2026-05-09 **Created:** 2026-05-09
**Updated:** 2026-05-09 (Phase 3 complete — all 7 sub-phases done) **Updated:** 2026-05-09 (user decisions incorporated)
**Status:** ✅ Complete **Status:** Planning
**Depends on:** Phase 1 (Complete), Phase 2 (Complete) **Depends on:** Phase 1 (Complete), Phase 2 (Complete)
--- ---
## 1. Overview ## 1. Overview
Phase 3 adds YouTube live stream (and VOD) playback as an alternative to file upload. User pastes a YouTube URL → backend extracts stream URLs via yt-dlp (separate video-only + audio-only for VODs; combined HLS for live) → backend proxies HLS manifests and .ts segments (zero re-encoding) → frontend plays video in `<video>` via hls.js, routes audio through hidden `<audio>` element → AudioContext.createMediaElementSource(audioElement) → existing ASR pipeline (WebSocket → DashScope) → transcript flows into QueryInput → Phase 1 RAG pipeline. Phase 3 adds YouTube live stream (and VOD) playback as an alternative to file upload. User pastes a YouTube URL → backend extracts separate video-only and audio-only HLS streams via yt-dlp → backend proxies HLS manifests and .ts segments (zero re-encoding) → frontend plays video in `<video>` via hls.js, routes audio through hidden `<audio>` element → AudioContext.createMediaElementSource(audioElement) → existing ASR pipeline (WebSocket → DashScope) → transcript flows into QueryInput → Phase 1 RAG pipeline.
**Same code works identically for live streams and VODs.** **Same code works identically for live streams and VODs.**
@ -20,16 +20,14 @@ YouTube's official iframe player does not expose the audio track to Web Audio AP
### Audio Routing ### Audio Routing
``` ```
YouTube HLS stream (combined video+audio for live; separate tracks for VOD) YouTube HLS audio-only stream
→ hls.js loads into <video> (muted) and hidden <audio> element → hls.js loads into hidden <audio> element
→ AudioContext.createMediaElementSource(audioElement) → AudioContext.createMediaElementSource(audioElement)
→ ScriptProcessorNode (Float32 PCM) → ScriptProcessorNode (Float32 PCM)
→ WebSocket → FastAPI → DashScope realtime ASR → WebSocket → FastAPI → DashScope realtime ASR
→ transcript → QueryInput → transcript → QueryInput
``` ```
Note: For VODs, separate video-only and audio-only tracks are used. For live streams, YouTube provides combined formats only — the same HLS manifest URL is used for both elements; hls.js demuxes them independently.
### Integration With Existing Pipeline ### Integration With Existing Pipeline
This phase reuses the existing ASR infrastructure entirely: This phase reuses the existing ASR infrastructure entirely:
@ -57,65 +55,51 @@ This phase reuses the existing ASR infrastructure entirely:
## 3. Sub-Phases ## 3. Sub-Phases
### Phase 3.1 — Configuration & Infrastructure Setup ✅ Complete ### Phase 3.1 — Configuration & Infrastructure Setup (0.5 day)
Add config fields, install dependencies, create skeletons, register router. Add config fields, install dependencies, create skeletons, register router.
**Test:** `test_phase3_config.py` (11 tests) **Test:** `test_phase3_config.py`
**Tasks:** **Tasks:**
| # | Task | File | Status | | # | Task | File |
|---|------|------|--------| |---|------|------|
| 3.1.1 | Add config fields: `youtube_proxy_enabled`, `yt_dlp_timeout`, `yt_dlp_cache_ttl` | `core/config.py` | Done | | 3.1.1 | Add config fields: `youtube_proxy_enabled`, `yt_dlp_timeout`, `yt_dlp_cache_ttl` | `core/config.py` |
| 3.1.2 | Update `.env.example` | `.env.example` | Done | | 3.1.2 | Update `.env.example` | `.env.example` |
| 3.1.3 | Add deps: `yt-dlp>=2024.0.0` to `requirements.txt`, `hls.js@^1.5.0` to `package.json` | `requirements.txt`, `package.json` | Done | | 3.1.3 | Add deps: `yt-dlp>=2024.0.0` to `requirements.txt`, `hls.js@^1.5.0` to `package.json` | `requirements.txt`, `package.json` |
| 3.1.4 | Create `models/youtube.py` — `YouTubeExtractRequest`, `YouTubeStreamResponse`, `StreamFormat` | `models/youtube.py` | Done | | 3.1.4 | Create `models/youtube.py` — `YouTubeExtractRequest`, `YouTubeStreamResponse`, `StreamFormat` | `models/youtube.py` |
| 3.1.5 | Create `services/youtube_service.py` stub | `services/youtube_service.py` | Done | | 3.1.5 | Create `services/youtube_service.py` stub | `services/youtube_service.py` |
| 3.1.6 | Create `services/hls_proxy.py` stub | `services/hls_proxy.py` | Done | | 3.1.6 | Create `services/hls_proxy.py` stub | `services/hls_proxy.py` |
| 3.1.7 | Create `routers/youtube.py` stub: `POST /youtube/extract`, `GET /youtube/proxy/{stream_type}/{path}` | `routers/youtube.py` | Done | | 3.1.7 | Create `routers/youtube.py` stub: `POST /youtube/extract`, `GET /youtube/proxy/{stream_type}/{path}` | `routers/youtube.py` |
| 3.1.8 | Register router in `main.py` | `main.py` | Done | | 3.1.8 | Register router in `main.py` | `main.py` |
| 3.1.9 | Write and pass `test_phase3_config.py` | `app/test/` | Done (11/11 pass) | | 3.1.9 | Write and pass `test_phase3_config.py` | `app/test/` |
--- ---
### Phase 3.2 — YouTube URL Extraction Backend ✅ Complete ### Phase 3.2 — YouTube URL Extraction Backend (0.5 day)
yt-dlp wrapper service that extracts stream URLs and formats. Returns proxy-wrapped URLs pointing back to our HLS proxy. yt-dlp wrapper service that extracts separate video-only and audio-only HLS URLs. Returns proxy-wrapped URLs pointing back to our HLS proxy.
**Test:** `test_phase3_youtube_extract.py` (18 tests) **Test:** `test_phase3_youtube_extract.py`
**Acceptance Criteria:** **Acceptance Criteria:**
- `POST /api/v1/youtube/extract` accepts `{"url": "https://www.youtube.com/watch?v=..."}` - `POST /api/v1/youtube/extract` accepts `{"url": "https://www.youtube.com/watch?v=..."}`
- Returns `{ video_id, title, is_live, video_proxy_url, audio_proxy_url, thumbnail_url, formats, error }` - Returns `{ video_id, title, is_live, video_proxy_url, audio_proxy_url, thumbnail_url }`
- VODs: extracts separate video-only + audio-only tracks, selects best ≤480p + highest-bitrate audio - VODs: extracts ~2–10 formats, returns best video+audio pair
- Live streams: extracts combined HLS formats, uses same URL for video and audio (hls.js demuxes) - Live streams: uses `ios` client for HLS, returns current live edge
- Upcoming/scheduled streams: returns `is_upcoming: true` with no proxy URLs - Upcoming/scheduled streams: returns `is_upcoming: true` with scheduled start time
- Invalid/private URLs: returns 200 with error field populated (yt-dlp exception caught) - Invalid/private URLs: returns clear error
- URL expiration: in-memory cache with TTL (5 min for live, 30 min for VOD) - URL expiration: caches extraction result with TTL (5 min for live, 30 min for VOD)
- Service singleton: `@lru_cache` on `_get_youtube_service()` for cache persistence across requests
**Implementation Discoveries:**
- **No iOS client needed** — default yt-dlp works for both VOD (separate tracks) and live (combined HLS)
- **Live streams use combined formats** — all live formats include both video+audio; same HLS URL serves both `<video>` and `<audio>` elements
- **Format selection** (`_pick_best_video`): prefers ≤480p with HLS first, then falls back to ascending height + HLS preference
- **Error response pattern**: extraction errors return HTTP 200 with `error` field (not 4xx); the API call itself succeeds but YouTube returned an error
- **Proxy URL construction** (`_build_proxy_url`): URL-encodes upstream URL into `/api/v1/youtube/proxy/manifest.m3u8?url=<encoded>`
**Real-URL Verification:**
```
VOD: https://www.youtube.com/watch?v=5bF3tkO5jAA → 24 formats, separate video+audio ✓
Live: https://www.youtube.com/watch?v=fN9uYWCjQaw → 6 combined formats, same URL ✓
```
**Tasks:** **Tasks:**
| # | Task | File | Status | | # | Task | File |
|---|------|------|--------| |---|------|------|
| 3.2.1 | Write tests first | `app/test/test_phase3_youtube_extract.py` | Done | | 3.2.1 | Write tests first | `app/test/test_phase3_youtube_extract.py` |
| 3.2.2 | Implement `YouTubeService.extract_streams()` — yt-dlp wrapper with format selection | `services/youtube_service.py` | Done | | 3.2.2 | Implement `YouTubeService.extract_streams()` — yt-dlp wrapper with format selection | `services/youtube_service.py` |
| 3.2.3 | Implement `YouTubeService._select_best_formats()` + `_pick_best_video()` — separate video/audio from format list, prefer ≤480p, combined fallback | `services/youtube_service.py` | Done | | 3.2.3 | Implement `YouTubeService._select_best_formats()` — separate video/audio from format list, prefer ≤480p | `services/youtube_service.py` |
| 3.2.4 | Implement format URL caching with TTL (live 5 min, VOD 30 min) | `services/youtube_service.py` | Done | | 3.2.4 | Implement format URL caching with TTL | `services/youtube_service.py` |
| 3.2.5 | Implement `POST /api/v1/youtube/extract` route with response model + error handling | `routers/youtube.py` | Done | | 3.2.5 | Implement `POST /api/v1/youtube/extract` route | `routers/youtube.py` |
| 3.2.6 | Run tests → pass → verified with real URLs | — | Done (82/82 pass) | | 3.2.6 | Run tests → pass → commit | — |
--- ---
@ -147,165 +131,108 @@ Proxy service that rewrites HLS manifests and proxies .ts segments. StreamingRes
--- ---
### Phase 3.4 — Frontend: YouTube Input + Video Player ✅ Complete ### Phase 3.4 — Frontend: YouTube Input + Video Player (1 day)
URL input component and hls.js-based video player. Two media elements: visible `<video muted>` and hidden `<audio>` (for Web Audio API routing). URL input component and hls.js-based video player. Two hidden media elements: visible `<video>` (video-only, muted) and hidden `<audio>` (audio-only, for Web Audio API routing).
**Tests:** `test_phase3_YouTubeInput.test.tsx` (7 tests), `test_phase3_YouTubeVideoPlayer.test.tsx` (9 tests) **Tests:** `test_phase3_YouTubeInput.test.tsx`, `test_phase3_YouTubeVideoPlayer.test.tsx`
**Acceptance Criteria:** **Acceptance Criteria:**
- `YouTubeInput` accepts URL, validates format (youtube.com/watch, youtu.be, /live/, /shorts/), shows loading/error states - `YouTubeInput` accepts URL, validates format, shows loading/error states
- `YouTubeVideoPlayer` uses `forwardRef<HTMLVideoElement>` (same pattern as `VideoPlayer`) - `YouTubeVideoPlayer` uses `forwardRef<HTMLVideoElement>` (same pattern as `VideoPlayer`)
- Video HLS loaded via hls.js into `<video muted>` element, quality capped ≤480p via `capLevelsTo480()` - Video HLS loaded via hls.js into `<video muted>` element at 360p–480p (auto-best ≤ 480p)
- Audio HLS loaded via hls.js into hidden `<audio>` element, exposed via `onAudioReady` callback - Audio HLS loaded via hls.js into hidden `<audio>` element
- Audio element exposes ref for parent to connect to AudioContext
- Thumbnail displayed as placeholder until user presses play; video element replaces it on play - Thumbnail displayed as placeholder until user presses play; video element replaces it on play
- Video does NOT auto-play on load (waits for manual user play) - Video does NOT auto-play on load (waits for manual user play)
- Loading spinner, error overlay, "LIVE" badge for live streams - Loading spinner, error overlay, "LIVE" badge for live streams
- hls.js: dynamic `import('hls.js')` with fallback if not supported (SSR-safe) - **HLS error recovery**: on `hls.js` fatal error → re-extract stream URL → retry up to 3× → show "Service unavailable" on exhaustion
- CrossOrigin="anonymous" on both elements (required for AudioContext graph) - CrossOrigin="anonymous" on both elements (required for AudioContext graph)
- No quality selector (low resolution only, sufficient for reference video) - No quality selector (low resolution only, sufficient for reference video)
**Implementation Notes:**
- hls.js installed as npm dependency (was already in package.json from Phase 3.1)
- YouTubeVideoPlayer uses `useImperativeHandle`-style callback ref for audio element exposure
- Quality capping: on `MANIFEST_PARSED`, sets `hls.autoLevelCapping` to highest level with height ≤ 480
- Thumbnail overlay: absolute-positioned `<img>` that hides on video `onPlay` event
**Tasks:** **Tasks:**
| # | Task | File | Status | | # | Task | File |
|---|------|------|--------| |---|------|------|
| 3.4.1 | Write tests first | `src/test/test_phase3_YouTubeInput.test.tsx`, `src/test/test_phase3_YouTubeVideoPlayer.test.tsx` | Done | | 3.4.1 | Write tests first | `src/test/test_phase3_YouTubeInput.test.tsx`, `src/test/test_phase3_YouTubeVideoPlayer.test.tsx` |
| 3.4.2 | Add YouTube types to `types/index.ts` | `types/index.ts` | Done | | 3.4.2 | Add YouTube types to `types/index.ts` | `types/index.ts` |
| 3.4.3 | Add API functions to `lib/api.ts` | `lib/api.ts` | Done | | 3.4.3 | Add API functions to `lib/api.ts` | `lib/api.ts` |
| 3.4.4 | Add TanStack Query hooks to `lib/queries.tsx` | `lib/queries.tsx` | Done | | 3.4.4 | Add TanStack Query hooks to `lib/queries.tsx` | `lib/queries.tsx` |
| 3.4.5 | Create `components/YouTubeInput.tsx` — URL input, validation, loading/error states | `components/YouTubeInput.tsx` | Done | | 3.4.5 | Create `components/YouTubeInput.tsx` — URL input, validation, loading/error states | `components/YouTubeInput.tsx` |
| 3.4.6 | Create `components/YouTubeVideoPlayer.tsx` — hls.js dual-element player, forwardRef, onAudioReady | `components/YouTubeVideoPlayer.tsx` | Done | | 3.4.6 | Create `components/YouTubeVideoPlayer.tsx` — hls.js dual-element player, forwardRef | `components/YouTubeVideoPlayer.tsx` |
| 3.4.7 | Run tests → pass → commit | — | Done (16/16 pass) | | 3.4.7 | Run tests → pass → commit | — |
--- ---
### Phase 3.5 — Integration: YouTube → ASR Pipeline ✅ Complete ### Phase 3.5 — Integration: YouTube → ASR Pipeline (1 day)
Wire YouTube audio output into existing ASR pipeline. Creates `useYouTubeASR` hook (adapted from `useVideoASR`) and integrates YouTube components into `LTTPage` with a source toggle. Wire YouTube audio output into existing ASR pipeline. The key challenge: `useVideoASR` currently captures from `<video>` element; we need it to capture from the `<audio>` element loaded by hls.js.
**Tests:** `test_phase3_useYouTubeASR.test.ts` (11 tests), `test_phase3_LTTPage_integration.test.tsx` (7 tests) **Tests:** `test_phase3_useYouTubeASR.test.ts`, `test_phase3_LTTPage_integration.test.tsx`
**Acceptance Criteria:** **Acceptance Criteria:**
- `useYouTubeASR` hook: accepts `audioElement` + `videoElement`, sets up AudioContext graph on mount - `useYouTubeASR` hook: accepts `audioElement` ref, sets up AudioContext graph on mount
- AudioContext.createMediaElementSource(audioElement) → ScriptProcessorNode → WebSocket (same as useVideoASR, but audio source from `<audio>` element) - AudioContext.createMediaElementSource(audioElement) → ScriptProcessorNode → WebSocket
- Play/pause/ended events on `videoElement` (user controls video, audio follows)
- Auto-starts ASR on play, stops on pause/end (same lifecycle as `useVideoASR`) - Auto-starts ASR on play, stops on pause/end (same lifecycle as `useVideoASR`)
- Transcript flows into QueryInput (same `onFinalTranscript` + `partialTranscript` callbacks) - Transcript flows into QueryInput (same `onFinalTranscript` callback)
- QueryInput remains editable during streaming — user can type corrections while ASR appends (already worked, no changes needed) - QueryInput remains editable during streaming — user can type corrections while ASR appends
- "Full Transcript" button hidden when YouTube source is active - "Full Transcript" button hidden when YouTube source is active
- Source toggle: "Upload" / "YouTube" tabs at top of upper-left panel - Switching between "Upload" and "YouTube" sources clears previous state
- Switching between "Upload" and "YouTube" sources clears previous YouTube state
- Upload video state preserved when switching to YouTube and back
**Implementation Notes:**
- Both `useVideoASR` and `useYouTubeASR` initialized unconditionally at top of LTTPage
- Hooks gracefully handle null elements (AudioContext setup aborts early if element is null)
- Unified `asr` variable: `const asr = source === 'youtube' ? youtubeASR : uploadASR`
- Source toggle uses `Upload`/`Youtube` icons from lucide-react, blue active / gray inactive state
- `QueryInput.tsx` — zero changes needed (already supports `partialText` + `value` from any source)
- `YouTubeVideoPlayer` exposes audio element via `onAudioReady` callback → LTTPage wires to `useYouTubeASR`
**Tasks:** **Tasks:**
| # | Task | File | Status | | # | Task | File |
|---|------|------|--------| |---|------|------|
| 3.5.1 | Write tests first | `src/test/test_phase3_useYouTubeASR.test.ts` | Done (11 tests) | | 3.5.1 | Write tests first | `src/test/test_phase3_useYouTubeASR.test.ts` |
| 3.5.2 | Create `hooks/useYouTubeASR.ts` | `hooks/useYouTubeASR.ts` | Done | | 3.5.2 | Create `hooks/useYouTubeASR.ts` — adapted from `useVideoASR.ts`, targets `<audio>` element | `hooks/useYouTubeASR.ts` |
| 3.5.3 | Update `QueryInput.tsx` | `components/QueryInput.tsx` | Done (no-op: already works) | | 3.5.3 | Update `QueryInput.tsx` — accept transcript from either source | `components/QueryInput.tsx` |
| 3.5.4 | Update `LTTPage.tsx` — source toggle, wire components | `pages/LTTPage.tsx` | Done | | 3.5.4 | Update `LTTPage.tsx` — add source toggle (Upload / YouTube), wire YouTubeInput + YouTubeVideoPlayer | `pages/LTTPage.tsx` |
| 3.5.5 | Create LTTPage integration test | `src/test/test_phase3_LTTPage_integration.test.tsx` | Done (7 tests) | | 3.5.5 | Create `test_phase3_LTTPage_integration.test.tsx` | `src/test/` |
| 3.5.6 | Run tests → pass → commit | — | Done (189/189 pass) | | 3.5.6 | Run tests → pass → commit | — |
--- ---
### Phase 3.6 — Integration & Acceptance Testing ✅ Complete ### Phase 3.6 — Integration & Acceptance Testing (1 day)
**Tests:** `test_integration_phase3.py` (6 tests), `test_acceptance_phase3_youtube.py` (3 tests), `test_acceptance_phase3_live.py` (3 tests) **Tests:** `test_integration_phase3.py`, `test_acceptance_phase3_youtube.py`, `test_acceptance_phase3_live.py`
**Integration test** (`backend/app/test/test_integration_phase3.py`):
- `TestExtractAndProxyFlow` — full extract→proxy flow (VOD manifest, VOD segment, live manifest), cache hit verification
- `TestProxyAfterExtract` — upstream manifest unavailable after extract → 502
- `TestExtractDisabled` — extract returns 503 when `youtube_proxy_enabled=false`
- Mocked yt-dlp, real FastAPI TestClient, real HLSProxyService
**Acceptance test VOD** (`backend/app/test/acceptance/test_acceptance_phase3_youtube.py`):
- Real YouTube VOD extraction and proxy verification
- Manifest proxy → verify M3U8 structure and CORS
- Segment proxy → follow master→variant→segment chain, verify MPEG-TS data
- Skips gracefully if `YOUTUBE_TEST_VOD_URL` not set
**Acceptance test live** (`backend/app/test/acceptance/test_acceptance_phase3_live.py`):
- Real YouTube live extraction (is_live=True, combined formats)
- Live manifest proxy → verify no #EXT-X-ENDLIST
- Cache refresh verification (same video_id on re-extract)
- Skips gracefully if `YOUTUBE_TEST_LIVE_URL` not set or stream offline
**How to run acceptance tests:**
```bash
cd backend && YOUTUBE_TEST_VOD_URL="https://www.youtube.com/watch?v=5bF3tkO5jAA" \
YOUTUBE_TEST_LIVE_URL="https://www.youtube.com/watch?v=fN9uYWCjQaw" \
python -m pytest app/test/acceptance/test_acceptance_phase3_youtube.py \
app/test/acceptance/test_acceptance_phase3_live.py -v -m acceptance
```
**Tasks:** **Tasks:**
| # | Task | Status | | # | Task |
|---|------|--------| |---|------|
| 3.6.1 | Integration test (mocked yt-dlp, real httpx + HLSProxyService) | Done (6 tests) | | 3.6.1 | Implement integration test (mocked yt-dlp, real httpx proxy + hls.js) |
| 3.6.2 | Acceptance: real YouTube VOD → extract → proxy | Done (3 tests) | | 3.6.2 | Implement acceptance: real YouTube VOD → extract → proxy → play |
| 3.6.3 | Acceptance: real YouTube live → extract → proxy | Done (3 tests) | | 3.6.3 | Implement acceptance: real YouTube live stream → extract → proxy → play + ASR |
| 3.6.4 | Full regression run | Done (234 pass, 1 pre-existing config mismatch) | | 3.6.4 | Full regression run (Phase 1 + 2 + 3 tests) |
| 3.6.5 | Fix failures, commit | Done | | 3.6.5 | Fix failures, final commit |
--- ---
### Phase 3.7 — Polish & Deployment ✅ Complete ### Phase 3.7 — Polish & Deployment (0.5 day)
**Tasks:** | # | Task |
|---|------|
| # | Task | Status | | 3.7.1 | Handle PO token expiration for live streams (log warning, auto-re-extract on failure) |
|---|------|--------| | 3.7.2 | Update Dockerfile — ensure ffmpeg + yt-dlp available in container |
| 3.7.1 | PO token expiration handling | Done — `_is_po_token_error()` detection, cache invalidation, 2 tests | | 3.7.3 | Update `docker-compose.yml` — add any new volumes/env vars |
| 3.7.2 | Dockerfile — verify ffmpeg + yt-dlp | Done — ffmpeg already installed, yt-dlp in requirements.txt | | 3.7.4 | Verify production build (`npm run build`, `docker compose up -d --build`) |
| 3.7.3 | docker-compose.yml — verify volumes/env vars | Done — no new volumes needed (in-memory only), env vars in .env.example | | 3.7.5 | Update `README.md` — YouTube feature section |
| 3.7.4 | Verify production build | Done — `npm run build` succeeds (27s, hls chunk 523KB) | | 3.7.6 | Update `development_plan.md` — mark Phase 3 status |
| 3.7.5 | README.md — YouTube feature section | Done — added after Video Q&A section | | 3.7.7 | Final commit |
| 3.7.6 | development_plan.md — mark Phase 3 complete | Done — Phase 3 row added, status updated to "Phase 1-3 Complete" |
| 3.7.7 | Final commit | In progress |
**PO Token Handling:**
- `_is_po_token_error(msg)` helper detects YouTube bot-detection / PO token errors
- On detection: logs warning, invalidates URL cache (forces re-extract on next attempt)
- Graceful degradation: returns error field to frontend, which can retry
- Indicators: "sign in to confirm", "not a bot", "bot detection", "po token", "potoken"
**Docker/Infra:**
- Dockerfile already includes ffmpeg and all Python deps via requirements.txt
- docker-compose.yml unchanged (no new volumes or env vars needed)
- Frontend production build: 1403 modules, builds clean in ~27s
**Documentation:**
- README.md: new "YouTube Live Stream Proxy (Phase 3)" section with architecture, usage, config, limitations
- development_plan.md: Phase 3 timeline row, Phase 3 section (backend/frontend additions, design decisions)
--- ---
## 4. Timeline ## 4. Timeline
| Sub-Phase | Description | Effort | Depends On | Status | | Sub-Phase | Description | Effort | Depends On |
|---|---|---|---|---|---| |---|---|---|---|
| 3.1 | Config & Infrastructure | 0.5 day | — | ✅ Complete | | 3.1 | Config & Infrastructure | 0.5 day | — |
| 3.2 | YouTube URL Extraction | 0.5 day | 3.1 | ✅ Complete | | 3.2 | YouTube URL Extraction | 0.5 day | 3.1 |
| 3.3 | HLS Proxy Backend | 1 day | 3.1 | ✅ Complete | | 3.3 | HLS Proxy Backend | 1 day | 3.1 |
| 3.4 | Frontend Input + Player | 1 day | 3.2, 3.3 | ✅ Complete | | 3.4 | Frontend Input + Player | 1 day | 3.2, 3.3 |
| 3.5 | YouTube → ASR Integration | 1 day | 3.4 | ✅ Complete | | 3.5 | YouTube → ASR Integration | 1 day | 3.4 |
| 3.6 | Integration & Acceptance | 1 day | 3.5 | ✅ Complete | | 3.6 | Integration & Acceptance | 1 day | 3.5 |
| 3.7 | Polish & Deployment | 0.5 day | 3.6 | ✅ Complete | | 3.7 | Polish & Deployment | 0.5 day | 3.6 |
| **Total** | | **5.5 days** | | **7/7 done ✅** | | **Total** | | **5.5 days** | |
3.2 (extraction) and 3.3 (proxy) can run concurrently.
--- ---
@ -338,16 +265,13 @@ YT_DLP_CACHE_TTL=300
## 7. Key Design Decisions ## 7. Key Design Decisions
| Decision | Choice | Why | | Decision | Choice | Why |
|---|---|---|---| |---|---|---|
| Streaming protocol | HLS (m3u8) | hls.js plays it natively; DASH requires dash.js | | Streaming protocol | HLS (m3u8) | hls.js plays it natively; DASH requires dash.js |
| yt-dlp client | **Default** (no special client) | Default extractor works for both VOD (separate tracks) and live (combined HLS); iOS client caused "No video formats" errors on some live streams | | yt-dlp client | `ios` for live, `web` for VOD | `ios` returns HLS for live streams with 60fps support; format selector prefers ≤480p |
| Live format strategy | **Combined formats, same URL** | Live HLS formats include both video+audio; same URL for `<video>` and `<audio>` elements — hls.js demuxes each independently |
| HTTP client for proxy | httpx (already present) | Streaming support via `httpx.stream()`; no new dependency | | HTTP client for proxy | httpx (already present) | Streaming support via `httpx.stream()`; no new dependency |
| Manifest rewriting | Line-by-line streaming | Live manifests can be large; never buffer whole file | | Manifest rewriting | Line-by-line streaming | Live manifests can be large; never buffer whole file |
| Audio element | Hidden `<audio>` + hls.js | `createMediaElementSource` works on `<audio>` elements | | Audio element | Hidden `<audio>` + hls.js | `createMediaElementSource` works on `<audio>` elements |
| URL caching | In-memory dict with TTL | yt-dlp extraction is slow (~2-5s); reuse for 5 min live, 30 min VOD | | URL caching | In-memory dict with TTL | yt-dlp extraction is slow (~2-5s); reuse for 5 min |
| Service lifetime | `@lru_cache` singleton | Cache must persist across HTTP requests for caching to work |
| Error response | **HTTP 200 with error field** | API call succeeded; YouTube error is a content-level failure, not a protocol failure |
| **Full Transcript for YouTube** | **Disabled** | Button hidden; real-time streaming ASR only | | **Full Transcript for YouTube** | **Disabled** | Button hidden; real-time streaming ASR only |
| **QueryInput during streaming** | **Editable** | User can type corrections while transcript streams (same as existing ASR) | | **QueryInput during streaming** | **Editable** | User can type corrections while transcript streams (same as existing ASR) |
| **Video quality** | **360p–480p auto-best** | Low resolution sufficient for reference; no quality selector | | **Video quality** | **360p–480p auto-best** | Low resolution sufficient for reference; no quality selector |
@ -363,47 +287,46 @@ YT_DLP_CACHE_TTL=300
### New Files ### New Files
``` ```
backend/ backend/
app/models/youtube.py ✅ Created (3.1) app/models/youtube.py
app/services/youtube_service.py ✅ Created (3.1), implemented (3.2) app/services/youtube_service.py
app/services/hls_proxy.py ✅ Stub created (3.1) app/services/hls_proxy.py
app/routers/youtube.py ✅ Created (3.1), implemented (3.2) app/routers/youtube.py
app/test/test_phase3_config.py ✅ Written (3.1, 11 tests) app/test/test_phase3_config.py
app/test/test_phase3_youtube_extract.py ✅ Written (3.2, 18 tests) app/test/test_phase3_youtube_extract.py
app/test/test_phase3_hls_proxy.py ⏳ Pending (3.3) app/test/test_phase3_hls_proxy.py
app/test/test_phase3_hls_manifest.py ⏳ Pending (3.3) app/test/test_phase3_hls_manifest.py
app/test/test_integration_phase3.py ⏳ Pending (3.6) app/test/test_integration_phase3.py
app/test/acceptance/test_acceptance_phase3_youtube.py ⏳ Pending (3.6) app/test/acceptance/test_acceptance_phase3_youtube.py
app/test/acceptance/test_acceptance_phase3_live.py ⏳ Pending (3.6) app/test/acceptance/test_acceptance_phase3_live.py
frontend/src/ frontend/src/
components/YouTubeInput.tsx ✅ Created (3.4) components/YouTubeInput.tsx
components/YouTubeVideoPlayer.tsx ✅ Created (3.4) components/YouTubeVideoPlayer.tsx
hooks/useYouTubeASR.ts ✅ Created (3.5) hooks/useYouTubeASR.ts
pages/LTTPage.tsx ✅ Updated (3.5) test/test_phase3_YouTubeInput.test.tsx
test/test_phase3_YouTubeInput.test.tsx ✅ Written (3.4, 7 tests) test/test_phase3_YouTubeVideoPlayer.test.tsx
test/test_phase3_YouTubeVideoPlayer.test.tsx ✅ Written (3.4, 9 tests) test/test_phase3_useYouTubeASR.test.ts
test/test_phase3_useYouTubeASR.test.ts ✅ Written (3.5, 11 tests) test/test_phase3_LTTPage_integration.test.tsx
test/test_phase3_LTTPage_integration.test.tsx ✅ Written (3.5, 7 tests)
``` ```
### Modified Files ### Modified Files
``` ```
backend/app/core/config.py ✅ Done (3 fields) backend/app/core/config.py # Add 3 config fields
backend/.env.example ✅ Done (3 vars) backend/.env.example # Add 3 env vars
backend/main.py ✅ Done (router registered) backend/main.py # Register youtube router
backend/requirements.txt ✅ Done (yt-dlp added) backend/requirements.txt # Add yt-dlp
frontend/package.json ✅ Done (hls.js added) frontend/package.json # Add hls.js
frontend/src/types/index.ts ✅ Done (3.4) frontend/src/types/index.ts # Add YouTube types
frontend/src/lib/api.ts ✅ Done (3.4) frontend/src/lib/api.ts # Add extractYouTube(), getYouTubeProxyUrl()
frontend/src/lib/queries.tsx ✅ Done (3.4) frontend/src/lib/queries.tsx # Add useYouTubeExtract() mutation
frontend/src/pages/LTTPage.tsx ✅ Done (3.5) frontend/src/pages/LTTPage.tsx # Add source toggle + YouTube components
frontend/src/components/QueryInput.tsx ✅ Done (3.5 — no-op, already compatible) frontend/src/components/QueryInput.tsx # Accept transcript from either source
Dockerfile ⏳ Pending (3.7) Dockerfile # Add yt-dlp install step
docker-compose.yml ⏳ Pending (3.7) docker-compose.yml # Add env vars if needed
README.md ⏳ Pending (3.7) README.md # YouTube feature section
development_plan.md ⏳ Pending (3.7) development_plan.md # Mark Phase 3 status
``` ```
--- ---
@ -414,7 +337,7 @@ development_plan.md ⏳ Pending (3.7)
|---|---|---| |---|---|---|
| PO Token expiration (live streams cut at 30s) | High — live streams unusable without token | Auto-re-extract on HLS failure; document cookie-based workaround; acceptance test to quantify | | PO Token expiration (live streams cut at 30s) | High — live streams unusable without token | Auto-re-extract on HLS failure; document cookie-based workaround; acceptance test to quantify |
| yt-dlp extraction slow (2-5s) | Medium — poor UX on "Load Stream" click | Cache results with TTL; show progress indicator | | yt-dlp extraction slow (2-5s) | Medium — poor UX on "Load Stream" click | Cache results with TTL; show progress indicator |
| YouTube format changes break yt-dlp | Medium — sudden breakage | Pin yt-dlp version; CI test with known-good URLs; `pip install -U yt-dlp` in maintenance. **Note**: iOS client caused "No video formats" on Phoenix TV live stream; default extractor works for both tested URLs. Monitor for regressions. | | YouTube format changes break yt-dlp | Medium — sudden breakage | Pin yt-dlp version; CI test with known-good URLs; `pip install -U yt-dlp` in maintenance |
| hls.js audio sync drift vs video | Low — separate streams may drift | hls.js `liveSyncDuration` keeps both near live edge; test with 10+ min streams | | hls.js audio sync drift vs video | Low — separate streams may drift | hls.js `liveSyncDuration` keeps both near live edge; test with 10+ min streams |
| Safari `createMediaElementSource` on HLS | Low — known Safari bug with native HLS | hls.js uses MSE, not native HLS — works around Safari bug; Chrome/Firefox unaffected | | Safari `createMediaElementSource` on HLS | Low — known Safari bug with native HLS | hls.js uses MSE, not native HLS — works around Safari bug; Chrome/Firefox unaffected |
| YouTube ToS for proxy | Low for internal demo | Personal/enterprise internal demo is generally fine; review for public product | | YouTube ToS for proxy | Low for internal demo | Personal/enterprise internal demo is generally fine; review for public product |
@ -425,31 +348,17 @@ development_plan.md ⏳ Pending (3.7)
``` ```
POST /api/v1/youtube/extract POST /api/v1/youtube/extract
Body: {"url": "https://www.youtube.com/watch?v=5bF3tkO5jAA"} Body: {"url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ"}
Response: { Response: {
"video_id": "5bF3tkO5jAA", "video_id": "dQw4w9WgXcQ",
"title": "《2026年稅務(修訂)(自動交換資料)條例草案》委員會會議", "title": "Rick Astley - Never Gonna Give You Up",
"is_live": false, "is_live": false,
"is_upcoming": false, "video_proxy_url": "/api/v1/youtube/proxy/manifest.m3u8?url=...&type=video",
"video_proxy_url": "/api/v1/youtube/proxy/manifest.m3u8?url=https%3A%2F%2Frr2---sn-jna...", "audio_proxy_url": "/api/v1/youtube/proxy/manifest.m3u8?url=...&type=audio",
"audio_proxy_url": "/api/v1/youtube/proxy/manifest.m3u8?url=https%3A%2F%2Frr2---sn-jna...", "thumbnail_url": "https://i.ytimg.com/vi/dQw4w9WgXcQ/hqdefault.jpg"
"thumbnail_url": "https://i.ytimg.com/vi/5bF3tkO5jAA/hqdefault.jpg",
"formats": [...],
"error": null
} }
# Live stream (combined formats → same URL for video and audio) GET /api/v1/youtube/proxy/manifest.m3u8?url=<encoded_upstream_m3u8>&type=video
POST /api/v1/youtube/extract
Body: {"url": "https://www.youtube.com/watch?v=fN9uYWCjQaw"}
Response: {
"video_id": "fN9uYWCjQaw",
"is_live": true,
"video_proxy_url": "/api/v1/youtube/proxy/manifest.m3u8?url=...",
"audio_proxy_url": "/api/v1/youtube/proxy/manifest.m3u8?url=...",
# video_proxy_url == audio_proxy_url (same combined HLS manifest)
}
GET /api/v1/youtube/proxy/manifest.m3u8?url=<encoded_upstream_m3u8>
→ Fetches upstream manifest from googlevideo.com → Fetches upstream manifest from googlevideo.com
→ Rewrites segment URLs: → Rewrites segment URLs:
segment_0.ts → /api/v1/youtube/proxy/segment.ts?url=<encoded_segment_url> segment_0.ts → /api/v1/youtube/proxy/segment.ts?url=<encoded_segment_url>
@ -470,34 +379,3 @@ GET /api/v1/youtube/proxy/segment.ts?url=<encoded_upstream_ts>
- **hls.js API docs**: [github.com/video-dev/hls.js/blob/master/docs/API.md](https://github.com/video-dev/hls.js/blob/master/docs/API.md) - **hls.js API docs**: [github.com/video-dev/hls.js/blob/master/docs/API.md](https://github.com/video-dev/hls.js/blob/master/docs/API.md)
- **hls.js low-latency live**: `lowLatencyMode: true`, `liveSyncDuration: 1.5` - **hls.js low-latency live**: `lowLatencyMode: true`, `liveSyncDuration: 1.5`
- **Existing code patterns**: `.plans/phase2_implementation_plan.md`, `backend/app/routers/video.py`, `frontend/src/hooks/useVideoASR.ts` - **Existing code patterns**: `.plans/phase2_implementation_plan.md`, `backend/app/routers/video.py`, `frontend/src/hooks/useVideoASR.ts`
---
## 12. Test Results (Current)
| Suite | Tests | Status |
|-------|-------|--------|
| Phase 2 backend (existing) | 53 | ✅ All pass |
| Phase 2 frontend (existing) | 51 | ✅ All pass |
| Phase 3.1 (config) | 11 | ✅ All pass |
| Phase 3.2 (extraction) | 18 | ✅ All pass |
| Phase 3.3 (HLS proxy) | 22 | ✅ All pass |
| Phase 3.4 frontend (YouTube components) | 16 | ✅ All pass |
| Phase 3.5 frontend (ASR integration) | 18 | ✅ All pass |
| Phase 3.6 integration | 6 | ✅ All pass |
| Phase 3.6 acceptance (VOD) | 3 | ⏭ Skip (needs env) |
| Phase 3.6 acceptance (live) | 3 | ⏭ Skip (needs env) |
| Phase 3.7 | 2 tests (PO token) | ✅ All pass |
| **Total CI** | **197** | **0 failures** |
**Pre-existing failures** (not from Phase 3):
- `test_phase1_config.py::test_config_default_values` — model version mismatch (3.5 vs 3.6)
- `test_phase3_history_service.py` (13 errors) — missing `highlight_prompt` column
- `test_phase3_sqlite_db.py::test_seed_default_profiles_idempotent` — stale assertion
- `e2e/query_flow.test.tsx` (3 failures) — Phase 4 file input tests, unrelated
### Real-URL Smoke Tests
| URL | Type | Result |
|-----|------|--------|
| `5bF3tkO5jAA` (LegCo meeting) | VOD | 24 formats, separate video+audio ✅ |
| `fN9uYWCjQaw` (Phoenix TV 24h) | Live | 6 combined HLS formats, same URL ✅ |

38
README.md
View File

@ -244,44 +244,6 @@ Video → Audio → DashScope ASR → Transcript → QueryInput → RAG Pipeline
- `ffmpeg` on server (for batch transcription) - `ffmpeg` on server (for batch transcription)
- `dashscope` Python package (in `requirements.txt`) - `dashscope` Python package (in `requirements.txt`)
### YouTube Live Stream Proxy (Phase 3)
Proxy YouTube live streams and VODs through the backend, with real-time ASR transcription piped into the RAG pipeline — no file upload needed.
```
YouTube URL → yt-dlp extract → HLS manifest URLs
↓
HLS Proxy (backend): rewrites segment URLs → client fetches via proxy
↓
Frontend: hls.js plays video/audio → AudioContext → WebSocket → ASR → transcript
```
**How to use:**
1. Toggle source from "Upload" to "YouTube" in the video panel
2. Paste a YouTube URL (live stream or VOD)
3. Click "Load Stream" — backend extracts streams via yt-dlp
4. Press play — video plays via hls.js, audio feeds real-time ASR
5. Transcript flows into QueryInput as you watch
**Configuration:**
| Variable | Default | Description |
|----------|---------|-------------|
| `YOUTUBE_PROXY_ENABLED` | `false` | Enable YouTube proxy feature |
| `YT_DLP_TIMEOUT` | `30` | yt-dlp extraction timeout (seconds) |
| `YT_DLP_CACHE_TTL` | `300` | Cache TTL for extracted stream info |
**Requirements:**
- `YOUTUBE_PROXY_ENABLED=true` in `.env`
- `yt-dlp` (auto-installed via `requirements.txt`)
- `DASHSCOPE_API_KEY` in `.env` (for ASR)
**Known limitations:**
- YouTube may require PO tokens for some videos (especially live streams) — stream may need re-extraction if tokens expire
- Video quality limited to 480p max (no quality selector in UI — low resolution sufficient for reference viewing)
- YouTube segment URLs expire after ~6 hours
- "Full Transcript" button hidden for YouTube source (streaming ASR only)
### Installing ffmpeg ### Installing ffmpeg
```bash ```bash

5
backend/.env.example
View File

@ -36,8 +36,3 @@ ASR_REALTIME_MODEL_NAME=qwen3-asr-flash-realtime
# Video upload (Phase 2) # Video upload (Phase 2)
VIDEO_UPLOAD_DIR=./uploads VIDEO_UPLOAD_DIR=./uploads
MAX_VIDEO_SIZE_MB=300 MAX_VIDEO_SIZE_MB=300
# YouTube Proxy (Phase 3)
YOUTUBE_PROXY_ENABLED=true
YT_DLP_TIMEOUT=30
YT_DLP_CACHE_TTL=300

5
backend/app/core/config.py
View File

@ -54,11 +54,6 @@ class Settings(BaseSettings):
max_video_size_mb: int = 300 max_video_size_mb: int = 300
supported_video_formats: list[str] = [".mp4", ".webm", ".mov", ".avi", ".mkv"] supported_video_formats: list[str] = [".mp4", ".webm", ".mov", ".avi", ".mkv"]
# YouTube Proxy (Phase 3)
youtube_proxy_enabled: bool = True
yt_dlp_timeout: int = 30
yt_dlp_cache_ttl: int = 300 # seconds (live=5min shared; VOD=30min computed in service)
# Development helpers # Development helpers
model_config = {"env_file": ".env", "env_file_encoding": "utf-8"} model_config = {"env_file": ".env", "env_file_encoding": "utf-8"}

3
backend/app/main.py
View File

@ -7,7 +7,7 @@ from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import FileResponse from fastapi.responses import FileResponse
from app.routers import ingest, query, documents, prompts, history, chunks, video, ws_asr, youtube from app.routers import ingest, query, documents, prompts, history, chunks, video, ws_asr
from app.core.config import get_settings from app.core.config import get_settings
from app.core.sqlite_db import ( from app.core.sqlite_db import (
get_prompts_db, get_prompts_db,
@ -58,7 +58,6 @@ app.include_router(history.router)
app.include_router(chunks.router) app.include_router(chunks.router)
app.include_router(video.router, prefix="/api/v1") app.include_router(video.router, prefix="/api/v1")
app.include_router(ws_asr.router) app.include_router(ws_asr.router)
app.include_router(youtube.router, prefix="/api/v1")
_prompts_conn = get_prompts_db() _prompts_conn = get_prompts_db()
init_prompts_db(_prompts_conn) init_prompts_db(_prompts_conn)

28
backend/app/models/youtube.py
View File

@ -1,28 +0,0 @@
"""YouTube stream extraction models (Phase 3)."""
from pydantic import BaseModel
class YouTubeExtractRequest(BaseModel):
url: str
class StreamFormat(BaseModel):
format_id: str
url: str
resolution: str | None = None
is_audio_only: bool = False
is_video_only: bool = False
codec: str | None = None
class YouTubeStreamResponse(BaseModel):
video_id: str
title: str
is_live: bool = False
is_upcoming: bool = False
video_proxy_url: str | None = None
audio_proxy_url: str | None = None
thumbnail_url: str | None = None
formats: list[StreamFormat] = []
error: str | None = None

123
backend/app/routers/youtube.py
View File

@ -1,123 +0,0 @@
import logging
import time
from functools import lru_cache
from urllib.parse import unquote
import httpx
from fastapi import APIRouter, HTTPException, Query
from fastapi.responses import StreamingResponse
from app.models.youtube import YouTubeExtractRequest, YouTubeStreamResponse, StreamFormat
logger = logging.getLogger(__name__)
router = APIRouter(tags=["youtube"])
@lru_cache
def _get_youtube_service():
from app.core.config import get_settings
from app.services.youtube_service import YouTubeService
s = get_settings()
return YouTubeService(timeout=s.yt_dlp_timeout, cache_ttl=s.yt_dlp_cache_ttl)
@router.post("/youtube/extract", response_model=YouTubeStreamResponse)
async def extract_youtube_stream(req: YouTubeExtractRequest):
from app.core.config import get_settings
settings = get_settings()
if not settings.youtube_proxy_enabled:
raise HTTPException(status_code=503, detail="YouTube proxy is disabled")
service = _get_youtube_service()
started = time.monotonic()
logger.info("youtube-extract-started url=%s", req.url)
try:
data = await service.extract_streams(req.url)
except Exception as e:
logger.error("youtube-extract-failed url=%s error=%s", req.url, e)
raise HTTPException(status_code=500, detail=str(e))
if data.get("error"):
logger.warning(
"youtube-extract-error url=%s error=%s duration=%.1fs",
req.url,
data["error"],
time.monotonic() - started,
)
return YouTubeStreamResponse(
video_id=data.get("video_id", ""),
title=data.get("title", ""),
error=data["error"],
)
formats = [
StreamFormat(
format_id=f.get("format_id", ""),
url=f.get("url", ""),
resolution=f.get("resolution"),
is_audio_only=f.get("acodec", "none") != "none" and f.get("vcodec", "none") == "none",
is_video_only=f.get("vcodec", "none") != "none" and f.get("acodec", "none") == "none",
codec=f.get("vcodec") or f.get("acodec"),
)
for f in data.get("formats", [])
]
logger.info(
"youtube-extract-completed url=%s video_id=%s is_live=%s fmt_count=%d duration=%.1fs",
req.url,
data["video_id"],
data["is_live"],
len(formats),
time.monotonic() - started,
)
return YouTubeStreamResponse(
video_id=data["video_id"],
title=data["title"],
is_live=data["is_live"],
is_upcoming=data["is_upcoming"],
video_proxy_url=data.get("video_proxy_url"),
audio_proxy_url=data.get("audio_proxy_url"),
thumbnail_url=data.get("thumbnail_url"),
formats=formats,
)
@router.get("/youtube/proxy/manifest.m3u8")
async def proxy_manifest(url: str = Query(..., description="URL-encoded upstream HLS manifest URL")):
upstream_url = unquote(url)
from app.services.hls_proxy import HLSProxyService
client = httpx.AsyncClient(timeout=30.0)
req = client.build_request("GET", upstream_url)
upstream = await client.send(req, stream=True)
if upstream.status_code != 200:
await upstream.aclose()
await client.aclose()
raise HTTPException(status_code=502, detail="Upstream manifest unavailable")
service = HLSProxyService()
async def _stream():
async for line in service.rewrite_manifest(upstream_url, upstream):
yield line.encode("utf-8")
await upstream.aclose()
await client.aclose()
return StreamingResponse(
_stream(),
media_type="application/vnd.apple.mpegurl",
headers={"access-control-allow-origin": "*"},
)
@router.get("/youtube/proxy/segment.ts")
async def proxy_segment(url: str = Query(..., description="URL-encoded upstream .ts segment URL")):
upstream_url = unquote(url)
from app.services.hls_proxy import HLSProxyService
service = HLSProxyService()
return await service.proxy_segment(upstream_url)

75
backend/app/services/hls_proxy.py
View File

@ -1,75 +0,0 @@
"""HLS manifest proxy service (Phase 3.3).
Rewrites HLS manifests and proxies .ts segments so the browser treats
them as same-origin, enabling Web Audio API access to the audio track.
"""
import logging
import re
from typing import AsyncGenerator
from urllib.parse import quote, urljoin
import httpx
from fastapi.responses import StreamingResponse
logger = logging.getLogger(__name__)
class HLSProxyService:
"""Streams and rewrites HLS manifests; proxies .ts segments with zero re-encoding."""
async def rewrite_manifest(self, upstream_url: str, upstream: httpx.Response) -> AsyncGenerator[str, None]:
base_url = upstream_url
async for line in upstream.aiter_lines():
rewritten = self._rewrite_line(line, base_url)
yield rewritten + "\n"
def _rewrite_line(self, line: str, base_url: str) -> str:
stripped = line.rstrip("\r\n")
if not stripped:
return stripped
if stripped.startswith("#"):
if stripped.startswith("#EXT-X-KEY:") and 'URI="' in stripped:
return self._rewrite_key_uri(stripped, base_url)
return stripped
if "://" in stripped:
absolute_uri = stripped
else:
absolute_uri = urljoin(base_url, stripped)
return self._build_proxy_url_for_uri(absolute_uri)
def _rewrite_key_uri(self, line: str, base_url: str) -> str:
match = re.match(r'(#EXT-X-KEY:.*URI=")(.+?)(".*)', line)
if not match:
return line
prefix, uri, suffix = match.group(1), match.group(2), match.group(3)
if "://" in uri:
absolute_uri = uri
else:
absolute_uri = urljoin(base_url, uri)
proxy_uri = self._build_proxy_url_for_uri(absolute_uri)
return f"{prefix}{proxy_uri}{suffix}"
def _resolve_url(self, uri: str, base_url: str) -> str:
return urljoin(base_url, uri)
def _build_proxy_url_for_uri(self, absolute_uri: str) -> str:
encoded = quote(absolute_uri, safe="")
if absolute_uri.endswith(".m3u8"):
return f"/api/v1/youtube/proxy/manifest.m3u8?url={encoded}"
return f"/api/v1/youtube/proxy/segment.ts?url={encoded}"
async def proxy_segment(self, upstream_url: str) -> StreamingResponse:
async with httpx.AsyncClient(timeout=30.0) as client:
req = client.build_request("GET", upstream_url)
upstream = await client.send(req, stream=True)
return StreamingResponse(
upstream.aiter_bytes(),
status_code=upstream.status_code,
media_type="video/mp2t",
headers={"access-control-allow-origin": "*"},
)

148
backend/app/services/youtube_service.py
View File

@ -1,148 +0,0 @@
import asyncio
import logging
import time
from typing import Any
from urllib.parse import quote
import yt_dlp
logger = logging.getLogger(__name__)
def _is_po_token_error(msg: str) -> bool:
"""Detect PO token expiration or bot detection errors from yt-dlp."""
indicators = [
"sign in to confirm",
"not a bot",
"bot detection",
"po token",
"potoken",
]
msg_lower = msg.lower()
return any(indicator in msg_lower for indicator in indicators)
class YouTubeService:
def __init__(self, timeout: int, cache_ttl: int):
self.timeout = timeout
self.cache_ttl = cache_ttl
self._cache: dict[str, tuple[float, dict]] = {}
async def extract_streams(self, url: str) -> dict:
now = time.monotonic()
if url in self._cache:
cached_at, cached_data = self._cache[url]
is_live = cached_data.get("is_live", False)
ttl = self.cache_ttl if is_live else self.cache_ttl * 6
if now - cached_at < ttl:
logger.debug("Cache hit for URL=%s age=%.1fs", url, now - cached_at)
return cached_data
logger.debug("Cache expired for URL=%s", url)
try:
loop = asyncio.get_running_loop()
info = await loop.run_in_executor(None, lambda: self._extract_sync(url))
except yt_dlp.utils.DownloadError as e:
error_msg = str(e)
logger.warning("yt-dlp extraction failed for URL=%s: %s", url, error_msg[:200])
if _is_po_token_error(error_msg):
logger.warning(
"PO token expired or bot detected for URL=%s — invalidating cache, retry with fresh tokens recommended",
url,
)
self._cache.pop(url, None)
return {"error": error_msg[:500], "video_id": "", "title": "", "formats": []}
live_status = info.get("live_status", "not_live")
is_live = live_status == "is_live"
is_upcoming = live_status == "is_upcoming"
result = {
"video_id": info.get("id", ""),
"title": info.get("title", ""),
"is_live": is_live,
"is_upcoming": is_upcoming,
"thumbnail_url": info.get("thumbnail"),
"formats": info.get("formats", []),
"error": None,
}
if not is_upcoming and info.get("formats"):
try:
video_fmt, audio_fmt = self._select_best_formats(info["formats"])
result["video_proxy_url"] = self._build_proxy_url(video_fmt["url"])
result["audio_proxy_url"] = self._build_proxy_url(audio_fmt["url"])
except ValueError as e:
result["error"] = str(e)
ttl = self.cache_ttl if is_live else self.cache_ttl * 6
self._cache[url] = (now, result)
return result
def _extract_sync(self, url: str) -> dict:
opts = self._get_ydl_opts(url)
with yt_dlp.YoutubeDL(opts) as ydl:
return ydl.extract_info(url, download=False)
def _get_ydl_opts(self, url: str) -> dict:
opts: dict[str, Any] = {
"quiet": True,
"no_warnings": True,
"extract_flat": False,
}
return opts
def _select_best_formats(self, formats: list[dict]) -> tuple[dict, dict]:
video_only = [
f
for f in formats
if f.get("vcodec", "none") != "none" and f.get("acodec", "none") == "none"
]
audio_only = [
f
for f in formats
if f.get("acodec", "none") != "none" and f.get("vcodec", "none") == "none"
]
combined = [
f
for f in formats
if f.get("vcodec", "none") != "none"
and f.get("acodec", "none") != "none"
]
has_content = bool(combined or video_only or audio_only)
if not has_content:
raise ValueError("No streamable formats found")
if video_only and audio_only:
video_fmt = self._pick_best_video(video_only)
audio_fmt = max(audio_only, key=lambda f: f.get("abr") or 0)
return video_fmt, audio_fmt
if combined and audio_only:
combined_sorted = sorted(combined, key=lambda f: f.get("height") or 9999)
return combined_sorted[0], audio_only[0]
if combined:
best_combined = self._pick_best_video(combined)
return best_combined, best_combined
if video_only:
raise ValueError("No streamable audio format found")
raise ValueError("No streamable video format found")
def _pick_best_video(self, candidates: list[dict]) -> dict:
def _sort_key(f: dict) -> tuple[int, int, int, int]:
height = f.get("height") or 9999
tbr = f.get("tbr") or 0
is_m3u8 = 0 if f.get("protocol") in ("m3u8_native", "m3u8") else 1
at_or_under_480 = 0 if height <= 480 else 1
if at_or_under_480 == 0:
return (0, is_m3u8, -height, -tbr)
return (1, is_m3u8, height, -tbr)
return sorted(candidates, key=_sort_key)[0]
def _build_proxy_url(self, upstream_url: str) -> str:
encoded = quote(upstream_url, safe="")
return f"/api/v1/youtube/proxy/manifest.m3u8?url={encoded}"

97
backend/app/test/acceptance/test_acceptance_phase3_live.py
View File

@ -1,97 +0,0 @@
"""Acceptance test: Phase 3 YouTube live stream extraction and HLS proxy.
Prerequisites:
- BACKEND_URL env var set (default: http://localhost:8000)
- YOUTUBE_TEST_LIVE_URL env var set (a real YouTube live stream URL, e.g., https://www.youtube.com/watch?v=fN9uYWCjQaw)
- Backend server running with youtube_proxy_enabled=true
- Network access to YouTube
"""
import os
import time
import pytest
import requests
@pytest.mark.acceptance
@pytest.mark.slow
class TestYouTubeLiveAcceptance:
"""Real YouTube live stream extraction and HLS proxy verification."""
@pytest.fixture(autouse=True)
def check_prerequisites(self):
"""Skip if prerequisites not met."""
backend_url = os.getenv("BACKEND_URL", "http://localhost:8000")
yt_url = os.getenv("YOUTUBE_TEST_LIVE_URL")
if not yt_url:
pytest.skip("YOUTUBE_TEST_LIVE_URL not set")
self.base_url = backend_url.rstrip("/")
self.yt_url = yt_url
def _extract_live(self):
"""Helper: extract live stream, skip on error."""
resp = requests.post(
f"{self.base_url}/api/v1/youtube/extract",
json={"url": self.yt_url},
timeout=60,
)
assert resp.status_code == 200, f"Extract failed: {resp.text}"
data = resp.json()
if data.get("error"):
pytest.skip(f"Live stream error: {data['error']}")
return data
def test_extract_real_youtube_live(self):
"""Extract a real YouTube live URL and verify live-specific properties."""
data = self._extract_live()
assert data["video_id"], "video_id should be non-empty"
assert data["title"], "title should be non-empty"
assert data["is_live"] is True, f"Expected is_live=True, got {data['is_live']}"
assert data["video_proxy_url"], "video_proxy_url should be present for live"
assert data["audio_proxy_url"], "audio_proxy_url should be present for live"
assert data["thumbnail_url"], "thumbnail_url should be present"
assert len(data["formats"]) > 0, "formats list should have entries (live uses combined formats)"
for fmt in data["formats"]:
assert "format_id" in fmt
assert "url" in fmt
def test_proxy_live_manifest(self):
"""Extract live -> fetch manifest via proxy -> verify live HLS structure."""
data = self._extract_live()
proxy_url = data["video_proxy_url"]
if proxy_url.startswith("/"):
proxy_url = f"{self.base_url}{proxy_url}"
resp = requests.get(proxy_url, timeout=30)
assert resp.status_code == 200, f"Manifest proxy failed: {resp.status_code} {resp.text[:200]}"
assert (
"application/vnd.apple.mpegurl" in resp.headers.get("content-type", "")
), f"Unexpected content-type: {resp.headers.get('content-type')}"
assert (
resp.headers.get("access-control-allow-origin") == "*"
), "CORS header missing"
content = resp.text.strip()
assert content.startswith("#EXTM3U"), f"Manifest should start with #EXTM3U, got: {content[:100]}"
assert (
"#EXT-X-ENDLIST" not in content
), "Live manifest should NOT contain #EXT-X-ENDLIST (stream is ongoing)"
assert (
"#EXT-X-TARGETDURATION" in content or "#EXT-X-STREAM-INF" in content
), "Manifest should contain #EXT-X-TARGETDURATION or #EXT-X-STREAM-INF"
def test_live_cache_refresh(self):
"""Live streams should have shorter cache TTL — verify re-extract works."""
data1 = self._extract_live()
video_id_1 = data1["video_id"]
time.sleep(1)
data2 = self._extract_live()
assert data2["video_id"] == video_id_1, "Re-extract should return same video_id"
assert data1["formats"] is not None
assert data2["formats"] is not None

148
backend/app/test/acceptance/test_acceptance_phase3_youtube.py
View File

@ -1,148 +0,0 @@
"""Acceptance test: Phase 3 YouTube VOD extraction and HLS proxy.
Prerequisites:
- BACKEND_URL env var set (default: http://localhost:8000)
- YOUTUBE_TEST_VOD_URL env var set (a real YouTube VOD URL, e.g., https://www.youtube.com/watch?v=5bF3tkO5jAA)
- Backend server running with youtube_proxy_enabled=true
- Network access to YouTube
"""
import json
import os
import time
import pytest
import requests
@pytest.mark.acceptance
@pytest.mark.slow
class TestYouTubeVODAcceptance:
"""Real YouTube VOD extraction and HLS proxy verification."""
@pytest.fixture(autouse=True)
def check_prerequisites(self):
"""Skip if prerequisites not met."""
backend_url = os.getenv("BACKEND_URL", "http://localhost:8000")
yt_url = os.getenv("YOUTUBE_TEST_VOD_URL")
if not yt_url:
pytest.skip("YOUTUBE_TEST_VOD_URL not set")
self.base_url = backend_url.rstrip("/")
self.yt_url = yt_url
def test_extract_real_youtube_vod(self):
"""Extract a real YouTube VOD URL and verify response structure."""
resp = requests.post(
f"{self.base_url}/api/v1/youtube/extract",
json={"url": self.yt_url},
timeout=60,
)
assert resp.status_code == 200, f"Extract failed: {resp.text}"
data = resp.json()
assert data["error"] is None, f"Extraction returned error: {data['error']}"
assert data["video_id"], "video_id should be non-empty"
assert data["title"], "title should be non-empty"
assert data["is_live"] is False, "VOD should not be live"
assert data["is_upcoming"] is False, "VOD should not be upcoming"
assert data["video_proxy_url"], "video_proxy_url should be present"
assert data["audio_proxy_url"], "audio_proxy_url should be present"
assert data["thumbnail_url"], "thumbnail_url should be present"
assert len(data["formats"]) > 0, "formats list should be non-empty"
for fmt in data["formats"]:
assert "format_id" in fmt
assert "url" in fmt
def test_proxy_manifest_from_real_vod(self):
"""Extract -> fetch manifest via proxy -> verify valid M3U8."""
resp = requests.post(
f"{self.base_url}/api/v1/youtube/extract",
json={"url": self.yt_url},
timeout=60,
)
assert resp.status_code == 200, f"Extract failed: {resp.text}"
data = resp.json()
assert data["video_proxy_url"], "No video_proxy_url in extract response"
proxy_url = data["video_proxy_url"]
if proxy_url.startswith("/"):
proxy_url = f"{self.base_url}{proxy_url}"
resp = requests.get(proxy_url, timeout=30)
assert resp.status_code == 200, f"Manifest proxy failed: {resp.status_code} {resp.text[:200]}"
assert (
"application/vnd.apple.mpegurl" in resp.headers.get("content-type", "")
), f"Unexpected content-type: {resp.headers.get('content-type')}"
assert (
resp.headers.get("access-control-allow-origin") == "*"
), "CORS header missing"
body = resp.text.strip()
assert body.startswith("#EXTM3U"), f"Manifest should start with #EXTM3U, got: {body[:100]}"
assert (
"#EXT-X-STREAM-INF" in body or "#EXTINF" in body
), "Manifest should contain #EXT-X-STREAM-INF or #EXTINF"
def test_proxy_segment_from_real_vod(self):
"""Proxy a TS segment and verify it returns video data."""
resp = requests.post(
f"{self.base_url}/api/v1/youtube/extract",
json={"url": self.yt_url},
timeout=60,
)
assert resp.status_code == 200, f"Extract failed: {resp.text}"
data = resp.json()
assert data["video_proxy_url"], "No video_proxy_url in extract response"
proxy_url = data["video_proxy_url"]
if proxy_url.startswith("/"):
proxy_url = f"{self.base_url}{proxy_url}"
resp = requests.get(proxy_url, timeout=30)
assert resp.status_code == 200, f"Master manifest failed: {resp.status_code}"
master_body = resp.text.strip()
variant_url = None
lines = master_body.split("\n")
for line in lines:
stripped = line.strip()
if stripped.startswith("/api/v1/youtube/proxy/manifest.m3u8?url="):
variant_url = stripped
break
if variant_url is None:
pytest.skip("No variant manifest URL found (may be a non-master manifest)")
if variant_url.startswith("/"):
variant_url = f"{self.base_url}{variant_url}"
resp = requests.get(variant_url, timeout=30)
assert resp.status_code == 200, f"Variant manifest failed: {resp.status_code}"
variant_body = resp.text.strip()
assert variant_body.startswith("#EXTM3U"), "Variant should start with #EXTM3U"
segment_url = None
for line in variant_body.split("\n"):
stripped = line.strip()
if stripped.startswith("/api/v1/youtube/proxy/segment.ts?url="):
segment_url = stripped
break
assert segment_url is not None, "No segment URL found in variant manifest"
if segment_url.startswith("/"):
segment_url = f"{self.base_url}{segment_url}"
seg_resp = requests.get(segment_url, timeout=30)
if seg_resp.status_code != 200:
pytest.skip(
f"Segment proxy returned {seg_resp.status_code} (YouTube segment may have expired)"
)
assert (
seg_resp.headers.get("access-control-allow-origin") == "*"
), "CORS header missing on segment"
assert len(seg_resp.content) > 0, "Segment body should be non-empty"
assert (
seg_resp.content[0] == 0x47
), "MPEG-TS segment should start with 0x47 sync byte"

361
backend/app/test/test_integration_phase3.py
View File

@ -1,361 +0,0 @@
"""Integration test: Phase 3 end-to-end YouTube extract → HLS proxy.
Covers:
- Full extract → proxy flow with mocked yt-dlp and upstream HTTP
- Proxy manifest rewriting through real HLSProxyService
- Proxy segment passthrough through real HLSProxyService
- Error handling: upstream failure after extract
- Config: youtube_proxy_enabled flag
All yt-dlp calls and upstream HTTP servers are mocked.
Real FastAPI TestClient with real YouTube router and HLSProxyService.
"""
from unittest.mock import AsyncMock, MagicMock, patch
from urllib.parse import quote
import pytest
from fastapi import FastAPI
from fastapi.testclient import TestClient
# ---------------------------------------------------------------------------
# Helpers — fake yt-dlp format data
# ---------------------------------------------------------------------------
def _make_format(
format_id: str,
height: int | None = None,
vcodec: str = "none",
acodec: str = "none",
ext: str = "mp4",
protocol: str = "https",
url: str = "",
abr: float | None = None,
tbr: float | None = None,
resolution: str | None = None,
) -> dict:
return {
"format_id": format_id,
"height": height,
"width": height * 16 // 9 if height else None,
"vcodec": vcodec,
"acodec": acodec,
"ext": ext,
"protocol": protocol,
"url": url or f"https://example.com/{format_id}.{ext}",
"abr": abr,
"tbr": tbr,
"resolution": resolution or (f"{height * 16 // 9}x{height}" if height else None),
}
def _vod_info(video_id: str = "abc123") -> dict:
return {
"id": video_id,
"title": "Test VOD Video",
"thumbnail": f"https://i.ytimg.com/vi/{video_id}/hqdefault.jpg",
"live_status": "not_live",
"duration": 300,
"formats": [
_make_format("137", height=1080, vcodec="avc1.640028", acodec="none", tbr=5000),
_make_format("136", height=720, vcodec="avc1.640028", acodec="none", tbr=2500),
_make_format("135", height=480, vcodec="avc1.640028", acodec="none", tbr=1200),
_make_format("134", height=360, vcodec="avc1.640028", acodec="none", tbr=600),
_make_format("133", height=240, vcodec="avc1.640028", acodec="none", tbr=300),
_make_format("140", acodec="mp4a.40.2", vcodec="none", abr=128),
_make_format("251", acodec="opus", vcodec="none", abr=160),
_make_format("18", height=360, vcodec="avc1.42001E", acodec="mp4a.40.2", tbr=500),
],
}
def _vod_info_hls(video_id: str = "abc123") -> dict:
return {
"id": video_id,
"title": "Test VOD with HLS",
"thumbnail": f"https://i.ytimg.com/vi/{video_id}/hqdefault.jpg",
"live_status": "not_live",
"duration": 600,
"formats": [
_make_format("136", height=720, vcodec="avc1.640028", acodec="none", ext="m3u8", protocol="m3u8_native", tbr=2500),
_make_format("135", height=480, vcodec="avc1.640028", acodec="none", ext="m3u8", protocol="m3u8_native", tbr=1200),
_make_format("140", acodec="mp4a.40.2", vcodec="none", ext="m3u8", protocol="m3u8_native", abr=128),
],
}
def _live_info(video_id: str = "live999") -> dict:
return {
"id": video_id,
"title": "Live Stream Test",
"thumbnail": f"https://i.ytimg.com/vi/{video_id}/hqdefault_live.jpg",
"live_status": "is_live",
"duration": None,
"formats": [
_make_format("91", height=144, vcodec="avc1.42C00B", acodec="mp4a.40.5", ext="mp4", protocol="m3u8_native"),
_make_format("92", height=240, vcodec="avc1.4D4015", acodec="mp4a.40.5", ext="mp4", protocol="m3u8_native"),
_make_format("93", height=360, vcodec="avc1.4D401E", acodec="mp4a.40.2", ext="mp4", protocol="m3u8_native"),
_make_format("94", height=480, vcodec="avc1.4D401F", acodec="mp4a.40.2", ext="mp4", protocol="m3u8_native", tbr=1200),
_make_format("95", height=720, vcodec="avc1.4D401F", acodec="mp4a.40.2", ext="mp4", protocol="m3u8_native"),
],
}
def _make_mock_ydl(return_value: dict | Exception) -> MagicMock:
mock_instance = MagicMock()
if isinstance(return_value, Exception):
mock_instance.extract_info.side_effect = return_value
else:
mock_instance.extract_info.return_value = return_value
mock_ydl = MagicMock()
mock_ydl.__enter__.return_value = mock_instance
mock_ydl.__exit__.return_value = None
return mock_ydl
# ---------------------------------------------------------------------------
# Helpers — mock httpx responses
# ---------------------------------------------------------------------------
def _make_mock_stream_response(status_code: int = 200, **kwargs) -> MagicMock:
mock = MagicMock()
mock.status_code = status_code
mock.aclose = AsyncMock()
mock.__aenter__ = AsyncMock(return_value=mock)
mock.__aexit__ = AsyncMock(return_value=None)
for key, value in kwargs.items():
setattr(mock, key, value)
return mock
def _make_mock_client(resp_mock: MagicMock) -> MagicMock:
client = MagicMock()
client.stream = MagicMock(return_value=resp_mock)
client.send = AsyncMock(return_value=resp_mock)
client.build_request = MagicMock(return_value=MagicMock())
client.aclose = AsyncMock()
client.__aenter__ = AsyncMock(return_value=client)
client.__aexit__ = AsyncMock(return_value=None)
return client
async def _async_iter_lines(text: str):
for line in text.split("\n"):
yield line
async def _async_iter_bytes(chunks: list[bytes]):
for chunk in chunks:
yield chunk
# ---------------------------------------------------------------------------
# Shared fixture builder
# ---------------------------------------------------------------------------
def _build_client(monkeypatch, enabled: bool = True):
from app.routers.youtube import router, _get_youtube_service
from app.core.config import get_settings
_get_youtube_service.cache_clear()
get_settings.cache_clear()
monkeypatch.setenv("YOUTUBE_PROXY_ENABLED", "true" if enabled else "false")
monkeypatch.setenv("YT_DLP_TIMEOUT", "30")
get_settings.cache_clear()
app = FastAPI()
app.include_router(router, prefix="/api/v1")
return TestClient(app)
# ---------------------------------------------------------------------------
# Integration: Full extract → proxy flow
# ---------------------------------------------------------------------------
class TestExtractAndProxyFlow:
"""Full extract → proxy flow with mocked yt-dlp and upstream."""
@pytest.fixture
def client(self, monkeypatch):
return _build_client(monkeypatch, enabled=True)
def test_extract_vod_then_proxy_manifest(self, client):
mock_ydl = _make_mock_ydl(_vod_info_hls("vod1"))
with patch("app.services.youtube_service.yt_dlp.YoutubeDL", return_value=mock_ydl):
extract_resp = client.post(
"/api/v1/youtube/extract",
json={"url": "https://www.youtube.com/watch?v=vod1"},
)
assert extract_resp.status_code == 200
data = extract_resp.json()
proxy_url = data["video_proxy_url"]
assert proxy_url is not None
assert "manifest.m3u8?url=" in proxy_url
upstream_manifest = (
"#EXTM3U\n"
"#EXT-X-VERSION:3\n"
"#EXTINF:6.0,\n"
"segment_0.ts\n"
"#EXT-X-ENDLIST\n"
)
upstream = _make_mock_stream_response(
status_code=200,
aiter_lines=lambda: _async_iter_lines(upstream_manifest),
)
with patch("app.routers.youtube.httpx.AsyncClient") as mock_cls:
mock_client = _make_mock_client(upstream)
mock_cls.return_value = mock_client
proxy_resp = client.get(proxy_url)
assert proxy_resp.status_code == 200
assert proxy_resp.headers.get("access-control-allow-origin") == "*"
content = proxy_resp.text
assert "#EXTM3U" in content
assert "/api/v1/youtube/proxy/segment.ts?url=" in content
assert "#EXT-X-ENDLIST" in content
def test_extract_vod_then_proxy_segment(self, client):
mock_ydl = _make_mock_ydl(_vod_info_hls("vod2"))
with patch("app.services.youtube_service.yt_dlp.YoutubeDL", return_value=mock_ydl):
extract_resp = client.post(
"/api/v1/youtube/extract",
json={"url": "https://www.youtube.com/watch?v=vod2"},
)
assert extract_resp.status_code == 200
segment_upstream_url = "https://example.com/segment_0.ts"
segment_proxy_url = (
f"/api/v1/youtube/proxy/segment.ts?url={quote(segment_upstream_url, safe='')}"
)
resp_mock = _make_mock_stream_response(
status_code=200,
headers={"content-type": "video/mp2t"},
aiter_bytes=lambda: _async_iter_bytes([b"\x47" * 188]),
)
client_mock = _make_mock_client(resp_mock)
with patch("app.services.hls_proxy.httpx.AsyncClient", return_value=client_mock):
seg_resp = client.get(segment_proxy_url)
assert seg_resp.status_code == 200
assert seg_resp.headers.get("access-control-allow-origin") == "*"
assert seg_resp.headers.get("content-type") == "video/mp2t"
def test_extract_live_then_proxy_manifest(self, client):
mock_ydl = _make_mock_ydl(_live_info("live1"))
with patch("app.services.youtube_service.yt_dlp.YoutubeDL", return_value=mock_ydl):
extract_resp = client.post(
"/api/v1/youtube/extract",
json={"url": "https://www.youtube.com/watch?v=live1"},
)
assert extract_resp.status_code == 200
data = extract_resp.json()
assert data["is_live"] is True
proxy_url = data["video_proxy_url"]
assert proxy_url is not None
live_manifest = (
"#EXTM3U\n"
"#EXT-X-VERSION:3\n"
"#EXT-X-TARGETDURATION:4\n"
"#EXTINF:4.0,\n"
"segment_live_0.ts\n"
"#EXTINF:4.0,\n"
"segment_live_1.ts\n"
)
upstream = _make_mock_stream_response(
status_code=200,
aiter_lines=lambda: _async_iter_lines(live_manifest),
)
with patch("app.routers.youtube.httpx.AsyncClient") as mock_cls:
mock_client = _make_mock_client(upstream)
mock_cls.return_value = mock_client
proxy_resp = client.get(proxy_url)
assert proxy_resp.status_code == 200
content = proxy_resp.text
assert "#EXTM3U" in content
assert "#EXT-X-ENDLIST" not in content
assert "/api/v1/youtube/proxy/segment.ts?url=" in content
def test_extract_cache_hit_bypasses_ytdlp(self, client):
mock_ydl = _make_mock_ydl(_vod_info("cached_vod"))
instance = mock_ydl.__enter__.return_value
with patch("app.services.youtube_service.yt_dlp.YoutubeDL", return_value=mock_ydl):
r1 = client.post(
"/api/v1/youtube/extract",
json={"url": "https://www.youtube.com/watch?v=cached_vod"},
)
r2 = client.post(
"/api/v1/youtube/extract",
json={"url": "https://www.youtube.com/watch?v=cached_vod"},
)
assert r1.status_code == 200
assert r2.status_code == 200
assert r1.json()["video_id"] == r2.json()["video_id"]
assert instance.extract_info.call_count == 1
# ---------------------------------------------------------------------------
# Integration: Upstream failures after successful extraction
# ---------------------------------------------------------------------------
class TestProxyAfterExtract:
"""Error scenarios where upstream fails after successful extraction."""
@pytest.fixture
def client(self, monkeypatch):
return _build_client(monkeypatch, enabled=True)
def test_upstream_manifest_unavailable_after_extract(self, client):
mock_ydl = _make_mock_ydl(_vod_info_hls("err1"))
with patch("app.services.youtube_service.yt_dlp.YoutubeDL", return_value=mock_ydl):
extract_resp = client.post(
"/api/v1/youtube/extract",
json={"url": "https://www.youtube.com/watch?v=err1"},
)
assert extract_resp.status_code == 200
proxy_url = extract_resp.json()["video_proxy_url"]
upstream = _make_mock_stream_response(status_code=404)
with patch("app.routers.youtube.httpx.AsyncClient") as mock_cls:
mock_client = _make_mock_client(upstream)
mock_cls.return_value = mock_client
proxy_resp = client.get(proxy_url)
assert proxy_resp.status_code == 502
# ---------------------------------------------------------------------------
# Config: youtube_proxy_enabled flag
# ---------------------------------------------------------------------------
class TestExtractDisabled:
"""Config flag youtube_proxy_enabled=False."""
def test_extract_returns_503_when_disabled(self, monkeypatch):
client = _build_client(monkeypatch, enabled=False)
resp = client.post(
"/api/v1/youtube/extract",
json={"url": "https://www.youtube.com/watch?v=abc123"},
)
assert resp.status_code == 503

177
backend/app/test/test_phase3_config.py
View File

@ -1,177 +0,0 @@
"""Phase 3.1 tests: Configuration and infrastructure setup for YouTube proxy.
Covers:
- Config fields: youtube_proxy_enabled, yt_dlp_timeout, yt_dlp_cache_ttl defaults and env loading
- Model schemas: YouTubeExtractRequest, YouTubeStreamResponse, StreamFormat
- Service stubs: YouTubeService, HLSProxyService instantiation
- Router registration: youtube.router mounted, endpoint responds 200 with mock
"""
from unittest.mock import MagicMock, patch
import pytest
from fastapi import FastAPI
from fastapi.testclient import TestClient
class TestYouTubeProxyConfig:
"""Config fields for YouTube proxy exist with correct defaults."""
@pytest.fixture(autouse=True)
def clear_cache(self):
from app.core.config import get_settings
get_settings.cache_clear()
yield
get_settings.cache_clear()
def test_defaults(self):
from app.core.config import get_settings
s = get_settings()
assert s.youtube_proxy_enabled is True
assert s.yt_dlp_timeout == 30
assert s.yt_dlp_cache_ttl == 300
def test_env_override(self, monkeypatch):
monkeypatch.setenv("YOUTUBE_PROXY_ENABLED", "false")
monkeypatch.setenv("YT_DLP_TIMEOUT", "60")
monkeypatch.setenv("YT_DLP_CACHE_TTL", "600")
from app.core.config import get_settings
s = get_settings()
assert s.youtube_proxy_enabled is False
assert s.yt_dlp_timeout == 60
assert s.yt_dlp_cache_ttl == 600
def test_bool_parsing(self, monkeypatch):
"""Bool fields accept 'true'/'false', '1'/'0' (pydantic-settings)."""
monkeypatch.setenv("YOUTUBE_PROXY_ENABLED", "0")
from app.core.config import get_settings
s = get_settings()
assert s.youtube_proxy_enabled is False
class TestYouTubeModels:
"""Pydantic models for YouTube stream extraction."""
def test_extract_request(self):
from app.models.youtube import YouTubeExtractRequest
req = YouTubeExtractRequest(url="https://www.youtube.com/watch?v=abc123")
assert req.url == "https://www.youtube.com/watch?v=abc123"
def test_stream_response_defaults(self):
from app.models.youtube import YouTubeStreamResponse
resp = YouTubeStreamResponse(video_id="abc123", title="Test Video")
assert resp.video_id == "abc123"
assert resp.title == "Test Video"
assert resp.is_live is False
assert resp.is_upcoming is False
assert resp.video_proxy_url is None
assert resp.audio_proxy_url is None
assert resp.formats == []
assert resp.error is None
def test_stream_format(self):
from app.models.youtube import StreamFormat
fmt = StreamFormat(
format_id="140",
url="https://example.com/audio.m3u8",
is_audio_only=True,
codec="mp4a.40.2",
)
assert fmt.format_id == "140"
assert fmt.is_audio_only is True
assert fmt.is_video_only is False
assert fmt.resolution is None
class TestYouTubeServices:
"""Service stubs can be imported and instantiated."""
def test_youtube_service_instantiate(self):
from app.services.youtube_service import YouTubeService
svc = YouTubeService(timeout=30, cache_ttl=300)
assert svc.timeout == 30
assert svc.cache_ttl == 300
def test_youtube_service_extract_is_async(self):
from app.services.youtube_service import YouTubeService
svc = YouTubeService(timeout=30, cache_ttl=300)
import inspect
assert inspect.iscoroutinefunction(svc.extract_streams)
def test_hls_proxy_instantiate(self):
from app.services.hls_proxy import HLSProxyService
svc = HLSProxyService()
assert svc is not None
class TestYouTubeRouter:
"""YouTube router is mounted and stub endpoint responds correctly."""
@pytest.fixture
def youtube_client(self):
from app.routers.youtube import router
from app.core.config import get_settings
get_settings.cache_clear()
app = FastAPI()
app.include_router(router, prefix="/api/v1")
return TestClient(app)
def test_extract_responds_with_mocked_ytdlp(self, youtube_client):
from app.routers.youtube import _get_youtube_service
_get_youtube_service.cache_clear()
vod_info = {
"id": "test123",
"title": "Test",
"thumbnail": "https://example.com/thumb.jpg",
"live_status": "not_live",
"formats": [
{
"format_id": "135", "height": 480,
"vcodec": "avc1", "acodec": "none",
"ext": "mp4", "protocol": "https",
"url": "https://example.com/video.mp4", "tbr": 1200,
},
{
"format_id": "140",
"vcodec": "none", "acodec": "mp4a",
"ext": "m4a", "protocol": "https",
"url": "https://example.com/audio.m4a", "abr": 128,
},
],
}
mock_ydl = MagicMock()
mock_instance = MagicMock()
mock_instance.extract_info.return_value = vod_info
mock_ydl.__enter__.return_value = mock_instance
with patch("app.services.youtube_service.yt_dlp.YoutubeDL", return_value=mock_ydl):
resp = youtube_client.post(
"/api/v1/youtube/extract",
json={"url": "https://www.youtube.com/watch?v=test123"},
)
assert resp.status_code == 200
data = resp.json()
assert data["video_id"] == "test123"
assert data["video_proxy_url"] is not None
assert data["audio_proxy_url"] is not None
def test_router_tag(self):
from app.routers.youtube import router
assert any(tag == "youtube" for tag in router.tags)

337
backend/app/test/test_phase3_hls_proxy.py
View File

@ -1,337 +0,0 @@
"""Phase 3.3 tests: HLS proxy service — manifest rewriting and segment proxying.
Covers:
- Manifest line rewriting: segments, sub-manifests, EXT-X-KEY URIs, pass-through tags
- URL resolution: relative paths, absolute paths, absolute URLs
- Segment proxying: StreamingResponse with correct Content-Type and CORS headers
- Route integration: GET /youtube/proxy/manifest.m3u8 and /segment.ts
- Error handling: upstream failures → 502, client disconnect
- CORS headers on every response
"""
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from httpx import Response, Request
def _make_mock_stream_response(status_code: int = 200, **kwargs) -> MagicMock:
mock = MagicMock()
mock.status_code = status_code
mock.aclose = AsyncMock()
mock.__aenter__ = AsyncMock(return_value=mock)
mock.__aexit__ = AsyncMock(return_value=None)
for key, value in kwargs.items():
setattr(mock, key, value)
return mock
def _make_mock_client(resp_mock: MagicMock) -> MagicMock:
client = MagicMock()
client.stream = MagicMock(return_value=resp_mock)
client.send = AsyncMock(return_value=resp_mock)
client.build_request = MagicMock(return_value=MagicMock())
client.aclose = AsyncMock()
client.__aenter__ = AsyncMock(return_value=client)
client.__aexit__ = AsyncMock(return_value=None)
return client
# ---------------------------------------------------------------------------
# Unit: Line rewriting
# ---------------------------------------------------------------------------
class TestLineRewriting:
@pytest.fixture
def svc(self):
from app.services.hls_proxy import HLSProxyService
return HLSProxyService()
def test_passes_through_comment_tags(self, svc):
base = "https://example.com/manifest.m3u8"
assert svc._rewrite_line("#EXTM3U", base) == "#EXTM3U"
assert svc._rewrite_line("#EXT-X-VERSION:3", base) == "#EXT-X-VERSION:3"
assert svc._rewrite_line("#EXT-X-TARGETDURATION:6", base) == "#EXT-X-TARGETDURATION:6"
assert svc._rewrite_line("#EXT-X-MEDIA-SEQUENCE:0", base) == "#EXT-X-MEDIA-SEQUENCE:0"
assert svc._rewrite_line("#EXT-X-ENDLIST", base) == "#EXT-X-ENDLIST"
assert svc._rewrite_line("# This is a comment", base) == "# This is a comment"
assert svc._rewrite_line("#EXT-X-STREAM-INF:BANDWIDTH=800000,RESOLUTION=640x360", base) == "#EXT-X-STREAM-INF:BANDWIDTH=800000,RESOLUTION=640x360"
def test_passes_through_empty_lines(self, svc):
assert svc._rewrite_line("", "https://example.com/base.m3u8") == ""
def test_rewrites_ts_segment(self, svc):
base = "https://example.com/path/manifest.m3u8"
result = svc._rewrite_line("segment_0.ts", base)
assert result.startswith("/api/v1/youtube/proxy/segment.ts?url=")
def test_rewrites_m3u8_submanifest(self, svc):
base = "https://example.com/path/manifest.m3u8"
result = svc._rewrite_line("variant_360p.m3u8", base)
assert result.startswith("/api/v1/youtube/proxy/manifest.m3u8?url=")
def test_rewrites_ext_x_key_uri(self, svc):
base = "https://example.com/manifest.m3u8"
result = svc._rewrite_line('#EXT-X-KEY:METHOD=AES-128,URI="key.bin",IV=0x1234', base)
assert result.startswith("#EXT-X-KEY:METHOD=AES-128,URI=\"")
assert "/api/v1/youtube/proxy/segment.ts?url=" in result
def test_rewrites_m3u8_key_uri(self, svc):
base = "https://example.com/manifest.m3u8"
result = svc._rewrite_line('#EXT-X-KEY:METHOD=AES-128,URI="keys/variant.m3u8"', base)
assert result.startswith("#EXT-X-KEY:METHOD=AES-128,URI=\"")
assert "/api/v1/youtube/proxy/manifest.m3u8?url=" in result
def test_rewrites_absolute_url_segment(self, svc):
base = "https://example.com/manifest.m3u8"
result = svc._rewrite_line("https://cdn.example.com/segments/0.ts", base)
assert result.startswith("/api/v1/youtube/proxy/segment.ts?url=")
def test_passes_through_inf_tag_with_commas(self, svc):
base = "https://example.com/manifest.m3u8"
result = svc._rewrite_line("#EXTINF:6.000,Some description, with commas", base)
assert result == "#EXTINF:6.000,Some description, with commas"
# ---------------------------------------------------------------------------
# Unit: URL resolution
# ---------------------------------------------------------------------------
class TestURLResolution:
@pytest.fixture
def svc(self):
from app.services.hls_proxy import HLSProxyService
return HLSProxyService()
def test_relative_path_resolved(self, svc):
result = svc._resolve_url("segment_0.ts", "https://example.com/path/manifest.m3u8")
assert result == "https://example.com/path/segment_0.ts"
def test_absolute_path_resolved(self, svc):
result = svc._resolve_url("/segments/0.ts", "https://example.com/path/manifest.m3u8")
assert result == "https://example.com/segments/0.ts"
def test_absolute_url_passthrough(self, svc):
result = svc._resolve_url("https://cdn.example.com/0.ts", "https://example.com/manifest.m3u8")
assert result == "https://cdn.example.com/0.ts"
def test_parent_dir_resolved(self, svc):
result = svc._resolve_url("../segments/0.ts", "https://example.com/path/to/manifest.m3u8")
assert result == "https://example.com/path/segments/0.ts"
# ---------------------------------------------------------------------------
# Unit: Proxy URL construction
# ---------------------------------------------------------------------------
class TestProxyURLConstruction:
@pytest.fixture
def svc(self):
from app.services.hls_proxy import HLSProxyService
return HLSProxyService()
def test_segment_extension_uses_segment_proxy(self, svc):
from urllib.parse import unquote
upstream = "https://cdn.example.com/segments/0.ts"
proxy = svc._build_proxy_url_for_uri(upstream)
assert proxy.startswith("/api/v1/youtube/proxy/segment.ts?url=")
encoded = proxy.split("url=", 1)[1]
assert unquote(encoded) == upstream
def test_m3u8_extension_uses_manifest_proxy(self, svc):
from urllib.parse import unquote
upstream = "https://cdn.example.com/variants/360p.m3u8"
proxy = svc._build_proxy_url_for_uri(upstream)
assert proxy.startswith("/api/v1/youtube/proxy/manifest.m3u8?url=")
encoded = proxy.split("url=", 1)[1]
assert unquote(encoded) == upstream
def test_unknown_extension_uses_segment_proxy(self, svc):
from urllib.parse import unquote
upstream = "https://cdn.example.com/init.mp4"
proxy = svc._build_proxy_url_for_uri(upstream)
assert proxy.startswith("/api/v1/youtube/proxy/segment.ts?url=")
# ---------------------------------------------------------------------------
# Integration: Manifest rewriting with mocked httpx
# ---------------------------------------------------------------------------
SAMPLE_MANIFEST = """#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:6
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:6.000,
segment_0.ts
#EXTINF:6.000,
segment_1.ts
#EXT-X-ENDLIST
"""
class TestManifestRewriting:
@pytest.fixture
def svc(self):
from app.services.hls_proxy import HLSProxyService
return HLSProxyService()
@pytest.mark.asyncio
async def test_full_manifest_rewritten(self, svc):
upstream = _make_mock_stream_response(
status_code=200,
aiter_lines=lambda: _async_iter_lines(SAMPLE_MANIFEST),
)
lines = []
async for line in svc.rewrite_manifest("https://example.com/video.m3u8", upstream):
lines.append(line)
assert lines[0] == "#EXTM3U\n"
assert lines[1] == "#EXT-X-VERSION:3\n"
assert lines[2] == "#EXT-X-TARGETDURATION:6\n"
assert lines[3] == "#EXT-X-MEDIA-SEQUENCE:0\n"
assert lines[4] == "#EXTINF:6.000,\n"
assert "/api/v1/youtube/proxy/segment.ts?url=" in lines[5]
assert lines[6] == "#EXTINF:6.000,\n"
assert "/api/v1/youtube/proxy/segment.ts?url=" in lines[7]
assert lines[8] == "#EXT-X-ENDLIST\n"
@pytest.mark.asyncio
async def test_master_manifest_with_variants(self, svc):
master = """#EXTM3U
#EXT-X-STREAM-INF:BANDWIDTH=800000,RESOLUTION=640x360
variant_360p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1400000,RESOLUTION=842x480
variant_480p.m3u8
"""
upstream = _make_mock_stream_response(
status_code=200,
aiter_lines=lambda: _async_iter_lines(master),
)
lines = [line async for line in svc.rewrite_manifest("https://example.com/master.m3u8", upstream)]
assert "#EXT-X-STREAM-INF" in lines[1]
assert "/api/v1/youtube/proxy/manifest.m3u8?url=" in lines[2]
assert "/api/v1/youtube/proxy/manifest.m3u8?url=" in lines[4]
# ---------------------------------------------------------------------------
# Integration: Segment proxying with mocked httpx
# ---------------------------------------------------------------------------
class TestSegmentProxying:
@pytest.fixture
def svc(self):
from app.services.hls_proxy import HLSProxyService
return HLSProxyService()
@pytest.mark.asyncio
async def test_proxy_segment_returns_streaming_response(self, svc):
resp_mock = _make_mock_stream_response(
status_code=200,
headers={"content-type": "video/mp2t"},
aiter_bytes=lambda: _async_iter_bytes([b"\x47"] * 100),
)
client_mock = _make_mock_client(resp_mock)
with patch("app.services.hls_proxy.httpx.AsyncClient", return_value=client_mock):
from fastapi.responses import StreamingResponse
result = await svc.proxy_segment("https://cdn.example.com/0.ts")
assert isinstance(result, StreamingResponse)
assert result.media_type == "video/mp2t"
assert result.headers.get("access-control-allow-origin") == "*"
# ---------------------------------------------------------------------------
# Integration: Route tests
# ---------------------------------------------------------------------------
class TestProxyRoutes:
@pytest.fixture
def proxy_client(self):
from app.routers.youtube import router
from app.core.config import get_settings
get_settings.cache_clear()
from fastapi import FastAPI
from fastapi.testclient import TestClient
app = FastAPI()
app.include_router(router, prefix="/api/v1")
return TestClient(app)
def test_manifest_proxy_returns_cors_header(self, proxy_client):
upstream = _make_mock_stream_response(
status_code=200,
aiter_lines=lambda: _async_iter_lines("#EXTM3U\n#EXT-X-ENDLIST\n"),
)
with patch("app.routers.youtube.httpx.AsyncClient") as mock_client_cls:
mock_client = _make_mock_client(upstream)
mock_client_cls.return_value = mock_client
from urllib.parse import quote
encoded_url = quote("https://example.com/video.m3u8", safe="")
resp = proxy_client.get(f"/api/v1/youtube/proxy/manifest.m3u8?url={encoded_url}")
assert resp.status_code == 200
assert resp.headers.get("access-control-allow-origin") == "*"
def test_segment_proxy_returns_correct_content_type(self, proxy_client):
resp_mock = _make_mock_stream_response(
status_code=200,
headers={"content-type": "video/mp2t"},
aiter_bytes=lambda: _async_iter_bytes([b"\x47"] * 50),
)
client_mock = _make_mock_client(resp_mock)
with patch("app.services.hls_proxy.httpx.AsyncClient", return_value=client_mock):
from urllib.parse import quote
encoded_url = quote("https://cdn.example.com/0.ts", safe="")
resp = proxy_client.get(f"/api/v1/youtube/proxy/segment.ts?url={encoded_url}")
assert resp.status_code == 200
assert resp.headers.get("access-control-allow-origin") == "*"
assert resp.headers.get("content-type") == "video/mp2t"
def test_proxy_missing_url_parameter_returns_422(self, proxy_client):
resp = proxy_client.get("/api/v1/youtube/proxy/manifest.m3u8")
assert resp.status_code == 422
def test_proxy_upstream_404_returns_502(self, proxy_client):
upstream = _make_mock_stream_response(status_code=404)
with patch("app.routers.youtube.httpx.AsyncClient") as mock_client_cls:
mock_client = _make_mock_client(upstream)
mock_client_cls.return_value = mock_client
from urllib.parse import quote
encoded_url = quote("https://cdn.example.com/missing.ts", safe="")
resp = proxy_client.get(f"/api/v1/youtube/proxy/manifest.m3u8?url={encoded_url}")
# Route checks upstream status before streaming → raises 502
assert resp.status_code == 502
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
async def _async_iter_lines(text: str):
for line in text.split("\n"):
yield line
async def _async_iter_bytes(chunks: list[bytes]):
for chunk in chunks:
yield chunk

477
backend/app/test/test_phase3_youtube_extract.py
View File

@ -1,477 +0,0 @@
"""Phase 3.2 tests: YouTube URL extraction via yt-dlp.
Covers:
- POST /api/v1/youtube/extract — VOD, live, upcoming, invalid URL
- Format selection: video-only ≤480p, best audio, HLS preference
- URL caching: in-memory with TTL, expiry triggers re-extract
- Proxy URL construction: upstream URL encoded in query param
- Error handling: DownloadError → 400, timeout → 504, disabled → 503
All yt-dlp external calls are mocked.
"""
import time
from unittest.mock import MagicMock, patch
import pytest
from fastapi import FastAPI
from fastapi.testclient import TestClient
# ---------------------------------------------------------------------------
# Helpers — fake yt-dlp format data
# ---------------------------------------------------------------------------
def _make_format(
format_id: str,
height: int | None = None,
vcodec: str = "none",
acodec: str = "none",
ext: str = "mp4",
protocol: str = "https",
url: str = "",
abr: float | None = None,
tbr: float | None = None,
resolution: str | None = None,
) -> dict:
return {
"format_id": format_id,
"height": height,
"width": height * 16 // 9 if height else None,
"vcodec": vcodec,
"acodec": acodec,
"ext": ext,
"protocol": protocol,
"url": url or f"https://example.com/{format_id}.{ext}",
"abr": abr,
"tbr": tbr,
"resolution": resolution or (f"{height * 16 // 9}x{height}" if height else None),
}
def _vod_info(video_id: str = "abc123") -> dict:
return {
"id": video_id,
"title": "Test VOD Video",
"thumbnail": "https://i.ytimg.com/vi/abc123/hqdefault.jpg",
"live_status": "not_live",
"duration": 300,
"formats": [
_make_format("137", height=1080, vcodec="avc1.640028", acodec="none", tbr=5000),
_make_format("136", height=720, vcodec="avc1.640028", acodec="none", tbr=2500),
_make_format("135", height=480, vcodec="avc1.640028", acodec="none", tbr=1200),
_make_format("134", height=360, vcodec="avc1.640028", acodec="none", tbr=600),
_make_format("133", height=240, vcodec="avc1.640028", acodec="none", tbr=300),
_make_format("140", acodec="mp4a.40.2", vcodec="none", abr=128),
_make_format("251", acodec="opus", vcodec="none", abr=160),
_make_format("18", height=360, vcodec="avc1.42001E", acodec="mp4a.40.2", tbr=500),
],
}
def _vod_info_hls(video_id: str = "abc123") -> dict:
return {
"id": video_id,
"title": "Test VOD with HLS",
"thumbnail": "https://i.ytimg.com/vi/abc123/hqdefault.jpg",
"live_status": "not_live",
"duration": 600,
"formats": [
_make_format("136", height=720, vcodec="avc1.640028", acodec="none", ext="m3u8", protocol="m3u8_native", tbr=2500),
_make_format("135", height=480, vcodec="avc1.640028", acodec="none", ext="m3u8", protocol="m3u8_native", tbr=1200),
_make_format("140", acodec="mp4a.40.2", vcodec="none", ext="m3u8", protocol="m3u8_native", abr=128),
],
}
def _live_info(video_id: str = "live999") -> dict:
return {
"id": video_id,
"title": "Live Stream Test",
"thumbnail": "https://i.ytimg.com/vi/live999/hqdefault_live.jpg",
"live_status": "is_live",
"duration": None,
"formats": [
_make_format("91", height=144, vcodec="avc1.42C00B", acodec="mp4a.40.5", ext="mp4", protocol="m3u8_native"),
_make_format("92", height=240, vcodec="avc1.4D4015", acodec="mp4a.40.5", ext="mp4", protocol="m3u8_native"),
_make_format("93", height=360, vcodec="avc1.4D401E", acodec="mp4a.40.2", ext="mp4", protocol="m3u8_native"),
_make_format("94", height=480, vcodec="avc1.4D401F", acodec="mp4a.40.2", ext="mp4", protocol="m3u8_native", tbr=1200),
_make_format("95", height=720, vcodec="avc1.4D401F", acodec="mp4a.40.2", ext="mp4", protocol="m3u8_native"),
],
}
def _upcoming_info(video_id: str = "up999") -> dict:
return {
"id": video_id,
"title": "Upcoming Stream",
"thumbnail": "https://i.ytimg.com/vi/up999/hqdefault.jpg",
"live_status": "is_upcoming",
"duration": None,
"formats": [],
}
def _private_info(video_id: str = "priv99") -> dict:
import yt_dlp
raise yt_dlp.utils.DownloadError("Private video. Sign in if you've been granted access to this video")
# ---------------------------------------------------------------------------
# Mock helpers
# ---------------------------------------------------------------------------
def _make_mock_ydl(return_value: dict | Exception) -> MagicMock:
"""Build a mock yt_dlp.YoutubeDL context manager with .extract_info."""
mock_instance = MagicMock()
if isinstance(return_value, Exception):
mock_instance.extract_info.side_effect = return_value
else:
mock_instance.extract_info.return_value = return_value
mock_ydl = MagicMock()
mock_ydl.__enter__.return_value = mock_instance
mock_ydl.__exit__.return_value = None
return mock_ydl
# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------
@pytest.fixture
def youtube_client(monkeypatch):
"""FastAPI TestClient with youtube router mounted, cached settings cleared."""
from app.routers.youtube import router
from app.core.config import get_settings
get_settings.cache_clear()
monkeypatch.setenv("YOUTUBE_PROXY_ENABLED", "true")
get_settings.cache_clear()
app = FastAPI()
app.include_router(router, prefix="/api/v1")
return TestClient(app)
# ---------------------------------------------------------------------------
# Unit: Format selection
# ---------------------------------------------------------------------------
class TestFormatSelection:
def test_selects_best_video_at_or_under_480p(self):
from app.services.youtube_service import YouTubeService
svc = YouTubeService(timeout=30, cache_ttl=300)
formats = _vod_info()["formats"]
video, audio = svc._select_best_formats(formats)
assert video is not None
assert audio is not None
assert video["height"] == 480
assert video["vcodec"] != "none"
assert video["acodec"] == "none"
assert audio["acodec"] != "none"
assert audio["vcodec"] == "none"
def test_falls_back_to_lowest_video_if_no_480p(self):
from app.services.youtube_service import YouTubeService
svc = YouTubeService(timeout=30, cache_ttl=300)
formats = [
_make_format("137", height=1080, vcodec="avc1", acodec="none", tbr=5000),
_make_format("136", height=720, vcodec="avc1", acodec="none", tbr=2500),
_make_format("140", acodec="mp4a", vcodec="none", abr=128),
]
video, audio = svc._select_best_formats(formats)
assert video is not None
assert video["height"] == 720 # Lowest available (no ≤480p exist)
def test_selects_highest_bitrate_audio(self):
from app.services.youtube_service import YouTubeService
svc = YouTubeService(timeout=30, cache_ttl=300)
formats = [
_make_format("137", height=480, vcodec="avc1", acodec="none", tbr=1200),
_make_format("140", acodec="mp4a", vcodec="none", abr=128),
_make_format("251", acodec="opus", vcodec="none", abr=160),
_make_format("250", acodec="opus", vcodec="none", abr=64),
]
video, audio = svc._select_best_formats(formats)
assert audio is not None
assert audio["format_id"] == "251" # Highest abr
def test_no_formats_raises(self):
from app.services.youtube_service import YouTubeService
svc = YouTubeService(timeout=30, cache_ttl=300)
with pytest.raises(ValueError, match="No streamable formats"):
svc._select_best_formats([])
def test_no_video_only_formats_falls_back_to_combined(self):
from app.services.youtube_service import YouTubeService
svc = YouTubeService(timeout=30, cache_ttl=300)
formats = [
_make_format("18", height=360, vcodec="avc1", acodec="mp4a", tbr=500),
_make_format("140", acodec="mp4a", vcodec="none", abr=128),
]
video, audio = svc._select_best_formats(formats)
# Fallback: combined format as video
assert video is not None
assert video["format_id"] == "18"
assert audio is not None
def test_hls_preference_for_live(self):
from app.services.youtube_service import YouTubeService
svc = YouTubeService(timeout=30, cache_ttl=300)
formats = [
_make_format("135", height=480, vcodec="avc1", acodec="none", ext="mp4", protocol="https", tbr=1200),
_make_format("301", height=480, vcodec="avc1", acodec="none", ext="m3u8", protocol="m3u8_native", tbr=1200),
_make_format("140", acodec="mp4a", vcodec="none", ext="m3u8", protocol="m3u8_native", abr=128),
]
video, audio = svc._select_best_formats(formats)
assert video["protocol"] == "m3u8_native"
assert audio["protocol"] == "m3u8_native"
def test_combined_only_all_combined_formats(self):
from app.services.youtube_service import YouTubeService
svc = YouTubeService(timeout=30, cache_ttl=300)
formats = [
_make_format("93", height=360, vcodec="avc1", acodec="mp4a", ext="mp4", protocol="m3u8_native"),
_make_format("94", height=480, vcodec="avc1", acodec="mp4a", ext="mp4", protocol="m3u8_native"),
_make_format("95", height=720, vcodec="avc1", acodec="mp4a", ext="mp4", protocol="m3u8_native"),
_make_format("96", height=1080, vcodec="avc1", acodec="mp4a", ext="mp4", protocol="m3u8_native"),
]
video, audio = svc._select_best_formats(formats)
assert video["height"] == 480
assert audio["height"] == 480
assert video["url"] == audio["url"]
# ---------------------------------------------------------------------------
# Integration: Route + mocked yt-dlp
# ---------------------------------------------------------------------------
class TestYouTubeExtractVOD:
def test_extract_vod_returns_proxy_urls(self, youtube_client):
mock_ydl = _make_mock_ydl(_vod_info("abc123"))
with patch("app.services.youtube_service.yt_dlp.YoutubeDL", return_value=mock_ydl):
resp = youtube_client.post(
"/api/v1/youtube/extract",
json={"url": "https://www.youtube.com/watch?v=abc123"},
)
assert resp.status_code == 200
data = resp.json()
assert data["video_id"] == "abc123"
assert data["title"] == "Test VOD Video"
assert data["is_live"] is False
assert data["is_upcoming"] is False
assert data["video_proxy_url"] is not None
assert data["audio_proxy_url"] is not None
assert data["video_proxy_url"].startswith("/api/v1/youtube/proxy/")
assert data["thumbnail_url"] == "https://i.ytimg.com/vi/abc123/hqdefault.jpg"
assert len(data["formats"]) > 0
def test_extract_vod_hls_returns_manifest_proxy_urls(self, youtube_client):
mock_ydl = _make_mock_ydl(_vod_info_hls("abc123"))
with patch("app.services.youtube_service.yt_dlp.YoutubeDL", return_value=mock_ydl):
resp = youtube_client.post(
"/api/v1/youtube/extract",
json={"url": "https://www.youtube.com/watch?v=abc123"},
)
assert resp.status_code == 200
data = resp.json()
assert "manifest.m3u8?url=" in data["video_proxy_url"]
assert "manifest.m3u8?url=" in data["audio_proxy_url"]
def test_error_field_is_none_on_success(self, youtube_client):
mock_ydl = _make_mock_ydl(_vod_info())
with patch("app.services.youtube_service.yt_dlp.YoutubeDL", return_value=mock_ydl):
resp = youtube_client.post(
"/api/v1/youtube/extract",
json={"url": "https://www.youtube.com/watch?v=abc123"},
)
assert resp.status_code == 200
assert resp.json()["error"] is None
class TestYouTubeExtractLive:
def test_extract_live_returns_is_live_true(self, youtube_client):
mock_ydl = _make_mock_ydl(_live_info())
with patch("app.services.youtube_service.yt_dlp.YoutubeDL", return_value=mock_ydl):
resp = youtube_client.post(
"/api/v1/youtube/extract",
json={"url": "https://www.youtube.com/watch?v=live999"},
)
assert resp.status_code == 200
data = resp.json()
assert data["video_id"] == "live999"
assert data["is_live"] is True
assert data["is_upcoming"] is False
assert data["video_proxy_url"] is not None
assert data["audio_proxy_url"] is not None
def test_live_combined_format_same_url_for_both(self, youtube_client):
mock_ydl = _make_mock_ydl(_live_info("combined_test"))
with patch("app.services.youtube_service.yt_dlp.YoutubeDL", return_value=mock_ydl):
resp = youtube_client.post(
"/api/v1/youtube/extract",
json={"url": "https://www.youtube.com/watch?v=combined_test"},
)
assert resp.status_code == 200
data = resp.json()
assert data["is_live"] is True
assert data["video_proxy_url"] == data["audio_proxy_url"]
class TestYouTubeExtractUpcoming:
def test_extract_upcoming_returns_is_upcoming_true(self, youtube_client):
mock_ydl = _make_mock_ydl(_upcoming_info())
with patch("app.services.youtube_service.yt_dlp.YoutubeDL", return_value=mock_ydl):
resp = youtube_client.post(
"/api/v1/youtube/extract",
json={"url": "https://www.youtube.com/watch?v=up999"},
)
assert resp.status_code == 200
data = resp.json()
assert data["video_id"] == "up999"
assert data["is_upcoming"] is True
assert data["is_live"] is False
assert data["video_proxy_url"] is None
assert data["audio_proxy_url"] is None
class TestYouTubeExtractErrors:
def test_private_video_returns_error_field(self, youtube_client):
import yt_dlp
exc = yt_dlp.utils.DownloadError("Private video")
mock_ydl = _make_mock_ydl(exc)
with patch("app.services.youtube_service.yt_dlp.YoutubeDL", return_value=mock_ydl):
resp = youtube_client.post(
"/api/v1/youtube/extract",
json={"url": "https://www.youtube.com/watch?v=priv99"},
)
assert resp.status_code == 200
data = resp.json()
assert data["error"] is not None
assert "Private video" in data["error"]
def test_po_token_error_invalidates_cache(self, monkeypatch):
import yt_dlp
from app.services.youtube_service import YouTubeService, _is_po_token_error
svc = YouTubeService(timeout=30, cache_ttl=300)
url = "https://www.youtube.com/watch?v=potest"
# Seed cache with a valid entry
svc._cache[url] = (100.0, {"video_id": "cached", "title": "Cached"})
# Mock yt-dlp to raise PO token error
exc = yt_dlp.utils.DownloadError("Sign in to confirm you're not a bot")
mock_ydl = _make_mock_ydl(exc)
with patch("app.services.youtube_service.yt_dlp.YoutubeDL", return_value=mock_ydl):
import asyncio
result = asyncio.new_event_loop().run_until_complete(svc.extract_streams(url))
assert result["error"] is not None
assert "not a bot" in result["error"]
# Cache should be invalidated — next extract would re-attempt
assert url not in svc._cache
def test_is_po_token_error_detection(self):
from app.services.youtube_service import _is_po_token_error
assert _is_po_token_error("Sign in to confirm you're not a bot")
assert _is_po_token_error("ERROR: [youtube] PO Token expired")
assert _is_po_token_error("bot detection triggered for this request")
assert not _is_po_token_error("Video unavailable")
assert not _is_po_token_error("Private video")
def test_disabled_proxy_returns_503(self, monkeypatch, youtube_client):
monkeypatch.setenv("YOUTUBE_PROXY_ENABLED", "false")
from app.core.config import get_settings
get_settings.cache_clear()
resp = youtube_client.post(
"/api/v1/youtube/extract",
json={"url": "https://www.youtube.com/watch?v=abc123"},
)
assert resp.status_code == 503
class TestURLCaching:
def test_cached_result_not_re_extracted(self, youtube_client):
mock_ydl = _make_mock_ydl(_vod_info("cached1"))
instance = mock_ydl.__enter__.return_value
with patch("app.services.youtube_service.yt_dlp.YoutubeDL", return_value=mock_ydl):
r1 = youtube_client.post(
"/api/v1/youtube/extract",
json={"url": "https://www.youtube.com/watch?v=cached1"},
)
r2 = youtube_client.post(
"/api/v1/youtube/extract",
json={"url": "https://www.youtube.com/watch?v=cached1"},
)
assert r1.status_code == 200
assert r2.status_code == 200
assert r1.json()["video_id"] == r2.json()["video_id"]
assert instance.extract_info.call_count == 1 # Cached, not called twice
def test_cache_expiry_triggers_re_extract(self, monkeypatch):
from app.services.youtube_service import YouTubeService
svc = YouTubeService(timeout=30, cache_ttl=0) # 0 TTL = immediate expiry
mock_ydl = _make_mock_ydl(_vod_info("exp1"))
instance = mock_ydl.__enter__.return_value
with patch("app.services.youtube_service.yt_dlp.YoutubeDL", return_value=mock_ydl):
import asyncio
asyncio.run(svc.extract_streams("https://www.youtube.com/watch?v=exp1"))
# Cache should be set but TTL=0 means expired
asyncio.run(svc.extract_streams("https://www.youtube.com/watch?v=exp1"))
assert instance.extract_info.call_count == 2
class TestProxyURLConstruction:
def test_proxy_url_encodes_upstream_url(self):
from app.services.youtube_service import YouTubeService
from urllib.parse import quote, unquote
svc = YouTubeService(timeout=30, cache_ttl=300)
upstream = "https://manifest.googlevideo.com/123/hls_playlist.m3u8?id=abc&key=def"
proxy = svc._build_proxy_url(upstream)
assert proxy.startswith("/api/v1/youtube/proxy/manifest.m3u8?url=")
# Extract and decode the URL parameter
encoded = proxy.split("url=", 1)[1]
decoded = unquote(encoded)
assert decoded == upstream

1
backend/requirements.txt
View File

@ -19,4 +19,3 @@ langchain-openai>=1.1.11,<1.2.0
dashscope>=0.4.0 dashscope>=0.4.0
aiofiles>=24.0.0 aiofiles>=24.0.0
zhconv>=1.4.0 zhconv>=1.4.0
yt-dlp>=2024.0.0

45
development_plan.md
View File

@ -135,51 +135,14 @@ User Question
--- ---
## Phase 3: YouTube Live Stream Proxy → ASR (5-6 days) ✅ Complete
### Overview
Proxy YouTube live streams and VODs through the backend, route audio into the existing ASR pipeline.
### Backend Additions
- YouTube URL extraction via yt-dlp (`POST /api/v1/youtube/extract`)
- Format selection: video-only ≤480p + best audio (VOD), combined HLS (live)
- HLS manifest proxy with line-by-line rewriting (`GET /api/v1/youtube/proxy/manifest.m3u8`)
- TS segment proxying with CORS headers (`GET /api/v1/youtube/proxy/segment.ts`)
- In-memory caching: 5 min TTL (live), 30 min TTL (VOD)
- PO token expiration detection with cache invalidation
### Frontend Additions
- YouTubeInput component: URL validation, extraction, loading/error states
- YouTubeVideoPlayer component: dual hls.js (video + hidden audio), thumbnail placeholder, LIVE badge
- useYouTubeASR hook: AudioContext from audio element → WebSocket → DashScope ASR
- LTTPage source toggle: Upload / YouTube tabs
- hls.js integration with dynamic import and quality capping (≤480p)
### Key Design Decisions
- No iOS client needed (default yt-dlp extractor handles both VOD and live)
- Dual-element architecture: `<video muted>` for display, `<audio hidden>` for AudioContext capture
- HLS proxy rewrites all URLs (segments, sub-manifests, EXT-X-KEY URIs)
- Upstream status checked BEFORE streaming (avoids "response already started" errors)
- Both useVideoASR and useYouTubeASR return identical shapes for transparent integration
### Architecture
```
YouTube URL → yt-dlp extract → HLS proxy → hls.js (video + audio)
↓
AudioContext → WebSocket → DashScope ASR → transcript
```
---
## Development Timeline ## Development Timeline
| Phase | Duration | Key Deliverables | Status | | Phase | Duration | Key Deliverables | Status |
|-----------------------------|--------------|------------------|--------| |-----------------------------|--------------|------------------|--------|
| Setup + Phase 1 Backend | 3-4 days | FastAPI + Chroma + Metadata + LLM client | ✅ Complete | | Setup + Phase 1 Backend | 3-4 days | FastAPI + Chroma + Metadata + LLM client | ✅ Complete |
| Phase 1 Frontend | 2-3 days | UI layout + text query flow | ✅ Complete | | Phase 1 Frontend | 2-3 days | UI layout + text query flow | ✅ Complete |
| Phase 2 Backend | 4-5 days | Video upload + WebSocket ASR + question extraction | ✅ Complete | | Phase 2 Backend | 4-5 days | Video upload + WebSocket ASR + question extraction | ⬜ Next |
| Phase 2 Frontend | 3-4 days | Video player + live transcript + auto/manual flow | ✅ Complete | | Phase 2 Frontend | 3-4 days | Video player + live transcript + auto/manual flow | ⬜ Pending |
| Phase 3 YouTube Proxy | 5-6 days | yt-dlp extraction + HLS proxy + YouTube ASR | ✅ Complete |
| Testing & Polish | 1-2 days | End-to-end testing + deployment scripts | ⬜ Pending | | Testing & Polish | 1-2 days | End-to-end testing + deployment scripts | ⬜ Pending |
**Total Estimated Effort**: 13-17 developer days (2-3 weeks) **Total Estimated Effort**: 13-17 developer days (2-3 weeks)
@ -201,5 +164,5 @@ YouTube URL → yt-dlp extract → HLS proxy → hls.js (video + audio)
**File Information** **File Information**
- Filename: `development_plan.md` - Filename: `development_plan.md`
- Last Updated: May 2026 - Last Updated: April 2026
- Status: Phase 1-3 Complete — YouTube proxy feature live - Status: Phase 1 Backend ✅, Phase 1 Frontend ✅ — Phase 2 next

7
frontend/package-lock.json generated
View File

@ -11,7 +11,6 @@
"@tanstack/react-query": "^5.0.0", "@tanstack/react-query": "^5.0.0",
"autoprefixer": "^10.5.0", "autoprefixer": "^10.5.0",
"axios": "^1.6.0", "axios": "^1.6.0",
"hls.js": "^1.6.16",
"lucide-react": "^0.190.0", "lucide-react": "^0.190.0",
"pdfjs-dist": "^5.6.205", "pdfjs-dist": "^5.6.205",
"react": "^18.2.0", "react": "^18.2.0",
@ -3802,12 +3801,6 @@
"url": "https://opencollective.com/unified" "url": "https://opencollective.com/unified"
} }
}, },
"node_modules/hls.js": {
"version": "1.6.16",
"resolved": "https://registry.npmjs.org/hls.js/-/hls.js-1.6.16.tgz",
"integrity": "sha512-VSIRpLfRwlAAdGL4wiTucx2ScRipo0ed1FBatWkyt832jC4CReKstga6yIhYVwGu9LOBjuX9wzmRMeQdBJtzEA==",
"license": "Apache-2.0"
},
"node_modules/html-encoding-sniffer": { "node_modules/html-encoding-sniffer": {
"version": "3.0.0", "version": "3.0.0",
"resolved": "https://registry.npmjs.org/html-encoding-sniffer/-/html-encoding-sniffer-3.0.0.tgz", "resolved": "https://registry.npmjs.org/html-encoding-sniffer/-/html-encoding-sniffer-3.0.0.tgz",

1
frontend/package.json
View File

@ -13,7 +13,6 @@
"@tanstack/react-query": "^5.0.0", "@tanstack/react-query": "^5.0.0",
"autoprefixer": "^10.5.0", "autoprefixer": "^10.5.0",
"axios": "^1.6.0", "axios": "^1.6.0",
"hls.js": "^1.6.16",
"lucide-react": "^0.190.0", "lucide-react": "^0.190.0",
"pdfjs-dist": "^5.6.205", "pdfjs-dist": "^5.6.205",
"react": "^18.2.0", "react": "^18.2.0",

110
frontend/src/components/YouTubeInput.tsx
View File

@ -1,110 +0,0 @@
import React, { useState } from 'react'
import { Loader2, AlertCircle, Play } from 'lucide-react'
import { useYouTubeExtract } from '../lib/queries'
import type { YouTubeStreamResponse } from '../types'
export interface YouTubeInputProps {
onExtractSuccess: (data: YouTubeStreamResponse) => void
isDisabled?: boolean
}
const YOUTUBE_URL_REGEX = /^(https?:\/\/)?(www\.|m\.)?(youtube\.com\/watch\?v=|youtube\.com\/live\/|youtube\.com\/shorts\/|youtu\.be\/)[\w-]+(&.*)?$/i
function isValidYouTubeUrl(url: string): boolean {
return YOUTUBE_URL_REGEX.test(url.trim())
}
export const YouTubeInput: React.FC<YouTubeInputProps> = ({ onExtractSuccess, isDisabled }) => {
const [url, setUrl] = useState('')
const [validationError, setValidationError] = useState<string | null>(null)
const extract = useYouTubeExtract()
const handleSubmit = (e: React.FormEvent) => {
e.preventDefault()
setValidationError(null)
extract.reset()
const trimmed = url.trim()
if (!trimmed) {
setValidationError('Please enter a YouTube URL')
return
}
if (!isValidYouTubeUrl(trimmed)) {
setValidationError('Please enter a valid YouTube URL (e.g., youtube.com/watch?v=... or youtu.be/...)')
return
}
extract.mutate(trimmed, {
onSuccess: (data: YouTubeStreamResponse) => {
if (data.error) {
setValidationError(data.error)
return
}
onExtractSuccess(data)
},
onError: (err: Error) => {
setValidationError(err.message)
},
})
}
const isLoading = extract.isPending
const hasError = validationError || extract.isError
const errorMessage = validationError || (extract.error?.message ?? null)
return (
<form onSubmit={handleSubmit} className="space-y-3">
<div className="flex gap-2">
<input
data-testid="youtube-url-input"
type="text"
value={url}
onChange={(e) => {
setUrl(e.target.value)
if (validationError) setValidationError(null)
}}
placeholder="Paste YouTube URL..."
disabled={isDisabled || isLoading}
className="flex-1 rounded border border-gray-300 px-3 py-2 text-sm focus:outline-none focus:ring-2 focus:ring-blue-500 focus:border-transparent disabled:bg-gray-100 disabled:cursor-not-allowed"
/>
<button
data-testid="youtube-load-btn"
type="submit"
disabled={isDisabled || isLoading || !url.trim()}
className="shrink-0 px-4 py-2 bg-blue-600 text-white font-medium rounded hover:bg-blue-700 focus:outline-none focus:ring-2 focus:ring-blue-500 focus:ring-offset-2 disabled:opacity-50 disabled:cursor-not-allowed disabled:hover:bg-blue-600 transition-all duration-200 flex items-center gap-2"
>
{isLoading ? (
<>
<Loader2 className="w-4 h-4 animate-spin" />
Loading...
</>
) : (
<>
<Play className="w-4 h-4" />
Load Stream
</>
)}
</button>
</div>
{isLoading && (
<div data-testid="youtube-loading" className="flex items-center gap-2 text-sm text-gray-600">
<Loader2 className="w-4 h-4 text-blue-600 animate-spin" />
Extracting stream info...
</div>
)}
{hasError && errorMessage && (
<div
data-testid="youtube-error"
className="p-3 bg-red-50 border border-red-200 rounded-lg flex items-start gap-2"
>
<AlertCircle className="w-4 h-4 text-red-500 shrink-0 mt-0.5" />
<div className="text-sm text-red-700">{errorMessage}</div>
</div>
)}
</form>
)
}

207
frontend/src/components/YouTubeVideoPlayer.tsx
View File

@ -1,207 +0,0 @@
import React, { forwardRef, useRef, useState, useEffect, useCallback } from 'react'
import { Loader2, AlertCircle } from 'lucide-react'
export interface YouTubeVideoPlayerProps {
videoProxyUrl: string
audioProxyUrl: string
thumbnailUrl: string | null
isLive: boolean
onAudioReady?: (audioElement: HTMLAudioElement) => void
}
const HLS_CONFIG = {
capLevelToPlayerSize: true,
maxBufferLength: 30,
maxMaxBufferLength: 60,
}
function capLevelsTo480(hls: any) {
if (!hls || !hls.levels) return
const maxIndex = hls.levels.length - 1
let capIndex = maxIndex
for (let i = maxIndex; i >= 0; i--) {
const level = hls.levels[i]
if (level.height && level.height <= 480) {
capIndex = i
break
}
}
hls.autoLevelCapping = capIndex
}
export const YouTubeVideoPlayer = forwardRef<HTMLVideoElement, YouTubeVideoPlayerProps>(
({ videoProxyUrl, audioProxyUrl, thumbnailUrl, isLive, onAudioReady }, videoRef) => {
const internalVideoRef = useRef<HTMLVideoElement | null>(null)
const audioRef = useRef<HTMLAudioElement | null>(null)
const [isLoading, setIsLoading] = useState(true)
const [hasError, setHasError] = useState(false)
const [hasStarted, setHasStarted] = useState(false)
const setVideoRef = useCallback(
(node: HTMLVideoElement | null) => {
internalVideoRef.current = node
if (typeof videoRef === 'function') {
videoRef(node)
} else if (videoRef) {
(videoRef as React.MutableRefObject<HTMLVideoElement | null>).current = node
}
},
[videoRef]
)
useEffect(() => {
if (audioRef.current && onAudioReady) {
onAudioReady(audioRef.current)
}
}, [onAudioReady])
useEffect(() => {
let videoHls: any = null
let audioHls: any = null
let destroyed = false
const initHls = async () => {
const Hls = (await import('hls.js')).default
if (destroyed) return
if (!Hls.isSupported()) {
setHasError(true)
setIsLoading(false)
return
}
const videoEl = internalVideoRef.current
const audioEl = audioRef.current
if (!videoEl || !audioEl) return
videoHls = new Hls(HLS_CONFIG)
videoHls.loadSource(videoProxyUrl)
videoHls.attachMedia(videoEl)
videoHls.on(Hls.Events.MANIFEST_PARSED, () => {
capLevelsTo480(videoHls)
})
videoHls.on(Hls.Events.ERROR, (_event: any, data: any) => {
if (data.fatal) {
setHasError(true)
setIsLoading(false)
}
})
audioHls = new Hls(HLS_CONFIG)
audioHls.loadSource(audioProxyUrl)
audioHls.attachMedia(audioEl)
audioHls.on(Hls.Events.ERROR, (_event: any, data: any) => {
if (data.fatal) {
setHasError(true)
setIsLoading(false)
}
})
const handleCanPlay = () => {
setIsLoading(false)
}
videoEl.addEventListener('canplay', handleCanPlay)
return () => {
videoEl.removeEventListener('canplay', handleCanPlay)
}
}
initHls()
return () => {
destroyed = true
if (videoHls) {
videoHls.destroy()
}
if (audioHls) {
audioHls.destroy()
}
}
}, [videoProxyUrl, audioProxyUrl])
const handlePlay = () => {
setHasStarted(true)
const audioEl = audioRef.current
if (audioEl && audioEl.paused) {
audioEl.play().catch(() => {})
}
}
const handlePause = () => {
const audioEl = audioRef.current
if (audioEl && !audioEl.paused) {
audioEl.pause()
}
}
if (hasError) {
return (
<div
data-testid="youtube-error"
className="w-full max-h-60 rounded-lg bg-gray-100 border border-gray-200 flex flex-col items-center justify-center p-6 min-h-[200px]"
>
<AlertCircle className="w-10 h-10 text-red-500 mb-2" />
<div className="text-sm text-gray-600 font-medium">Failed to load stream</div>
<div className="text-xs text-gray-400 mt-1">Please try reloading the page.</div>
</div>
)
}
return (
<div className="relative w-full">
{isLive && (
<div
data-testid="youtube-live-badge"
className="absolute top-2 left-2 z-20 px-2 py-0.5 bg-red-600 text-white text-xs font-bold rounded"
>
LIVE
</div>
)}
{isLoading && (
<div
data-testid="youtube-loading"
className="absolute inset-0 z-10 bg-gray-100 rounded-lg flex flex-col items-center justify-center"
>
<Loader2 className="w-8 h-8 text-blue-600 animate-spin mb-2" />
<div className="text-sm text-gray-600">Loading stream...</div>
</div>
)}
{thumbnailUrl && !hasStarted && (
<img
data-testid="youtube-thumbnail"
src={thumbnailUrl}
alt="Video thumbnail"
className="absolute inset-0 z-[5] w-full max-h-60 rounded-lg object-cover bg-black"
/>
)}
<video
ref={setVideoRef}
data-testid="youtube-video"
controls
muted
crossOrigin="anonymous"
className="w-full max-h-60 rounded-lg bg-black relative z-[1]"
onPlay={handlePlay}
onPause={handlePause}
/>
<audio
ref={audioRef}
data-testid="youtube-audio"
crossOrigin="anonymous"
className="hidden"
/>
</div>
)
}
)
YouTubeVideoPlayer.displayName = 'YouTubeVideoPlayer'

178
frontend/src/hooks/useYouTubeASR.ts
View File

@ -1,178 +0,0 @@
import { useState, useRef, useCallback, useEffect } from 'react'
import type { ASRMessage, ASRStatus } from '../types'
interface UseYouTubeASROptions {
videoId: string
videoElement: HTMLVideoElement | null
audioElement: HTMLAudioElement | null
language?: string
onFinalTranscript?: (text: string) => void
}
export function useYouTubeASR({
videoId,
videoElement,
audioElement,
language = 'yue',
onFinalTranscript,
}: UseYouTubeASROptions) {
const [transcript, setTranscript] = useState('')
const [partialTranscript, setPartialTranscript] = useState('')
const [status, setStatus] = useState<ASRStatus>('idle')
const [isStreaming, setIsStreaming] = useState(false)
const wsRef = useRef<WebSocket | null>(null)
const audioContextRef = useRef<AudioContext | null>(null)
const processorRef = useRef<ScriptProcessorNode | null>(null)
const sourceRef = useRef<MediaElementAudioSourceNode | null>(null)
const isStreamingRef = useRef(false)
const graphSetupRef = useRef(false)
const transcriptRef = useRef('')
const lastStashRef = useRef('')
const onFinalTranscriptRef = useRef(onFinalTranscript)
onFinalTranscriptRef.current = onFinalTranscript
const getWSURL = useCallback(() => {
const protocol = window.location.protocol === 'https:' ? 'wss:' : 'ws:'
const host = window.location.host
const langParam = language !== 'auto' ? `?language=${language}` : ''
const backendHost = import.meta.env.VITE_WS_HOST ?? host
return `${protocol}//${backendHost}/ws/asr/${videoId}${langParam}`
}, [videoId, language])
const connectWebSocket = useCallback(() => {
const ws = new WebSocket(getWSURL())
wsRef.current = ws
ws.onopen = () => {
isStreamingRef.current = true
setIsStreaming(true)
setStatus('streaming')
}
ws.onmessage = (e) => {
const msg: ASRMessage = JSON.parse(e.data)
if (msg.is_final && msg.full_text) {
transcriptRef.current = msg.full_text
lastStashRef.current = ''
setTranscript(msg.full_text)
setPartialTranscript('')
onFinalTranscriptRef.current?.(msg.full_text)
} else if (msg.delta) {
transcriptRef.current += msg.delta
lastStashRef.current = (msg as any).stash || ''
setTranscript(transcriptRef.current)
setPartialTranscript(transcriptRef.current)
}
}
ws.onerror = (e) => {
console.error('[useYouTubeASR] WebSocket error:', e)
setStatus('error')
}
ws.onclose = () => {
isStreamingRef.current = false
setIsStreaming(false)
setStatus('disconnected')
}
}, [getWSURL])
const closeWebSocket = useCallback(() => {
wsRef.current?.close()
wsRef.current = null
}, [])
const startStreaming = useCallback(() => {
if (!audioElement) return
try {
setStatus('connecting')
audioContextRef.current?.resume()
closeWebSocket()
connectWebSocket()
} catch (err) {
console.error('[useYouTubeASR] startStreaming failed:', err)
setStatus('error')
}
}, [audioElement, closeWebSocket, connectWebSocket])
const stopStreaming = useCallback(() => {
isStreamingRef.current = false
setIsStreaming(false)
closeWebSocket()
setStatus('idle')
let currentText = transcriptRef.current.trim()
const stash = lastStashRef.current.trim()
if (stash && !currentText.endsWith(stash)) {
currentText += stash
transcriptRef.current = currentText
}
lastStashRef.current = ''
if (currentText) {
onFinalTranscriptRef.current?.(currentText)
setPartialTranscript('')
}
}, [closeWebSocket])
useEffect(() => {
if (!audioElement || graphSetupRef.current) return
try {
const audioContext = new AudioContext({ sampleRate: 16000 })
audioContextRef.current = audioContext
const source = audioContext.createMediaElementSource(audioElement)
sourceRef.current = source
const processor = audioContext.createScriptProcessor(4096, 1, 1)
processorRef.current = processor
processor.onaudioprocess = (e) => {
const float32Data = e.inputBuffer.getChannelData(0)
const outputData = e.outputBuffer.getChannelData(0)
outputData.set(float32Data)
if (!isStreamingRef.current) return
if (!wsRef.current || wsRef.current.readyState !== WebSocket.OPEN) return
wsRef.current.send(float32Data.buffer)
}
source.connect(processor)
processor.connect(audioContext.destination)
graphSetupRef.current = true
} catch (err) {
console.error('[useYouTubeASR] audio graph setup failed:', err)
}
}, [audioElement])
useEffect(() => {
return () => {
isStreamingRef.current = false
processorRef.current?.disconnect()
sourceRef.current?.disconnect()
wsRef.current?.close()
audioContextRef.current?.close()
}
}, [])
useEffect(() => {
if (!videoElement) return
const onPlay = () => startStreaming()
const onPause = () => stopStreaming()
const onEnded = () => stopStreaming()
videoElement.addEventListener('play', onPlay)
videoElement.addEventListener('pause', onPause)
videoElement.addEventListener('ended', onEnded)
return () => {
videoElement.removeEventListener('play', onPlay)
videoElement.removeEventListener('pause', onPause)
videoElement.removeEventListener('ended', onEnded)
}
}, [videoElement, startStreaming, stopStreaming])
return {
transcript,
partialTranscript,
isStreaming,
status,
startStreaming,
stopStreaming,
}
}

7
frontend/src/lib/api.ts
View File

@ -1,5 +1,5 @@
import axios from 'axios' import axios from 'axios'
import type { QueryRequest, QueryResponse, QueryStreamEvent, IngestResponse, DocumentListResponse, ChunkInfo, DeleteResponse, PromptProfileListResponse, PromptSetResponse, PromptUpdateRequest, PromptBatchUpdateRequest, PromptActivateResponse, PromptStatusResponse, ProfileExportData, ProfileImportResponse, QueryHistoryList, QueryHistoryDetail, HistoryStats, HistoryDeleteResponse, FullTranscriptResponse, VideoUploadResponse, YouTubeStreamResponse } from '../types' import type { QueryRequest, QueryResponse, QueryStreamEvent, IngestResponse, DocumentListResponse, ChunkInfo, DeleteResponse, PromptProfileListResponse, PromptSetResponse, PromptUpdateRequest, PromptBatchUpdateRequest, PromptActivateResponse, PromptStatusResponse, ProfileExportData, ProfileImportResponse, QueryHistoryList, QueryHistoryDetail, HistoryStats, HistoryDeleteResponse, FullTranscriptResponse, VideoUploadResponse } from '../types'
const BASE_URL: string = import.meta.env.VITE_API_BASE_URL ?? 'http://localhost:8000/api/v1' const BASE_URL: string = import.meta.env.VITE_API_BASE_URL ?? 'http://localhost:8000/api/v1'
@ -174,8 +174,3 @@ export const requestFullTranscript = async (videoId: string): Promise<FullTransc
const resp = await apiClient.post<FullTranscriptResponse>(`/video/${videoId}/transcribe`) const resp = await apiClient.post<FullTranscriptResponse>(`/video/${videoId}/transcribe`)
return resp.data return resp.data
} }
export const extractYouTubeStream = async (url: string): Promise<YouTubeStreamResponse> => {
const resp = await apiClient.post<YouTubeStreamResponse>('/youtube/extract', { url })
return resp.data
}

10
frontend/src/lib/queries.tsx
View File

@ -1,7 +1,7 @@
import React from 'react' import React from 'react'
import { QueryClient, QueryClientProvider, useMutation, useQuery, useQueryClient } from '@tanstack/react-query' import { QueryClient, QueryClientProvider, useMutation, useQuery, useQueryClient } from '@tanstack/react-query'
import { queryDocument, queryDocumentStream, ingestDocument, listDocuments, listChunks, deleteDocument, deleteChunk, listPromptProfiles, getPromptProfile, activatePromptProfile, updatePrompt, updateAllPrompts, resetPrompts, exportProfile, importProfile, listQueryHistory, getQueryHistoryDetail, deleteQueryHistory, clearQueryHistory, getHistoryStats, uploadVideo, extractYouTubeStream } from './api' import { queryDocument, queryDocumentStream, ingestDocument, listDocuments, listChunks, deleteDocument, deleteChunk, listPromptProfiles, getPromptProfile, activatePromptProfile, updatePrompt, updateAllPrompts, resetPrompts, exportProfile, importProfile, listQueryHistory, getQueryHistoryDetail, deleteQueryHistory, clearQueryHistory, getHistoryStats, uploadVideo } from './api'
import type { QueryRequest, QueryResponse, QueryStreamEvent, SourceMetadata, SubQuestionSources, IngestResponse, DocumentListResponse, ChunkInfo, DeleteResponse, PromptProfileListResponse, PromptSetResponse, PromptUpdateRequest, PromptBatchUpdateRequest, PromptActivateResponse, PromptStatusResponse, ProfileExportData, ProfileImportResponse, QueryHistoryList, QueryHistoryDetail, HistoryStats, HistoryDeleteResponse, VideoUploadResponse, YouTubeStreamResponse } from '../types' import type { QueryRequest, QueryResponse, QueryStreamEvent, SourceMetadata, SubQuestionSources, IngestResponse, DocumentListResponse, ChunkInfo, DeleteResponse, PromptProfileListResponse, PromptSetResponse, PromptUpdateRequest, PromptBatchUpdateRequest, PromptActivateResponse, PromptStatusResponse, ProfileExportData, ProfileImportResponse, QueryHistoryList, QueryHistoryDetail, HistoryStats, HistoryDeleteResponse, VideoUploadResponse } from '../types'
import { useState, useCallback, useRef } from 'react' import { useState, useCallback, useRef } from 'react'
export const queryClient = new QueryClient() export const queryClient = new QueryClient()
@ -274,9 +274,3 @@ export const useVideoUpload = () => {
mutationFn: ({ file, onProgress }) => uploadVideo(file, onProgress), mutationFn: ({ file, onProgress }) => uploadVideo(file, onProgress),
}) })
} }
export const useYouTubeExtract = () => {
return useMutation<YouTubeStreamResponse, Error, string>({
mutationFn: extractYouTubeStream,
})
}

167
frontend/src/pages/LTTPage.tsx
View File

@ -1,9 +1,8 @@
import React, { useState, useCallback, useEffect } from 'react' import React, { useState, useCallback, useEffect } from 'react'
import { Loader2, AlertCircle, FileText, Upload, Youtube } from 'lucide-react' import { Loader2, AlertCircle, FileText } from 'lucide-react'
import { Group, Panel, Separator } from 'react-resizable-panels' import { Group, Panel, Separator } from 'react-resizable-panels'
import { useQueryDocumentStream } from '../lib/queries' import { useQueryDocumentStream } from '../lib/queries'
import { useVideoASR } from '../hooks/useVideoASR' import { useVideoASR } from '../hooks/useVideoASR'
import { useYouTubeASR } from '../hooks/useYouTubeASR'
import { useFullTranscript } from '../hooks/useFullTranscript' import { useFullTranscript } from '../hooks/useFullTranscript'
import { getVideoUrl } from '../lib/api' import { getVideoUrl } from '../lib/api'
import { QueryInput } from '../components/QueryInput' import { QueryInput } from '../components/QueryInput'
@ -11,23 +10,15 @@ import { ExtractedQuestionsDisplay } from '../components/ExtractedQuestionsDispl
import { ResponsePanel } from '../components/ResponsePanel' import { ResponsePanel } from '../components/ResponsePanel'
import { VideoUpload } from '../components/VideoUpload' import { VideoUpload } from '../components/VideoUpload'
import { VideoPlayer } from '../components/VideoPlayer' import { VideoPlayer } from '../components/VideoPlayer'
import { YouTubeInput } from '../components/YouTubeInput'
import { YouTubeVideoPlayer } from '../components/YouTubeVideoPlayer'
import type { YouTubeStreamResponse } from '../types'
type SourceType = 'upload' | 'youtube'
export const LTTPage: React.FC = () => { export const LTTPage: React.FC = () => {
const [source, setSource] = useState<SourceType>('upload')
const [currentVideoId, setCurrentVideoId] = useState<string | null>(null) const [currentVideoId, setCurrentVideoId] = useState<string | null>(null)
const [queryText, setQueryText] = useState('') const [queryText, setQueryText] = useState('')
const [videoEl, setVideoEl] = useState<HTMLVideoElement | null>(null) const [videoEl, setVideoEl] = useState<HTMLVideoElement | null>(null)
const [youtubeData, setYoutubeData] = useState<YouTubeStreamResponse | null>(null)
const [youtubeAudioEl, setYoutubeAudioEl] = useState<HTMLAudioElement | null>(null)
const queryStream = useQueryDocumentStream() const queryStream = useQueryDocumentStream()
const uploadASR = useVideoASR({ const asr = useVideoASR({
videoId: currentVideoId ?? '', videoId: currentVideoId ?? '',
videoElement: videoEl, videoElement: videoEl,
language: 'yue', language: 'yue',
@ -36,18 +27,6 @@ export const LTTPage: React.FC = () => {
}, },
}) })
const youtubeASR = useYouTubeASR({
videoId: youtubeData?.video_id ?? '',
videoElement: videoEl,
audioElement: youtubeAudioEl,
language: 'yue',
onFinalTranscript: (text) => {
setQueryText(text)
},
})
const asr = source === 'youtube' ? youtubeASR : uploadASR
const ft = useFullTranscript({ videoId: currentVideoId ?? '' }) const ft = useFullTranscript({ videoId: currentVideoId ?? '' })
useEffect(() => { useEffect(() => {
@ -60,24 +39,6 @@ export const LTTPage: React.FC = () => {
setCurrentVideoId(videoId) setCurrentVideoId(videoId)
}, []) }, [])
const handleYouTubeExtractSuccess = useCallback((data: YouTubeStreamResponse) => {
setYoutubeData(data)
setQueryText('')
}, [])
const handleYouTubeAudioReady = useCallback((audioEl: HTMLAudioElement) => {
setYoutubeAudioEl(audioEl)
}, [])
const handleSourceChange = useCallback((newSource: SourceType) => {
if (newSource === source) return
if (newSource === 'upload') {
setYoutubeData(null)
setYoutubeAudioEl(null)
}
setSource(newSource)
}, [source])
const handleQuerySubmit = (question: string): void => { const handleQuerySubmit = (question: string): void => {
queryStream.mutate({ question }) queryStream.mutate({ question })
setQueryText('') setQueryText('')
@ -91,14 +52,6 @@ export const LTTPage: React.FC = () => {
const videoUrl = currentVideoId ? getVideoUrl(currentVideoId) : '' const videoUrl = currentVideoId ? getVideoUrl(currentVideoId) : ''
const sourceTabClass = (active: boolean) =>
[
'flex items-center gap-2 px-4 py-2 text-sm font-medium rounded-lg transition-colors duration-200',
active
? 'bg-blue-600 text-white'
: 'bg-gray-100 text-gray-600 hover:bg-gray-200',
].join(' ')
return ( return (
<div className="h-full bg-gray-50"> <div className="h-full bg-gray-50">
<Group <Group
@ -112,88 +65,42 @@ export const LTTPage: React.FC = () => {
<Group orientation="horizontal" id="ltt-upper-group" className="h-full"> <Group orientation="horizontal" id="ltt-upper-group" className="h-full">
<Panel id="ltt-upper-left" minSize="30%" defaultSize={50}> <Panel id="ltt-upper-left" minSize="30%" defaultSize={50}>
<div className="h-full p-4 overflow-hidden flex flex-col gap-3"> <div className="h-full p-4 overflow-hidden flex flex-col gap-3">
<div data-testid="source-selector" className="flex gap-2 shrink-0"> {currentVideoId ? (
<button <>
data-testid="source-tab-upload" <VideoPlayer ref={setVideoEl} src={videoUrl} />
className={sourceTabClass(source === 'upload')} <button
onClick={() => handleSourceChange('upload')} onClick={handleRequestFullTranscript}
> disabled={ft.isLoading}
<Upload className="w-4 h-4" /> className="shrink-0 flex items-center justify-center gap-2 px-4 py-2 bg-gray-100 hover:bg-gray-200 text-gray-700 font-medium rounded-lg transition-colors duration-200 disabled:opacity-50 disabled:cursor-not-allowed"
Upload >
</button> {ft.isLoading ? (
<button <Loader2 className="w-4 h-4 animate-spin" />
data-testid="source-tab-youtube" ) : (
className={sourceTabClass(source === 'youtube')} <FileText className="w-4 h-4" />
onClick={() => handleSourceChange('youtube')} )}
> <span>{ft.isLoading ? 'Transcribing...' : 'Full Transcript'}</span>
<Youtube className="w-4 h-4" /> </button>
YouTube {ft.error && (
</button> <div
</div> data-testid="full-transcript-error"
className="flex items-start gap-2 text-sm text-red-600"
{source === 'upload' && (
currentVideoId ? (
<>
<VideoPlayer ref={setVideoEl} src={videoUrl} />
<button
onClick={handleRequestFullTranscript}
disabled={ft.isLoading}
className="shrink-0 flex items-center justify-center gap-2 px-4 py-2 bg-gray-100 hover:bg-gray-200 text-gray-700 font-medium rounded-lg transition-colors duration-200 disabled:opacity-50 disabled:cursor-not-allowed"
> >
{ft.isLoading ? ( <AlertCircle className="w-4 h-4 shrink-0 mt-0.5" />
<Loader2 className="w-4 h-4 animate-spin" /> <span>{ft.error}</span>
) : ( </div>
<FileText className="w-4 h-4" /> )}
)} {asr.status === 'error' && (
<span>{ft.isLoading ? 'Transcribing...' : 'Full Transcript'}</span> <div
</button> data-testid="asr-error-indicator"
{ft.error && ( className="flex items-center gap-2 text-xs text-red-600 bg-red-50 border border-red-200 rounded px-2 py-1"
<div >
data-testid="full-transcript-error" <AlertCircle className="w-3 h-3" />
className="flex items-start gap-2 text-sm text-red-600" <span>ASR error</span>
> </div>
<AlertCircle className="w-4 h-4 shrink-0 mt-0.5" /> )}
<span>{ft.error}</span> </>
</div> ) : (
)} <VideoUpload onUploadSuccess={handleUploadSuccess} />
{asr.status === 'error' && (
<div
data-testid="asr-error-indicator"
className="flex items-center gap-2 text-xs text-red-600 bg-red-50 border border-red-200 rounded px-2 py-1"
>
<AlertCircle className="w-3 h-3" />
<span>ASR error</span>
</div>
)}
</>
) : (
<VideoUpload onUploadSuccess={handleUploadSuccess} />
)
)}
{source === 'youtube' && (
youtubeData && youtubeData.video_proxy_url && youtubeData.audio_proxy_url ? (
<>
<YouTubeVideoPlayer
videoProxyUrl={youtubeData.video_proxy_url}
audioProxyUrl={youtubeData.audio_proxy_url}
thumbnailUrl={youtubeData.thumbnail_url}
isLive={youtubeData.is_live}
onAudioReady={handleYouTubeAudioReady}
/>
{asr.status === 'error' && (
<div
data-testid="asr-error-indicator"
className="flex items-center gap-2 text-xs text-red-600 bg-red-50 border border-red-200 rounded px-2 py-1"
>
<AlertCircle className="w-3 h-3" />
<span>ASR error</span>
</div>
)}
</>
) : (
<YouTubeInput onExtractSuccess={handleYouTubeExtractSuccess} />
)
)} )}
</div> </div>
</Panel> </Panel>

197
frontend/src/test/test_phase3_LTTPage_integration.test.tsx
View File

@ -1,197 +0,0 @@
/**
* Phase 3.5 integration tests: LTTPage YouTube source toggle and wiring.
* Tests verify that the YouTube source tab renders, switching between sources
* works, and components are conditionally rendered based on source state.
*/
import React from 'react'
import { render, screen, fireEvent } from '@testing-library/react'
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { LTTPage } from '../pages/LTTPage'
import type { YouTubeStreamResponse } from '../types'
const mockUseYouTubeExtract = () => ({
mutate: vi.fn(),
isPending: false,
isError: false,
error: null,
reset: vi.fn(),
})
vi.mock('../lib/queries', () => ({
useQueryDocumentStream: () => ({
extractedQuestions: null,
answer: null,
sources: null,
subQuestionSources: null,
phase: 'idle',
historyId: null,
error: null,
mutate: vi.fn(),
reset: vi.fn(),
}),
useVideoUpload: () => ({
mutate: vi.fn(),
isPending: false,
isError: false,
error: null,
reset: vi.fn(),
data: null,
}),
useYouTubeExtract: () => mockUseYouTubeExtract(),
}))
vi.mock('../hooks/useVideoASR', () => ({
useVideoASR: () => ({
transcript: '',
partialTranscript: '',
isStreaming: false,
status: 'idle',
startStreaming: vi.fn(),
stopStreaming: vi.fn(),
}),
}))
vi.mock('../hooks/useYouTubeASR', () => ({
useYouTubeASR: () => ({
transcript: '',
partialTranscript: '',
isStreaming: false,
status: 'idle',
startStreaming: vi.fn(),
stopStreaming: vi.fn(),
}),
}))
vi.mock('../hooks/useFullTranscript', () => ({
useFullTranscript: () => ({
fullTranscript: '',
isLoading: false,
error: null,
requestFullTranscript: vi.fn(),
}),
}))
vi.mock('hls.js', () => ({
default: class MockHls {
static isSupported = () => true
static Events = {
MANIFEST_PARSED: 'manifestParsed',
ERROR: 'error',
}
on = vi.fn()
loadSource = vi.fn()
attachMedia = vi.fn()
destroy = vi.fn()
levels = [{ height: 720 }, { height: 480 }, { height: 360 }]
autoLevelCapping = -1
},
}))
vi.mock('../lib/api', () => ({
getVideoUrl: (id: string) => `http://localhost:8000/api/v1/video/${id}`,
}))
describe('LTTPage YouTube source toggle (Phase 3.5)', () => {
beforeEach(() => {
vi.clearAllMocks()
})
it('source selector renders with Upload and YouTube tabs', () => {
render(<LTTPage />)
expect(screen.getByTestId('source-selector')).toBeInTheDocument()
expect(screen.getByTestId('source-tab-upload')).toBeInTheDocument()
expect(screen.getByTestId('source-tab-youtube')).toBeInTheDocument()
})
it('default source is upload — VideoUpload visible', () => {
render(<LTTPage />)
expect(screen.getByTestId('video-dropzone')).toBeInTheDocument()
})
it('switching to youtube shows YouTubeInput', () => {
render(<LTTPage />)
fireEvent.click(screen.getByTestId('source-tab-youtube'))
expect(screen.getByTestId('youtube-url-input')).toBeInTheDocument()
expect(screen.queryByTestId('video-dropzone')).not.toBeInTheDocument()
})
it('switching back to upload shows VideoUpload', () => {
render(<LTTPage />)
fireEvent.click(screen.getByTestId('source-tab-youtube'))
expect(screen.getByTestId('youtube-url-input')).toBeInTheDocument()
fireEvent.click(screen.getByTestId('source-tab-upload'))
expect(screen.getByTestId('video-dropzone')).toBeInTheDocument()
expect(screen.queryByTestId('youtube-url-input')).not.toBeInTheDocument()
})
it('Full Transcript button hidden when YouTube source is active', () => {
render(<LTTPage />)
expect(screen.queryByText(/full transcript/i)).not.toBeInTheDocument()
fireEvent.click(screen.getByTestId('source-tab-youtube'))
expect(screen.queryByText(/full transcript/i)).not.toBeInTheDocument()
})
it('QueryInput renders regardless of source', () => {
render(<LTTPage />)
expect(screen.getByPlaceholderText(/ask a question/i)).toBeInTheDocument()
fireEvent.click(screen.getByTestId('source-tab-youtube'))
expect(screen.getByPlaceholderText(/ask a question/i)).toBeInTheDocument()
})
it('YouTubeVideoPlayer appears after successful extraction', () => {
const mockStreamData: YouTubeStreamResponse = {
video_id: 'testvid123',
title: 'Test Stream',
is_live: false,
is_upcoming: false,
video_proxy_url: 'http://localhost:8000/api/v1/youtube/proxy/manifest.m3u8?url=video',
audio_proxy_url: 'http://localhost:8000/api/v1/youtube/proxy/manifest.m3u8?url=audio',
thumbnail_url: 'http://localhost:8000/thumb.jpg',
formats: [],
error: null,
}
let capturedOnSuccess: ((data: YouTubeStreamResponse) => void) | null = null
vi.doMock('../components/YouTubeInput', () => {
const MockYouTubeInput = ({ onExtractSuccess }: { onExtractSuccess: (data: YouTubeStreamResponse) => void }) => {
capturedOnSuccess = onExtractSuccess
return <div data-testid="youtube-url-input">YouTube Input Mock</div>
}
MockYouTubeInput.displayName = 'YouTubeInput'
return { YouTubeInput: MockYouTubeInput }
})
vi.doMock('../components/YouTubeVideoPlayer', () => {
const MockPlayer = () => <div data-testid="youtube-video-player-mock">Player Mock</div>
MockPlayer.displayName = 'YouTubeVideoPlayer'
return { YouTubeVideoPlayer: MockPlayer }
})
vi.resetModules()
return import('../pages/LTTPage').then(async ({ LTTPage: FreshLTTPage }) => {
render(<FreshLTTPage />)
fireEvent.click(screen.getByTestId('source-tab-youtube'))
expect(screen.getByTestId('youtube-url-input')).toBeInTheDocument()
if (capturedOnSuccess) {
capturedOnSuccess(mockStreamData)
}
expect(await screen.findByTestId('youtube-video-player-mock')).toBeInTheDocument()
expect(screen.queryByTestId('youtube-url-input')).not.toBeInTheDocument()
})
})
})

149
frontend/src/test/test_phase3_YouTubeInput.test.tsx
View File

@ -1,149 +0,0 @@
import React from 'react'
import { render, screen, fireEvent, waitFor } from '@testing-library/react'
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { YouTubeInput } from '../components/YouTubeInput'
const mockMutate = vi.fn()
const mockReset = vi.fn()
let mockIsPending = false
let mockIsError = false
let mockError: Error | null = null
vi.mock('../lib/queries', () => ({
useYouTubeExtract: () => ({
mutate: mockMutate,
isPending: mockIsPending,
isError: mockIsError,
error: mockError,
reset: mockReset,
}),
}))
describe('YouTubeInput (Phase 3.4)', () => {
const mockOnExtractSuccess = vi.fn()
beforeEach(() => {
vi.clearAllMocks()
mockMutate.mockClear()
mockReset.mockClear()
mockIsPending = false
mockIsError = false
mockError = null
})
it('renders URL input and Load Stream button', () => {
render(<YouTubeInput onExtractSuccess={mockOnExtractSuccess} />)
expect(screen.getByTestId('youtube-url-input')).toBeInTheDocument()
expect(screen.getByTestId('youtube-load-btn')).toBeInTheDocument()
expect(screen.getByTestId('youtube-load-btn')).toHaveTextContent(/load stream/i)
})
it('shows loading state during extraction', () => {
mockIsPending = true
render(<YouTubeInput onExtractSuccess={mockOnExtractSuccess} />)
expect(screen.getByTestId('youtube-loading')).toBeInTheDocument()
expect(screen.getByText(/extracting stream info/i)).toBeInTheDocument()
})
it('shows error state on failure', () => {
mockIsError = true
mockError = new Error('Extraction failed')
render(<YouTubeInput onExtractSuccess={mockOnExtractSuccess} />)
const errorEl = screen.getByTestId('youtube-error')
expect(errorEl).toBeInTheDocument()
expect(errorEl).toHaveTextContent(/extraction failed/i)
})
it('calls onExtractSuccess with data on success', async () => {
const mockResponse: import('../types').YouTubeStreamResponse = {
video_id: 'abc123',
title: 'Test Video',
is_live: false,
is_upcoming: false,
video_proxy_url: 'http://localhost:8000/api/v1/youtube/proxy/manifest.m3u8?url=vid',
audio_proxy_url: 'http://localhost:8000/api/v1/youtube/proxy/manifest.m3u8?url=aud',
thumbnail_url: 'http://localhost:8000/thumb.jpg',
formats: [],
error: null,
}
mockMutate.mockImplementation((_vars: any, options?: any) => {
if (options?.onSuccess) {
options.onSuccess(mockResponse)
}
})
render(<YouTubeInput onExtractSuccess={mockOnExtractSuccess} />)
const input = screen.getByTestId('youtube-url-input')
fireEvent.change(input, { target: { value: 'https://www.youtube.com/watch?v=abc123' } })
const button = screen.getByTestId('youtube-load-btn')
fireEvent.click(button)
await waitFor(() => {
expect(mockOnExtractSuccess).toHaveBeenCalledWith(mockResponse)
})
})
it('validates URL format and rejects non-YouTube URLs', () => {
render(<YouTubeInput onExtractSuccess={mockOnExtractSuccess} />)
const input = screen.getByTestId('youtube-url-input')
fireEvent.change(input, { target: { value: 'https://example.com/video' } })
const button = screen.getByTestId('youtube-load-btn')
fireEvent.click(button)
expect(screen.getByTestId('youtube-error')).toHaveTextContent(/valid youtube url/i)
expect(mockMutate).not.toHaveBeenCalled()
})
it('disables button when isDisabled is true', () => {
render(<YouTubeInput onExtractSuccess={mockOnExtractSuccess} isDisabled={true} />)
const input = screen.getByTestId('youtube-url-input')
expect(input).toBeDisabled()
const button = screen.getByTestId('youtube-load-btn')
expect(button).toBeDisabled()
})
it('accepts valid youtu.be short URL', async () => {
const mockResponse: import('../types').YouTubeStreamResponse = {
video_id: 'xyz789',
title: 'Short URL Video',
is_live: true,
is_upcoming: false,
video_proxy_url: 'http://localhost:8000/api/v1/youtube/proxy/manifest.m3u8?url=vid',
audio_proxy_url: 'http://localhost:8000/api/v1/youtube/proxy/manifest.m3u8?url=aud',
thumbnail_url: null,
formats: [],
error: null,
}
mockMutate.mockImplementation((_vars: any, options?: any) => {
if (options?.onSuccess) {
options.onSuccess(mockResponse)
}
})
render(<YouTubeInput onExtractSuccess={mockOnExtractSuccess} />)
const input = screen.getByTestId('youtube-url-input')
fireEvent.change(input, { target: { value: 'https://youtu.be/xyz789' } })
const button = screen.getByTestId('youtube-load-btn')
fireEvent.click(button)
await waitFor(() => {
expect(mockMutate).toHaveBeenCalledWith('https://youtu.be/xyz789', expect.any(Object))
})
})
})

168
frontend/src/test/test_phase3_YouTubeVideoPlayer.test.tsx
View File

@ -1,168 +0,0 @@
import React from 'react'
import { render, screen } from '@testing-library/react'
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { YouTubeVideoPlayer } from '../components/YouTubeVideoPlayer'
vi.mock('hls.js', () => ({
default: class MockHls {
static isSupported = () => true
static Events = {
MANIFEST_PARSED: 'manifestParsed',
ERROR: 'error',
}
on = vi.fn()
loadSource = vi.fn()
attachMedia = vi.fn()
destroy = vi.fn()
levels = [{ height: 720 }, { height: 480 }, { height: 360 }]
autoLevelCapping = -1
},
}))
describe('YouTubeVideoPlayer (Phase 3.4)', () => {
const mockVideoProxyUrl = 'http://localhost:8000/api/v1/youtube/proxy/manifest.m3u8?url=video'
const mockAudioProxyUrl = 'http://localhost:8000/api/v1/youtube/proxy/manifest.m3u8?url=audio'
const mockThumbnailUrl = 'http://localhost:8000/thumb.jpg'
beforeEach(() => {
vi.clearAllMocks()
})
it('renders thumbnail when thumbnailUrl is provided', () => {
render(
<YouTubeVideoPlayer
videoProxyUrl={mockVideoProxyUrl}
audioProxyUrl={mockAudioProxyUrl}
thumbnailUrl={mockThumbnailUrl}
isLive={false}
/>
)
const thumbnail = screen.getByTestId('youtube-thumbnail')
expect(thumbnail).toBeInTheDocument()
expect(thumbnail).toHaveAttribute('src', mockThumbnailUrl)
})
it('shows LIVE badge when isLive is true', () => {
render(
<YouTubeVideoPlayer
videoProxyUrl={mockVideoProxyUrl}
audioProxyUrl={mockAudioProxyUrl}
thumbnailUrl={null}
isLive={true}
/>
)
const badge = screen.getByTestId('youtube-live-badge')
expect(badge).toBeInTheDocument()
expect(badge).toHaveTextContent('LIVE')
})
it('does not show LIVE badge when isLive is false', () => {
render(
<YouTubeVideoPlayer
videoProxyUrl={mockVideoProxyUrl}
audioProxyUrl={mockAudioProxyUrl}
thumbnailUrl={null}
isLive={false}
/>
)
expect(screen.queryByTestId('youtube-live-badge')).not.toBeInTheDocument()
})
it('shows loading state initially', () => {
render(
<YouTubeVideoPlayer
videoProxyUrl={mockVideoProxyUrl}
audioProxyUrl={mockAudioProxyUrl}
thumbnailUrl={null}
isLive={false}
/>
)
expect(screen.getByTestId('youtube-loading')).toBeInTheDocument()
expect(screen.getByText(/loading stream/i)).toBeInTheDocument()
})
it('forwards ref to video element', () => {
const ref = React.createRef<HTMLVideoElement>()
render(
<YouTubeVideoPlayer
ref={ref}
videoProxyUrl={mockVideoProxyUrl}
audioProxyUrl={mockAudioProxyUrl}
thumbnailUrl={null}
isLive={false}
/>
)
expect(ref.current).not.toBeNull()
expect(ref.current?.tagName.toLowerCase()).toBe('video')
})
it('renders video element with correct attributes', () => {
render(
<YouTubeVideoPlayer
videoProxyUrl={mockVideoProxyUrl}
audioProxyUrl={mockAudioProxyUrl}
thumbnailUrl={null}
isLive={false}
/>
)
const video = screen.getByTestId('youtube-video')
expect(video).toBeInTheDocument()
expect(video).toHaveAttribute('controls')
expect(video).toHaveProperty('muted', true)
expect(video).toHaveAttribute('crossOrigin', 'anonymous')
expect(video).toHaveClass('w-full')
expect(video).toHaveClass('max-h-60')
})
it('renders audio element hidden', () => {
render(
<YouTubeVideoPlayer
videoProxyUrl={mockVideoProxyUrl}
audioProxyUrl={mockAudioProxyUrl}
thumbnailUrl={null}
isLive={false}
/>
)
const audio = screen.getByTestId('youtube-audio')
expect(audio).toBeInTheDocument()
expect(audio).toHaveAttribute('crossOrigin', 'anonymous')
expect(audio).toHaveClass('hidden')
})
it('renders without crashing with null thumbnail', () => {
render(
<YouTubeVideoPlayer
videoProxyUrl={mockVideoProxyUrl}
audioProxyUrl={mockAudioProxyUrl}
thumbnailUrl={null}
isLive={false}
/>
)
expect(screen.getByTestId('youtube-video')).toBeInTheDocument()
expect(screen.queryByTestId('youtube-thumbnail')).not.toBeInTheDocument()
})
it('calls onAudioReady with audio element', () => {
const mockOnAudioReady = vi.fn()
render(
<YouTubeVideoPlayer
videoProxyUrl={mockVideoProxyUrl}
audioProxyUrl={mockAudioProxyUrl}
thumbnailUrl={null}
isLive={false}
onAudioReady={mockOnAudioReady}
/>
)
expect(mockOnAudioReady).toHaveBeenCalledTimes(1)
expect(mockOnAudioReady.mock.calls[0][0]).toBeInstanceOf(HTMLAudioElement)
})
})

179
frontend/src/test/test_phase3_useYouTubeASR.test.ts
View File

@ -1,179 +0,0 @@
/**
* Phase 3.5 tests: useYouTubeASR hook state management.
*
* WebAudio (AudioContext, ScriptProcessorNode) and WebSocket are NOT available
* in jsdom, so these tests verify state management, return shape, and cleanup
* logic only. Full audio capture is covered by acceptance tests.
*/
import { renderHook, act } from '@testing-library/react'
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { useYouTubeASR } from '../hooks/useYouTubeASR'
import type { ASRStatus } from '../types'
beforeEach(() => {
vi.clearAllMocks()
})
describe('useYouTubeASR', () => {
it('test_initial_state', () => {
const { result } = renderHook(() =>
useYouTubeASR({
videoId: 'test-video-id',
videoElement: null,
audioElement: null,
})
)
expect(result.current.transcript).toBe('')
expect(result.current.partialTranscript).toBe('')
expect(result.current.isStreaming).toBe(false)
expect(result.current.status).toBe<ASRStatus>('idle')
})
it('test_returns_startStreaming_function', () => {
const { result } = renderHook(() =>
useYouTubeASR({
videoId: 'test-video-id',
videoElement: null,
audioElement: null,
})
)
expect(typeof result.current.startStreaming).toBe('function')
})
it('test_returns_stopStreaming_function', () => {
const { result } = renderHook(() =>
useYouTubeASR({
videoId: 'test-video-id',
videoElement: null,
audioElement: null,
})
)
expect(typeof result.current.stopStreaming).toBe('function')
})
it('test_stopStreaming_resets_state', () => {
const { result } = renderHook(() =>
useYouTubeASR({
videoId: 'test-video-id',
videoElement: null,
audioElement: null,
})
)
act(() => {
result.current.stopStreaming()
})
expect(result.current.status).toBe<ASRStatus>('idle')
expect(result.current.isStreaming).toBe(false)
expect(result.current.partialTranscript).toBe('')
})
it('test_startStreaming_without_elements_does_not_throw', () => {
const { result } = renderHook(() =>
useYouTubeASR({
videoId: 'test-video-id',
videoElement: null,
audioElement: null,
})
)
expect(() => {
act(() => {
result.current.startStreaming()
})
}).not.toThrow()
})
it('test_startStreaming_with_no_elements_sets_error_status', () => {
const { result } = renderHook(() =>
useYouTubeASR({
videoId: 'test-video-id',
videoElement: null,
audioElement: null,
})
)
act(() => {
result.current.startStreaming()
})
expect(['idle', 'error']).toContain(result.current.status)
})
it('test_cleanup_on_unmount', () => {
const { result, unmount } = renderHook(() =>
useYouTubeASR({
videoId: 'test-video-id',
videoElement: null,
audioElement: null,
})
)
expect(() => {
unmount()
}).not.toThrow()
})
it('test_accepts_language_option', () => {
const { result } = renderHook(() =>
useYouTubeASR({
videoId: 'test-video-id',
videoElement: null,
audioElement: null,
language: 'en',
})
)
expect(result.current.status).toBe<ASRStatus>('idle')
})
it('test_accepts_onFinalTranscript_callback', () => {
const onFinal = vi.fn()
const { result } = renderHook(() =>
useYouTubeASR({
videoId: 'test-video-id',
videoElement: null,
audioElement: null,
onFinalTranscript: onFinal,
})
)
expect(result.current.status).toBe<ASRStatus>('idle')
})
it('test_status_type_covers_all_states', () => {
const validStatuses: ASRStatus[] = ['idle', 'connecting', 'streaming', 'disconnected', 'error']
expect(validStatuses).toHaveLength(5)
expect(validStatuses).toContain('idle')
expect(validStatuses).toContain('connecting')
expect(validStatuses).toContain('streaming')
expect(validStatuses).toContain('disconnected')
expect(validStatuses).toContain('error')
})
it('test_startStreaming_with_audio_only', () => {
const mockAudioElement = {
crossOrigin: null,
addEventListener: vi.fn(),
removeEventListener: vi.fn(),
} as unknown as HTMLAudioElement
const { result } = renderHook(() =>
useYouTubeASR({
videoId: 'test-video-id',
videoElement: null,
audioElement: mockAudioElement,
})
)
expect(() => {
act(() => {
result.current.startStreaming()
})
}).not.toThrow()
})
})

23
frontend/src/types/index.ts
View File

@ -195,26 +195,3 @@ export interface VideoUploadResponse {
size_bytes: number size_bytes: number
url: string url: string
} }
// Phase 3 — YouTube Stream types
export interface YouTubeFormat {
format_id: string
url: string
resolution: string | null
is_audio_only: boolean
is_video_only: boolean
codec: string | null
}
export interface YouTubeStreamResponse {
video_id: string
title: string
is_live: boolean
is_upcoming: boolean
video_proxy_url: string | null
audio_proxy_url: string | null
thumbnail_url: string | null
formats: YouTubeFormat[]
error: string | null
}