feat: Phase 3.5 — YouTube → ASR integration with source toggle
- useYouTubeASR.ts: adapted from useVideoASR, captures audio from HTMLAudioElement (hls.js → <audio> → AudioContext.createMediaElementSource → ScriptProcessorNode → WebSocket) Play/pause events on videoElement; same return shape as useVideoASR - LTTPage.tsx: Source toggle (Upload/YouTube tabs), YouTubeInput + YouTubeVideoPlayer wired with handleExtractSuccess → handleAudioReady → useYouTubeASR Full Transcript button hidden for YouTube source; unified asr variable - QueryInput.tsx: no changes needed (already supports partialText/value from any source) - Tests: 18 new (11 useYouTubeASR, 7 LTTPage integration) - 189/189 total pass (zero regressions) - Updated plan: 3.5 marked Complete, 5/7 sub-phases done
This commit is contained in:
Woody
parent
a8eea54c0f
commit
1699a249b0
|
|
@ -1,8 +1,8 @@
|
|||
# Phase 3: YouTube Live Stream Proxy → ASR → RAG — Implementation Plan
|
||||
|
||||
**Created:** 2026-05-09
|
||||
**Updated:** 2026-05-09 (Phase 3.1–3.4 implemented)
|
||||
**Status:** In Progress (3.1 ✅, 3.2 ✅, 3.3 ✅, 3.4 ✅)
|
||||
**Updated:** 2026-05-09 (Phase 3.1–3.5 implemented)
|
||||
**Status:** In Progress (3.1 ✅, 3.2 ✅, 3.3 ✅, 3.4 ✅, 3.5 ✅)
|
||||
**Depends on:** Phase 1 (Complete), Phase 2 (Complete)
|
||||
|
||||
---
|
||||
|
|
@ -184,30 +184,41 @@ URL input component and hls.js-based video player. Two media elements: visible `
|
|||
|
||||
---
|
||||
|
||||
### Phase 3.5 — Integration: YouTube → ASR Pipeline (1 day)
|
||||
### Phase 3.5 — Integration: YouTube → ASR Pipeline ✅ Complete
|
||||
|
||||
Wire YouTube audio output into existing ASR pipeline. The key challenge: `useVideoASR` currently captures from `<video>` element; we need it to capture from the `<audio>` element loaded by hls.js.
|
||||
Wire YouTube audio output into existing ASR pipeline. Creates `useYouTubeASR` hook (adapted from `useVideoASR`) and integrates YouTube components into `LTTPage` with a source toggle.
|
||||
|
||||
**Tests:** `test_phase3_useYouTubeASR.test.ts`, `test_phase3_LTTPage_integration.test.tsx`
|
||||
**Tests:** `test_phase3_useYouTubeASR.test.ts` (11 tests), `test_phase3_LTTPage_integration.test.tsx` (7 tests)
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- `useYouTubeASR` hook: accepts `audioElement` ref, sets up AudioContext graph on mount
|
||||
- AudioContext.createMediaElementSource(audioElement) → ScriptProcessorNode → WebSocket
|
||||
- `useYouTubeASR` hook: accepts `audioElement` + `videoElement`, sets up AudioContext graph on mount
|
||||
- AudioContext.createMediaElementSource(audioElement) → ScriptProcessorNode → WebSocket (same as useVideoASR, but audio source from `<audio>` element)
|
||||
- Play/pause/ended events on `videoElement` (user controls video, audio follows)
|
||||
- Auto-starts ASR on play, stops on pause/end (same lifecycle as `useVideoASR`)
|
||||
- Transcript flows into QueryInput (same `onFinalTranscript` callback)
|
||||
- QueryInput remains editable during streaming — user can type corrections while ASR appends
|
||||
- Transcript flows into QueryInput (same `onFinalTranscript` + `partialTranscript` callbacks)
|
||||
- QueryInput remains editable during streaming — user can type corrections while ASR appends (already worked, no changes needed)
|
||||
- "Full Transcript" button hidden when YouTube source is active
|
||||
- Switching between "Upload" and "YouTube" sources clears previous state
|
||||
- Source toggle: "Upload" / "YouTube" tabs at top of upper-left panel
|
||||
- Switching between "Upload" and "YouTube" sources clears previous YouTube state
|
||||
- Upload video state preserved when switching to YouTube and back
|
||||
|
||||
**Implementation Notes:**
|
||||
- Both `useVideoASR` and `useYouTubeASR` initialized unconditionally at top of LTTPage
|
||||
- Hooks gracefully handle null elements (AudioContext setup aborts early if element is null)
|
||||
- Unified `asr` variable: `const asr = source === 'youtube' ? youtubeASR : uploadASR`
|
||||
- Source toggle uses `Upload`/`Youtube` icons from lucide-react, blue active / gray inactive state
|
||||
- `QueryInput.tsx` — zero changes needed (already supports `partialText` + `value` from any source)
|
||||
- `YouTubeVideoPlayer` exposes audio element via `onAudioReady` callback → LTTPage wires to `useYouTubeASR`
|
||||
|
||||
**Tasks:**
|
||||
| # | Task | File |
|
||||
|---|------|------|
|
||||
| 3.5.1 | Write tests first | `src/test/test_phase3_useYouTubeASR.test.ts` |
|
||||
| 3.5.2 | Create `hooks/useYouTubeASR.ts` — adapted from `useVideoASR.ts`, targets `<audio>` element | `hooks/useYouTubeASR.ts` |
|
||||
| 3.5.3 | Update `QueryInput.tsx` — accept transcript from either source | `components/QueryInput.tsx` |
|
||||
| 3.5.4 | Update `LTTPage.tsx` — add source toggle (Upload / YouTube), wire YouTubeInput + YouTubeVideoPlayer | `pages/LTTPage.tsx` |
|
||||
| 3.5.5 | Create `test_phase3_LTTPage_integration.test.tsx` | `src/test/` |
|
||||
| 3.5.6 | Run tests → pass → commit | — |
|
||||
| # | Task | File | Status |
|
||||
|---|------|------|--------|
|
||||
| 3.5.1 | Write tests first | `src/test/test_phase3_useYouTubeASR.test.ts` | Done (11 tests) |
|
||||
| 3.5.2 | Create `hooks/useYouTubeASR.ts` | `hooks/useYouTubeASR.ts` | Done |
|
||||
| 3.5.3 | Update `QueryInput.tsx` | `components/QueryInput.tsx` | Done (no-op: already works) |
|
||||
| 3.5.4 | Update `LTTPage.tsx` — source toggle, wire components | `pages/LTTPage.tsx` | Done |
|
||||
| 3.5.5 | Create LTTPage integration test | `src/test/test_phase3_LTTPage_integration.test.tsx` | Done (7 tests) |
|
||||
| 3.5.6 | Run tests → pass → commit | — | Done (189/189 pass) |
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -248,10 +259,10 @@ Wire YouTube audio output into existing ASR pipeline. The key challenge: `useVid
|
|||
| 3.2 | YouTube URL Extraction | 0.5 day | 3.1 | ✅ Complete |
|
||||
| 3.3 | HLS Proxy Backend | 1 day | 3.1 | ✅ Complete |
|
||||
| 3.4 | Frontend Input + Player | 1 day | 3.2, 3.3 | ✅ Complete |
|
||||
| 3.5 | YouTube → ASR Integration | 1 day | 3.4 | ⏳ Next |
|
||||
| 3.6 | Integration & Acceptance | 1 day | 3.5 | Pending |
|
||||
| 3.5 | YouTube → ASR Integration | 1 day | 3.4 | ✅ Complete |
|
||||
| 3.6 | Integration & Acceptance | 1 day | 3.5 | ⏳ Next |
|
||||
| 3.7 | Polish & Deployment | 0.5 day | 3.6 | Pending |
|
||||
| **Total** | | **5.5 days** | | **4/7 done** |
|
||||
| **Total** | | **5.5 days** | | **5/7 done** |
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -324,11 +335,12 @@ backend/
|
|||
frontend/src/
|
||||
components/YouTubeInput.tsx ✅ Created (3.4)
|
||||
components/YouTubeVideoPlayer.tsx ✅ Created (3.4)
|
||||
hooks/useYouTubeASR.ts ⏳ Pending (3.5)
|
||||
hooks/useYouTubeASR.ts ✅ Created (3.5)
|
||||
pages/LTTPage.tsx ✅ Updated (3.5)
|
||||
test/test_phase3_YouTubeInput.test.tsx ✅ Written (3.4, 7 tests)
|
||||
test/test_phase3_YouTubeVideoPlayer.test.tsx ✅ Written (3.4, 9 tests)
|
||||
test/test_phase3_useYouTubeASR.test.ts ⏳ Pending (3.5)
|
||||
test/test_phase3_LTTPage_integration.test.tsx ⏳ Pending (3.5)
|
||||
test/test_phase3_useYouTubeASR.test.ts ✅ Written (3.5, 11 tests)
|
||||
test/test_phase3_LTTPage_integration.test.tsx ✅ Written (3.5, 7 tests)
|
||||
```
|
||||
|
||||
### Modified Files
|
||||
|
|
@ -342,8 +354,8 @@ frontend/package.json ✅ Done (hls.js added)
|
|||
frontend/src/types/index.ts ✅ Done (3.4)
|
||||
frontend/src/lib/api.ts ✅ Done (3.4)
|
||||
frontend/src/lib/queries.tsx ✅ Done (3.4)
|
||||
frontend/src/pages/LTTPage.tsx ⏳ Pending (3.5)
|
||||
frontend/src/components/QueryInput.tsx ⏳ Pending (3.5)
|
||||
frontend/src/pages/LTTPage.tsx ✅ Done (3.5)
|
||||
frontend/src/components/QueryInput.tsx ✅ Done (3.5 — no-op, already compatible)
|
||||
|
||||
Dockerfile ⏳ Pending (3.7)
|
||||
docker-compose.yml ⏳ Pending (3.7)
|
||||
|
|
@ -427,8 +439,9 @@ GET /api/v1/youtube/proxy/segment.ts?url=<encoded_upstream_ts>
|
|||
| Phase 3.1 (config) | 11 | ✅ All pass |
|
||||
| Phase 3.2 (extraction) | 18 | ✅ All pass |
|
||||
| Phase 3.3 (HLS proxy) | 22 | ✅ All pass |
|
||||
| Phase 3.4 frontend | 16 | ✅ All pass |
|
||||
| **Total** | **171** | **0 failures** |
|
||||
| Phase 3.4 frontend (YouTube components) | 16 | ✅ All pass |
|
||||
| Phase 3.5 frontend (ASR integration) | 18 | ✅ All pass |
|
||||
| **Total** | **189** | **0 failures** |
|
||||
|
||||
### Real-URL Smoke Tests
|
||||
| URL | Type | Result |
|
||||
|
|
|
|||
|
|
@ -0,0 +1,178 @@
|
|||
import { useState, useRef, useCallback, useEffect } from 'react'
|
||||
import type { ASRMessage, ASRStatus } from '../types'
|
||||
|
||||
interface UseYouTubeASROptions {
|
||||
videoId: string
|
||||
videoElement: HTMLVideoElement | null
|
||||
audioElement: HTMLAudioElement | null
|
||||
language?: string
|
||||
onFinalTranscript?: (text: string) => void
|
||||
}
|
||||
|
||||
export function useYouTubeASR({
|
||||
videoId,
|
||||
videoElement,
|
||||
audioElement,
|
||||
language = 'yue',
|
||||
onFinalTranscript,
|
||||
}: UseYouTubeASROptions) {
|
||||
const [transcript, setTranscript] = useState('')
|
||||
const [partialTranscript, setPartialTranscript] = useState('')
|
||||
const [status, setStatus] = useState<ASRStatus>('idle')
|
||||
const [isStreaming, setIsStreaming] = useState(false)
|
||||
|
||||
const wsRef = useRef<WebSocket | null>(null)
|
||||
const audioContextRef = useRef<AudioContext | null>(null)
|
||||
const processorRef = useRef<ScriptProcessorNode | null>(null)
|
||||
const sourceRef = useRef<MediaElementAudioSourceNode | null>(null)
|
||||
const isStreamingRef = useRef(false)
|
||||
const graphSetupRef = useRef(false)
|
||||
const transcriptRef = useRef('')
|
||||
const lastStashRef = useRef('')
|
||||
const onFinalTranscriptRef = useRef(onFinalTranscript)
|
||||
onFinalTranscriptRef.current = onFinalTranscript
|
||||
|
||||
const getWSURL = useCallback(() => {
|
||||
const protocol = window.location.protocol === 'https:' ? 'wss:' : 'ws:'
|
||||
const host = window.location.host
|
||||
const langParam = language !== 'auto' ? `?language=${language}` : ''
|
||||
const backendHost = import.meta.env.VITE_WS_HOST ?? host
|
||||
return `${protocol}//${backendHost}/ws/asr/${videoId}${langParam}`
|
||||
}, [videoId, language])
|
||||
|
||||
const connectWebSocket = useCallback(() => {
|
||||
const ws = new WebSocket(getWSURL())
|
||||
wsRef.current = ws
|
||||
|
||||
ws.onopen = () => {
|
||||
isStreamingRef.current = true
|
||||
setIsStreaming(true)
|
||||
setStatus('streaming')
|
||||
}
|
||||
|
||||
ws.onmessage = (e) => {
|
||||
const msg: ASRMessage = JSON.parse(e.data)
|
||||
if (msg.is_final && msg.full_text) {
|
||||
transcriptRef.current = msg.full_text
|
||||
lastStashRef.current = ''
|
||||
setTranscript(msg.full_text)
|
||||
setPartialTranscript('')
|
||||
onFinalTranscriptRef.current?.(msg.full_text)
|
||||
} else if (msg.delta) {
|
||||
transcriptRef.current += msg.delta
|
||||
lastStashRef.current = (msg as any).stash || ''
|
||||
setTranscript(transcriptRef.current)
|
||||
setPartialTranscript(transcriptRef.current)
|
||||
}
|
||||
}
|
||||
|
||||
ws.onerror = (e) => {
|
||||
console.error('[useYouTubeASR] WebSocket error:', e)
|
||||
setStatus('error')
|
||||
}
|
||||
ws.onclose = () => {
|
||||
isStreamingRef.current = false
|
||||
setIsStreaming(false)
|
||||
setStatus('disconnected')
|
||||
}
|
||||
}, [getWSURL])
|
||||
|
||||
const closeWebSocket = useCallback(() => {
|
||||
wsRef.current?.close()
|
||||
wsRef.current = null
|
||||
}, [])
|
||||
|
||||
const startStreaming = useCallback(() => {
|
||||
if (!audioElement) return
|
||||
try {
|
||||
setStatus('connecting')
|
||||
audioContextRef.current?.resume()
|
||||
closeWebSocket()
|
||||
connectWebSocket()
|
||||
} catch (err) {
|
||||
console.error('[useYouTubeASR] startStreaming failed:', err)
|
||||
setStatus('error')
|
||||
}
|
||||
}, [audioElement, closeWebSocket, connectWebSocket])
|
||||
|
||||
const stopStreaming = useCallback(() => {
|
||||
isStreamingRef.current = false
|
||||
setIsStreaming(false)
|
||||
closeWebSocket()
|
||||
setStatus('idle')
|
||||
let currentText = transcriptRef.current.trim()
|
||||
const stash = lastStashRef.current.trim()
|
||||
if (stash && !currentText.endsWith(stash)) {
|
||||
currentText += stash
|
||||
transcriptRef.current = currentText
|
||||
}
|
||||
lastStashRef.current = ''
|
||||
if (currentText) {
|
||||
onFinalTranscriptRef.current?.(currentText)
|
||||
setPartialTranscript('')
|
||||
}
|
||||
}, [closeWebSocket])
|
||||
|
||||
useEffect(() => {
|
||||
if (!audioElement || graphSetupRef.current) return
|
||||
try {
|
||||
const audioContext = new AudioContext({ sampleRate: 16000 })
|
||||
audioContextRef.current = audioContext
|
||||
|
||||
const source = audioContext.createMediaElementSource(audioElement)
|
||||
sourceRef.current = source
|
||||
|
||||
const processor = audioContext.createScriptProcessor(4096, 1, 1)
|
||||
processorRef.current = processor
|
||||
|
||||
processor.onaudioprocess = (e) => {
|
||||
const float32Data = e.inputBuffer.getChannelData(0)
|
||||
const outputData = e.outputBuffer.getChannelData(0)
|
||||
outputData.set(float32Data)
|
||||
if (!isStreamingRef.current) return
|
||||
if (!wsRef.current || wsRef.current.readyState !== WebSocket.OPEN) return
|
||||
wsRef.current.send(float32Data.buffer)
|
||||
}
|
||||
|
||||
source.connect(processor)
|
||||
processor.connect(audioContext.destination)
|
||||
graphSetupRef.current = true
|
||||
} catch (err) {
|
||||
console.error('[useYouTubeASR] audio graph setup failed:', err)
|
||||
}
|
||||
}, [audioElement])
|
||||
|
||||
useEffect(() => {
|
||||
return () => {
|
||||
isStreamingRef.current = false
|
||||
processorRef.current?.disconnect()
|
||||
sourceRef.current?.disconnect()
|
||||
wsRef.current?.close()
|
||||
audioContextRef.current?.close()
|
||||
}
|
||||
}, [])
|
||||
|
||||
useEffect(() => {
|
||||
if (!videoElement) return
|
||||
const onPlay = () => startStreaming()
|
||||
const onPause = () => stopStreaming()
|
||||
const onEnded = () => stopStreaming()
|
||||
videoElement.addEventListener('play', onPlay)
|
||||
videoElement.addEventListener('pause', onPause)
|
||||
videoElement.addEventListener('ended', onEnded)
|
||||
return () => {
|
||||
videoElement.removeEventListener('play', onPlay)
|
||||
videoElement.removeEventListener('pause', onPause)
|
||||
videoElement.removeEventListener('ended', onEnded)
|
||||
}
|
||||
}, [videoElement, startStreaming, stopStreaming])
|
||||
|
||||
return {
|
||||
transcript,
|
||||
partialTranscript,
|
||||
isStreaming,
|
||||
status,
|
||||
startStreaming,
|
||||
stopStreaming,
|
||||
}
|
||||
}
|
||||
|
|
@ -1,8 +1,9 @@
|
|||
import React, { useState, useCallback, useEffect } from 'react'
|
||||
import { Loader2, AlertCircle, FileText } from 'lucide-react'
|
||||
import { Loader2, AlertCircle, FileText, Upload, Youtube } from 'lucide-react'
|
||||
import { Group, Panel, Separator } from 'react-resizable-panels'
|
||||
import { useQueryDocumentStream } from '../lib/queries'
|
||||
import { useVideoASR } from '../hooks/useVideoASR'
|
||||
import { useYouTubeASR } from '../hooks/useYouTubeASR'
|
||||
import { useFullTranscript } from '../hooks/useFullTranscript'
|
||||
import { getVideoUrl } from '../lib/api'
|
||||
import { QueryInput } from '../components/QueryInput'
|
||||
|
|
@ -10,15 +11,23 @@ import { ExtractedQuestionsDisplay } from '../components/ExtractedQuestionsDispl
|
|||
import { ResponsePanel } from '../components/ResponsePanel'
|
||||
import { VideoUpload } from '../components/VideoUpload'
|
||||
import { VideoPlayer } from '../components/VideoPlayer'
|
||||
import { YouTubeInput } from '../components/YouTubeInput'
|
||||
import { YouTubeVideoPlayer } from '../components/YouTubeVideoPlayer'
|
||||
import type { YouTubeStreamResponse } from '../types'
|
||||
|
||||
type SourceType = 'upload' | 'youtube'
|
||||
|
||||
export const LTTPage: React.FC = () => {
|
||||
const [source, setSource] = useState<SourceType>('upload')
|
||||
const [currentVideoId, setCurrentVideoId] = useState<string | null>(null)
|
||||
const [queryText, setQueryText] = useState('')
|
||||
const [videoEl, setVideoEl] = useState<HTMLVideoElement | null>(null)
|
||||
const [youtubeData, setYoutubeData] = useState<YouTubeStreamResponse | null>(null)
|
||||
const [youtubeAudioEl, setYoutubeAudioEl] = useState<HTMLAudioElement | null>(null)
|
||||
|
||||
const queryStream = useQueryDocumentStream()
|
||||
|
||||
const asr = useVideoASR({
|
||||
const uploadASR = useVideoASR({
|
||||
videoId: currentVideoId ?? '',
|
||||
videoElement: videoEl,
|
||||
language: 'yue',
|
||||
|
|
@ -27,6 +36,18 @@ export const LTTPage: React.FC = () => {
|
|||
},
|
||||
})
|
||||
|
||||
const youtubeASR = useYouTubeASR({
|
||||
videoId: youtubeData?.video_id ?? '',
|
||||
videoElement: videoEl,
|
||||
audioElement: youtubeAudioEl,
|
||||
language: 'yue',
|
||||
onFinalTranscript: (text) => {
|
||||
setQueryText(text)
|
||||
},
|
||||
})
|
||||
|
||||
const asr = source === 'youtube' ? youtubeASR : uploadASR
|
||||
|
||||
const ft = useFullTranscript({ videoId: currentVideoId ?? '' })
|
||||
|
||||
useEffect(() => {
|
||||
|
|
@ -39,6 +60,24 @@ export const LTTPage: React.FC = () => {
|
|||
setCurrentVideoId(videoId)
|
||||
}, [])
|
||||
|
||||
const handleYouTubeExtractSuccess = useCallback((data: YouTubeStreamResponse) => {
|
||||
setYoutubeData(data)
|
||||
setQueryText('')
|
||||
}, [])
|
||||
|
||||
const handleYouTubeAudioReady = useCallback((audioEl: HTMLAudioElement) => {
|
||||
setYoutubeAudioEl(audioEl)
|
||||
}, [])
|
||||
|
||||
const handleSourceChange = useCallback((newSource: SourceType) => {
|
||||
if (newSource === source) return
|
||||
if (newSource === 'upload') {
|
||||
setYoutubeData(null)
|
||||
setYoutubeAudioEl(null)
|
||||
}
|
||||
setSource(newSource)
|
||||
}, [source])
|
||||
|
||||
const handleQuerySubmit = (question: string): void => {
|
||||
queryStream.mutate({ question })
|
||||
setQueryText('')
|
||||
|
|
@ -52,6 +91,14 @@ export const LTTPage: React.FC = () => {
|
|||
|
||||
const videoUrl = currentVideoId ? getVideoUrl(currentVideoId) : ''
|
||||
|
||||
const sourceTabClass = (active: boolean) =>
|
||||
[
|
||||
'flex items-center gap-2 px-4 py-2 text-sm font-medium rounded-lg transition-colors duration-200',
|
||||
active
|
||||
? 'bg-blue-600 text-white'
|
||||
: 'bg-gray-100 text-gray-600 hover:bg-gray-200',
|
||||
].join(' ')
|
||||
|
||||
return (
|
||||
<div className="h-full bg-gray-50">
|
||||
<Group
|
||||
|
|
@ -65,7 +112,27 @@ export const LTTPage: React.FC = () => {
|
|||
<Group orientation="horizontal" id="ltt-upper-group" className="h-full">
|
||||
<Panel id="ltt-upper-left" minSize="30%" defaultSize={50}>
|
||||
<div className="h-full p-4 overflow-hidden flex flex-col gap-3">
|
||||
{currentVideoId ? (
|
||||
<div data-testid="source-selector" className="flex gap-2 shrink-0">
|
||||
<button
|
||||
data-testid="source-tab-upload"
|
||||
className={sourceTabClass(source === 'upload')}
|
||||
onClick={() => handleSourceChange('upload')}
|
||||
>
|
||||
<Upload className="w-4 h-4" />
|
||||
Upload
|
||||
</button>
|
||||
<button
|
||||
data-testid="source-tab-youtube"
|
||||
className={sourceTabClass(source === 'youtube')}
|
||||
onClick={() => handleSourceChange('youtube')}
|
||||
>
|
||||
<Youtube className="w-4 h-4" />
|
||||
YouTube
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{source === 'upload' && (
|
||||
currentVideoId ? (
|
||||
<>
|
||||
<VideoPlayer ref={setVideoEl} src={videoUrl} />
|
||||
<button
|
||||
|
|
@ -101,6 +168,32 @@ export const LTTPage: React.FC = () => {
|
|||
</>
|
||||
) : (
|
||||
<VideoUpload onUploadSuccess={handleUploadSuccess} />
|
||||
)
|
||||
)}
|
||||
|
||||
{source === 'youtube' && (
|
||||
youtubeData && youtubeData.video_proxy_url && youtubeData.audio_proxy_url ? (
|
||||
<>
|
||||
<YouTubeVideoPlayer
|
||||
videoProxyUrl={youtubeData.video_proxy_url}
|
||||
audioProxyUrl={youtubeData.audio_proxy_url}
|
||||
thumbnailUrl={youtubeData.thumbnail_url}
|
||||
isLive={youtubeData.is_live}
|
||||
onAudioReady={handleYouTubeAudioReady}
|
||||
/>
|
||||
{asr.status === 'error' && (
|
||||
<div
|
||||
data-testid="asr-error-indicator"
|
||||
className="flex items-center gap-2 text-xs text-red-600 bg-red-50 border border-red-200 rounded px-2 py-1"
|
||||
>
|
||||
<AlertCircle className="w-3 h-3" />
|
||||
<span>ASR error</span>
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
) : (
|
||||
<YouTubeInput onExtractSuccess={handleYouTubeExtractSuccess} />
|
||||
)
|
||||
)}
|
||||
</div>
|
||||
</Panel>
|
||||
|
|
|
|||
|
|
@ -0,0 +1,197 @@
|
|||
/**
|
||||
* Phase 3.5 integration tests: LTTPage YouTube source toggle and wiring.
|
||||
* Tests verify that the YouTube source tab renders, switching between sources
|
||||
* works, and components are conditionally rendered based on source state.
|
||||
*/
|
||||
import React from 'react'
|
||||
import { render, screen, fireEvent } from '@testing-library/react'
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest'
|
||||
import { LTTPage } from '../pages/LTTPage'
|
||||
import type { YouTubeStreamResponse } from '../types'
|
||||
|
||||
const mockUseYouTubeExtract = () => ({
|
||||
mutate: vi.fn(),
|
||||
isPending: false,
|
||||
isError: false,
|
||||
error: null,
|
||||
reset: vi.fn(),
|
||||
})
|
||||
|
||||
vi.mock('../lib/queries', () => ({
|
||||
useQueryDocumentStream: () => ({
|
||||
extractedQuestions: null,
|
||||
answer: null,
|
||||
sources: null,
|
||||
subQuestionSources: null,
|
||||
phase: 'idle',
|
||||
historyId: null,
|
||||
error: null,
|
||||
mutate: vi.fn(),
|
||||
reset: vi.fn(),
|
||||
}),
|
||||
useVideoUpload: () => ({
|
||||
mutate: vi.fn(),
|
||||
isPending: false,
|
||||
isError: false,
|
||||
error: null,
|
||||
reset: vi.fn(),
|
||||
data: null,
|
||||
}),
|
||||
useYouTubeExtract: () => mockUseYouTubeExtract(),
|
||||
}))
|
||||
|
||||
vi.mock('../hooks/useVideoASR', () => ({
|
||||
useVideoASR: () => ({
|
||||
transcript: '',
|
||||
partialTranscript: '',
|
||||
isStreaming: false,
|
||||
status: 'idle',
|
||||
startStreaming: vi.fn(),
|
||||
stopStreaming: vi.fn(),
|
||||
}),
|
||||
}))
|
||||
|
||||
vi.mock('../hooks/useYouTubeASR', () => ({
|
||||
useYouTubeASR: () => ({
|
||||
transcript: '',
|
||||
partialTranscript: '',
|
||||
isStreaming: false,
|
||||
status: 'idle',
|
||||
startStreaming: vi.fn(),
|
||||
stopStreaming: vi.fn(),
|
||||
}),
|
||||
}))
|
||||
|
||||
vi.mock('../hooks/useFullTranscript', () => ({
|
||||
useFullTranscript: () => ({
|
||||
fullTranscript: '',
|
||||
isLoading: false,
|
||||
error: null,
|
||||
requestFullTranscript: vi.fn(),
|
||||
}),
|
||||
}))
|
||||
|
||||
vi.mock('hls.js', () => ({
|
||||
default: class MockHls {
|
||||
static isSupported = () => true
|
||||
static Events = {
|
||||
MANIFEST_PARSED: 'manifestParsed',
|
||||
ERROR: 'error',
|
||||
}
|
||||
on = vi.fn()
|
||||
loadSource = vi.fn()
|
||||
attachMedia = vi.fn()
|
||||
destroy = vi.fn()
|
||||
levels = [{ height: 720 }, { height: 480 }, { height: 360 }]
|
||||
autoLevelCapping = -1
|
||||
},
|
||||
}))
|
||||
|
||||
vi.mock('../lib/api', () => ({
|
||||
getVideoUrl: (id: string) => `http://localhost:8000/api/v1/video/${id}`,
|
||||
}))
|
||||
|
||||
describe('LTTPage YouTube source toggle (Phase 3.5)', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
})
|
||||
|
||||
it('source selector renders with Upload and YouTube tabs', () => {
|
||||
render(<LTTPage />)
|
||||
|
||||
expect(screen.getByTestId('source-selector')).toBeInTheDocument()
|
||||
expect(screen.getByTestId('source-tab-upload')).toBeInTheDocument()
|
||||
expect(screen.getByTestId('source-tab-youtube')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('default source is upload — VideoUpload visible', () => {
|
||||
render(<LTTPage />)
|
||||
|
||||
expect(screen.getByTestId('video-dropzone')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('switching to youtube shows YouTubeInput', () => {
|
||||
render(<LTTPage />)
|
||||
|
||||
fireEvent.click(screen.getByTestId('source-tab-youtube'))
|
||||
|
||||
expect(screen.getByTestId('youtube-url-input')).toBeInTheDocument()
|
||||
expect(screen.queryByTestId('video-dropzone')).not.toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('switching back to upload shows VideoUpload', () => {
|
||||
render(<LTTPage />)
|
||||
|
||||
fireEvent.click(screen.getByTestId('source-tab-youtube'))
|
||||
expect(screen.getByTestId('youtube-url-input')).toBeInTheDocument()
|
||||
|
||||
fireEvent.click(screen.getByTestId('source-tab-upload'))
|
||||
expect(screen.getByTestId('video-dropzone')).toBeInTheDocument()
|
||||
expect(screen.queryByTestId('youtube-url-input')).not.toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('Full Transcript button hidden when YouTube source is active', () => {
|
||||
render(<LTTPage />)
|
||||
|
||||
expect(screen.queryByText(/full transcript/i)).not.toBeInTheDocument()
|
||||
|
||||
fireEvent.click(screen.getByTestId('source-tab-youtube'))
|
||||
expect(screen.queryByText(/full transcript/i)).not.toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('QueryInput renders regardless of source', () => {
|
||||
render(<LTTPage />)
|
||||
|
||||
expect(screen.getByPlaceholderText(/ask a question/i)).toBeInTheDocument()
|
||||
|
||||
fireEvent.click(screen.getByTestId('source-tab-youtube'))
|
||||
expect(screen.getByPlaceholderText(/ask a question/i)).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('YouTubeVideoPlayer appears after successful extraction', () => {
|
||||
const mockStreamData: YouTubeStreamResponse = {
|
||||
video_id: 'testvid123',
|
||||
title: 'Test Stream',
|
||||
is_live: false,
|
||||
is_upcoming: false,
|
||||
video_proxy_url: 'http://localhost:8000/api/v1/youtube/proxy/manifest.m3u8?url=video',
|
||||
audio_proxy_url: 'http://localhost:8000/api/v1/youtube/proxy/manifest.m3u8?url=audio',
|
||||
thumbnail_url: 'http://localhost:8000/thumb.jpg',
|
||||
formats: [],
|
||||
error: null,
|
||||
}
|
||||
|
||||
let capturedOnSuccess: ((data: YouTubeStreamResponse) => void) | null = null
|
||||
|
||||
vi.doMock('../components/YouTubeInput', () => {
|
||||
const MockYouTubeInput = ({ onExtractSuccess }: { onExtractSuccess: (data: YouTubeStreamResponse) => void }) => {
|
||||
capturedOnSuccess = onExtractSuccess
|
||||
return <div data-testid="youtube-url-input">YouTube Input Mock</div>
|
||||
}
|
||||
MockYouTubeInput.displayName = 'YouTubeInput'
|
||||
return { YouTubeInput: MockYouTubeInput }
|
||||
})
|
||||
|
||||
vi.doMock('../components/YouTubeVideoPlayer', () => {
|
||||
const MockPlayer = () => <div data-testid="youtube-video-player-mock">Player Mock</div>
|
||||
MockPlayer.displayName = 'YouTubeVideoPlayer'
|
||||
return { YouTubeVideoPlayer: MockPlayer }
|
||||
})
|
||||
|
||||
vi.resetModules()
|
||||
|
||||
return import('../pages/LTTPage').then(async ({ LTTPage: FreshLTTPage }) => {
|
||||
render(<FreshLTTPage />)
|
||||
|
||||
fireEvent.click(screen.getByTestId('source-tab-youtube'))
|
||||
expect(screen.getByTestId('youtube-url-input')).toBeInTheDocument()
|
||||
|
||||
if (capturedOnSuccess) {
|
||||
capturedOnSuccess(mockStreamData)
|
||||
}
|
||||
|
||||
expect(await screen.findByTestId('youtube-video-player-mock')).toBeInTheDocument()
|
||||
expect(screen.queryByTestId('youtube-url-input')).not.toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
})
|
||||
|
|
@ -0,0 +1,179 @@
|
|||
/**
|
||||
* Phase 3.5 tests: useYouTubeASR hook state management.
|
||||
*
|
||||
* WebAudio (AudioContext, ScriptProcessorNode) and WebSocket are NOT available
|
||||
* in jsdom, so these tests verify state management, return shape, and cleanup
|
||||
* logic only. Full audio capture is covered by acceptance tests.
|
||||
*/
|
||||
import { renderHook, act } from '@testing-library/react'
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest'
|
||||
import { useYouTubeASR } from '../hooks/useYouTubeASR'
|
||||
import type { ASRStatus } from '../types'
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
})
|
||||
|
||||
describe('useYouTubeASR', () => {
|
||||
it('test_initial_state', () => {
|
||||
const { result } = renderHook(() =>
|
||||
useYouTubeASR({
|
||||
videoId: 'test-video-id',
|
||||
videoElement: null,
|
||||
audioElement: null,
|
||||
})
|
||||
)
|
||||
|
||||
expect(result.current.transcript).toBe('')
|
||||
expect(result.current.partialTranscript).toBe('')
|
||||
expect(result.current.isStreaming).toBe(false)
|
||||
expect(result.current.status).toBe<ASRStatus>('idle')
|
||||
})
|
||||
|
||||
it('test_returns_startStreaming_function', () => {
|
||||
const { result } = renderHook(() =>
|
||||
useYouTubeASR({
|
||||
videoId: 'test-video-id',
|
||||
videoElement: null,
|
||||
audioElement: null,
|
||||
})
|
||||
)
|
||||
|
||||
expect(typeof result.current.startStreaming).toBe('function')
|
||||
})
|
||||
|
||||
it('test_returns_stopStreaming_function', () => {
|
||||
const { result } = renderHook(() =>
|
||||
useYouTubeASR({
|
||||
videoId: 'test-video-id',
|
||||
videoElement: null,
|
||||
audioElement: null,
|
||||
})
|
||||
)
|
||||
|
||||
expect(typeof result.current.stopStreaming).toBe('function')
|
||||
})
|
||||
|
||||
it('test_stopStreaming_resets_state', () => {
|
||||
const { result } = renderHook(() =>
|
||||
useYouTubeASR({
|
||||
videoId: 'test-video-id',
|
||||
videoElement: null,
|
||||
audioElement: null,
|
||||
})
|
||||
)
|
||||
|
||||
act(() => {
|
||||
result.current.stopStreaming()
|
||||
})
|
||||
|
||||
expect(result.current.status).toBe<ASRStatus>('idle')
|
||||
expect(result.current.isStreaming).toBe(false)
|
||||
expect(result.current.partialTranscript).toBe('')
|
||||
})
|
||||
|
||||
it('test_startStreaming_without_elements_does_not_throw', () => {
|
||||
const { result } = renderHook(() =>
|
||||
useYouTubeASR({
|
||||
videoId: 'test-video-id',
|
||||
videoElement: null,
|
||||
audioElement: null,
|
||||
})
|
||||
)
|
||||
|
||||
expect(() => {
|
||||
act(() => {
|
||||
result.current.startStreaming()
|
||||
})
|
||||
}).not.toThrow()
|
||||
})
|
||||
|
||||
it('test_startStreaming_with_no_elements_sets_error_status', () => {
|
||||
const { result } = renderHook(() =>
|
||||
useYouTubeASR({
|
||||
videoId: 'test-video-id',
|
||||
videoElement: null,
|
||||
audioElement: null,
|
||||
})
|
||||
)
|
||||
|
||||
act(() => {
|
||||
result.current.startStreaming()
|
||||
})
|
||||
|
||||
expect(['idle', 'error']).toContain(result.current.status)
|
||||
})
|
||||
|
||||
it('test_cleanup_on_unmount', () => {
|
||||
const { result, unmount } = renderHook(() =>
|
||||
useYouTubeASR({
|
||||
videoId: 'test-video-id',
|
||||
videoElement: null,
|
||||
audioElement: null,
|
||||
})
|
||||
)
|
||||
|
||||
expect(() => {
|
||||
unmount()
|
||||
}).not.toThrow()
|
||||
})
|
||||
|
||||
it('test_accepts_language_option', () => {
|
||||
const { result } = renderHook(() =>
|
||||
useYouTubeASR({
|
||||
videoId: 'test-video-id',
|
||||
videoElement: null,
|
||||
audioElement: null,
|
||||
language: 'en',
|
||||
})
|
||||
)
|
||||
|
||||
expect(result.current.status).toBe<ASRStatus>('idle')
|
||||
})
|
||||
|
||||
it('test_accepts_onFinalTranscript_callback', () => {
|
||||
const onFinal = vi.fn()
|
||||
const { result } = renderHook(() =>
|
||||
useYouTubeASR({
|
||||
videoId: 'test-video-id',
|
||||
videoElement: null,
|
||||
audioElement: null,
|
||||
onFinalTranscript: onFinal,
|
||||
})
|
||||
)
|
||||
|
||||
expect(result.current.status).toBe<ASRStatus>('idle')
|
||||
})
|
||||
|
||||
it('test_status_type_covers_all_states', () => {
|
||||
const validStatuses: ASRStatus[] = ['idle', 'connecting', 'streaming', 'disconnected', 'error']
|
||||
expect(validStatuses).toHaveLength(5)
|
||||
expect(validStatuses).toContain('idle')
|
||||
expect(validStatuses).toContain('connecting')
|
||||
expect(validStatuses).toContain('streaming')
|
||||
expect(validStatuses).toContain('disconnected')
|
||||
expect(validStatuses).toContain('error')
|
||||
})
|
||||
|
||||
it('test_startStreaming_with_audio_only', () => {
|
||||
const mockAudioElement = {
|
||||
crossOrigin: null,
|
||||
addEventListener: vi.fn(),
|
||||
removeEventListener: vi.fn(),
|
||||
} as unknown as HTMLAudioElement
|
||||
|
||||
const { result } = renderHook(() =>
|
||||
useYouTubeASR({
|
||||
videoId: 'test-video-id',
|
||||
videoElement: null,
|
||||
audioElement: mockAudioElement,
|
||||
})
|
||||
)
|
||||
|
||||
expect(() => {
|
||||
act(() => {
|
||||
result.current.startStreaming()
|
||||
})
|
||||
}).not.toThrow()
|
||||
})
|
||||
})
|
||||
Loading…
Reference in New Issue