docs: add Phase 5 OpenRouter ASR implementation plan

Complete implementation plan with architecture (Factory+Strategy pattern), provider comparison (DashScope vs OpenRouter), configuration, 7 implementation tasks, test plan, acceptance criteria, and implementation notes including decisions made (circular import resolution, separate API key, sync-to-async DashScope wrapper).

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

.plans/phase5_openrouter_asr.md

@@ -0,0 +1,423 @@
+# Phase 5: OpenRouter ASR Provider
+
+**Date:** 2026-05-18
+**Status:** ✅ Implemented (2026-05-19)
+**Source:** User request — add OpenRouter STT (`google/gemini-3.1-flash-lite`) as alternative ASR provider
+**Research:** OpenRouter STT docs + librarian agent (real-world code patterns) + explore agent (codebase architecture map)
+**Test Results:** 56/56 ASR tests pass (Phase 2 + Phase 5)
+
+---
+
+## 1. Objective
+
+Add OpenRouter as a second ASR provider for **batch transcription** (`transcribe_full`). The realtime WebSocket streaming mode remains DashScope-only because OpenRouter has no WebSocket STT endpoint.
+
+Users select the provider via a single env var. The existing REST endpoint `POST /api/v1/video/{video_id}/transcribe` and the WebSocket endpoint `/ws/asr/{video_id}` are unchanged from the frontend's perspective.
+
+---
+
+## 2. Scope
+
+| In Scope | Out of Scope |
+|----------|-------------|
+| OpenRouter batch transcription (`transcribe_full`) | Realtime WebSocket streaming (DashScope-only) |
+| `ASR_PROVIDER` env var switching | Frontend provider selector UI |
+| Provider abstraction (protocol class) | Changing existing DashScope code behavior |
+| Tests for new provider | Retraining/changing models |
+| `.env.example` update | Docker image rebuild |
+
+---
+
+## 3. Architecture
+
+### 3.1 Current Flow (DashScope-only)
+
+```
+POST /api/v1/video/{video_id}/transcribe
+  → video.py router
+  → VideoService.extract_audio() → WAV bytes
+  → ASRClient(settings).transcribe_full(audio_bytes, language)
+  → OpenAI SDK → DashScope Chat Completions API (audio input)
+  → return text
+```
+
+### 3.2 New Flow (Provider-based)
+
+```
+POST /api/v1/video/{video_id}/transcribe
+  → video.py router
+  → VideoService.extract_audio() → WAV bytes
+  → ASRClient(settings).transcribe_full(audio_bytes, language)
+      ├── ASR_PROVIDER=dashscope → DashScopeASRProvider (existing logic)
+      └── ASR_PROVIDER=openrouter → OpenRouterASRProvider (new)
+  → return text
+```
+
+### 3.3 Provider Interface (Factory + Strategy Pattern)
+
+Based on real-world multi-provider ASR patterns (DocsGPT, LiveKit, openai-agents-python), use **Factory + Strategy**:
+
+```python
+from abc import ABC, abstractmethod
+from typing import Protocol
+
+class ASRProvider(ABC):
+    """Abstract base for all ASR providers."""
+    @abstractmethod
+    async def transcribe(self, audio_bytes: bytes, language: str) -> str:
+        """Transcribe audio bytes to traditional Chinese text.
+
+        Raises ASRError on any failure (network, HTTP, empty response).
+        """
+        ...
+
+class ASRProviderFactory:
+    """Selects ASR provider based on settings."""
+    _providers: dict[str, type[ASRProvider]] = {}
+
+    @classmethod
+    def register(cls, name: str, provider_cls: type[ASRProvider]) -> None:
+        cls._providers[name] = provider_cls
+
+    @classmethod
+    def create(cls, name: str, settings) -> ASRProvider:
+        provider_cls = cls._providers.get(name)
+        if not provider_cls:
+            raise ValueError(f"Unknown ASR provider: {name}")
+        return provider_cls(settings)
+```
+
+**Why async?** The video router endpoint is already `async def`. The existing `transcribe_full` is sync (blocking), which blocks the event loop during 30-60s API calls. New providers should be async. Existing DashScope can be wrapped in `loop.run_in_executor()` temporarily.
+
+### 3.4 Existing Provider Pattern (LLMClient)
+
+The codebase already has a provider-switching pattern in `llm_client.py` — **single-class conditional branching**, not ABC/interface:
+
+```python
+# llm_client.py pattern:
+if settings.vllm_engine:
+    extra_body = {"chat_template_kwargs": {"enable_thinking": False}}
+else:
+    extra_body = {"reasoning": {"enabled": False}}
+```
+
+For ASR, the same pattern would mean `ASRClient` checks `settings.asr_provider` to select the right SDK/URL. However, since DashScope and OpenRouter use fundamentally different APIs (DashScope = Chat Completions + audio input; OpenRouter = dedicated STT endpoint), the **Factory+Strategy** pattern (Section 3.3) is cleaner for ASR — each provider gets its own class implementing a common interface.
+
+### 3.5 OpenRouter SDK vs Raw httpx
+
+| Trade-off | Raw httpx | OpenRouter SDK (`pip install openrouter`) |
+|-----------|-----------|------------------------------------------|
+| Type safety | Manual | Pydantic models |
+| Retry logic | Must implement (`tenacity`) | Built-in `retries=RetryConfig(...)` |
+| Production readiness | Battle-tested | Beta (auto-generated from OpenAPI) |
+| Dependencies | `httpx` (already installed) | SDK + Pydantic + extra deps |
+
+**Decision**: Use **raw httpx + tenacity** for Phase 5. This matches the approach used by most production Python projects (lethe, openclaw) and avoids beta SDK risk. The official SDK can be adopted later if it stabilizes.
+
+### 3.6 Retry & Error Handling
+
+Based on production OpenRouter STT implementations (lethe, openrouter-proxy):
+
+```python
+from tenacity import (
+    retry, stop_after_attempt, wait_random_exponential,
+    retry_if_exception_type
+)
+
+RETRIABLE_STATUS = {429, 500, 502, 503, 504}
+
+@retry(
+    reraise=True,
+    stop=stop_after_attempt(4),
+    wait=wait_random_exponential(multiplier=0.2, max=3.0),
+    retry=retry_if_exception_type((httpx.TransportError, httpx.HTTPStatusError)),
+)
+async def _call_stt_api(self, audio_b64: str, language: str) -> dict:
+    """Call OpenRouter STT with retry and exponential backoff."""
+    ...
+```
+
+Error categories to handle:
+| Error | Response | Retry? |
+|-------|----------|--------|
+| `httpx.HTTPStatusError` (429) | Rate limited | Yes (backoff) |
+| `httpx.HTTPStatusError` (5xx) | Server error | Yes (backoff) |
+| `httpx.HTTPStatusError` (4xx, non-429) | Client error | No |
+| `httpx.ConnectError` | Connection failed | Yes |
+| `httpx.TimeoutException` | Timeout (>120s) | Yes |
+| Empty `result["text"]` | No transcription | No |
+
+**Note:** `tenacity` is NOT currently in `requirements.txt`. Add it as a new dependency.
+
+### 3.7 API Differences
+
+| | DashScope | OpenRouter |
+|---|---|---|
+| Endpoint | `https://dashscope-intl.aliyuncs.com/compatible-mode/v1` | `https://openrouter.ai/api/v1/audio/transcriptions` |
+| Method | Chat Completions (`POST /chat/completions`) | Dedicated STT (`POST /audio/transcriptions`) |
+| Audio format | `data:audio/wav;base64,...` (data URL) | `{"data": "<base64>", "format": "wav"}` (raw base64) |
+| Auth | `DASHSCOPE_API_KEY` | `OPENROUTER_API_KEY` (separate key for accounting flexibility) |
+| Response | `choices[0].message.content` | `{"text": "...", "usage": {...}}` (no segments/timestamps/speaker labels) |
+| Response | `choices[0].message.content` | `{"text": "...", "usage": {...}}` |
+| SDK | `openai.OpenAI` | `httpx.AsyncClient` (no official SDK needed) |
+
+---
+
+## 4. Configuration
+
+### 4.1 New Env Vars
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `ASR_PROVIDER` | `dashscope` | ASR provider: `dashscope` or `openrouter` |
+| `OPENROUTER_API_KEY` | `""` | OpenRouter API key (for STT; separate from LLM_API_KEY for accounting) |
+| `ASR_OPENROUTER_MODEL` | `google/gemini-3.1-flash-lite` | OpenRouter STT model name |
+
+### 4.2 Settings Changes
+
+Add to `Settings` class in `config.py`:
+
+```python
+# ASR provider (Phase 5)
+asr_provider: str = "dashscope"  # "dashscope" or "openrouter"
+openrouter_api_key: str = ""     # separate from llm_api_key for accounting
+asr_openrouter_model: str = "google/gemini-3.1-flash-lite"
+```
+
+**Note:** OpenRouter STT uses:
+- `openrouter_api_key` — dedicated key (user preference for separate accounting)
+- `llm_base_url` — `https://openrouter.ai/api/v1` (base, STT endpoint appended: `/audio/transcriptions`)
+
+### 4.3 Validation
+
+Add a startup validation in `config.py` or `asr_client.py`:
+
+```python
+VALID_ASR_PROVIDERS = {"dashscope", "openrouter"}
+if settings.asr_provider not in VALID_ASR_PROVIDERS:
+    raise ValueError(f"Invalid ASR_PROVIDER: {settings.asr_provider}. Must be one of {VALID_ASR_PROVIDERS}")
+```
+
+---
+
+## 5. Implementation Tasks
+
+### Task 5.1: Add config vars and validation
+
+**File:** `backend/app/core/config.py`
+- Add `asr_provider: str = "dashscope"` 
+- Add `asr_openrouter_model: str = "google/gemini-3.1-flash-lite"`
+- Add `model_config` validation or runtime check in `get_settings()`
+
+**Test file:** `backend/app/test/test_phase5_config.py`
+
+### Task 5.2: Create OpenRouter ASR provider
+
+**File:** `backend/app/services/asr_providers.py` (new)
+
+```python
+class OpenRouterASRProvider:
+    def __init__(self, api_key: str, base_url: str, model: str):
+        self.api_key = api_key
+        # STT endpoint: base_url + /audio/transcriptions
+        self.stt_url = f"{base_url.rstrip('/')}/audio/transcriptions"
+        self.model = model
+        self._client: httpx.AsyncClient | None = None
+
+    async def transcribe(self, audio_bytes: bytes, language: str) -> str:
+        """Transcribe using OpenRouter STT endpoint."""
+        ...
+```
+
+**OpenRouter STT Request:**
+```python
+import base64
+import httpx
+
+audio_b64 = base64.b64encode(audio_bytes).decode("ascii")
+
+payload = {
+    "model": self.model,
+    "input_audio": {
+        "data": audio_b64,       # raw base64, NOT data URL
+        "format": "wav",
+    },
+}
+if language and language != "auto":
+    payload["language"] = language
+
+response = await client.post(
+    self.stt_url,
+    headers={
+        "Authorization": f"Bearer {self.api_key}",
+        "Content-Type": "application/json",
+    },
+    json=payload,
+    timeout=120.0,  # 60s upstream timeout + buffer
+)
+response.raise_for_status()
+result = response.json()
+return _to_traditional(result["text"])
+```
+
+**Key design notes:**
+- Uses `httpx.AsyncClient` (already in `requirements.txt`)
+- Base64 format: raw bytes, NOT `data:audio/wav;base64,...` (DashScope uses data URL; OpenRouter wants raw base64)
+- Timeout: 120s (OpenRouter docs say 60s upstream timeout; add buffer)
+- Error handling: raise custom `ASRError` on HTTP errors, network errors, or empty response text
+
+**Test file:** `backend/app/test/test_phase5_openrouter_provider.py`
+
+### Task 5.3: Refactor ASRClient to use provider abstraction
+
+**File:** `backend/app/services/asr_client.py`
+
+Changes:
+1. Define `ASRProvider` protocol (or ABC)
+2. Extract existing DashScope logic into `DashScopeASRProvider` (sync wrapper for now)
+3. `ASRClient.__init__` selects provider based on `settings.asr_provider`
+4. `ASRClient.transcribe_full` delegates to provider
+5. Make `transcribe_full` async (minor refactor to `video.py` router)
+
+**Backward compatibility:** Default `asr_provider=dashscope` means zero behavior change for existing deployments.
+
+**Test file:** `backend/app/test/test_phase2_asr_client.py` — update existing tests to work with new provider structure; add tests for provider switching.
+
+### Task 5.4: Update video router for async transcription
+
+**File:** `backend/app/routers/video.py`
+
+Minimal change — the `asr.transcribe_full()` call becomes `await asr.transcribe_full()`:
+
+```python
+# Before (line 113):
+text = asr.transcribe_full(audio_bytes, language=language)
+
+# After:
+text = await asr.transcribe_full(audio_bytes, language=language)
+```
+
+No other changes needed. The endpoint signature is already `async def`.
+
+### Task 5.5: Update .env.example and config documentation
+
+**File:** `backend/.env.example`
+- Add `ASR_PROVIDER` and `ASR_OPENROUTER_MODEL` comments
+
+**File:** `AGENTS.md` or development plan
+- Note the new Phase 5 capability
+
+### Task 5.6: Integration test (mock OpenRouter HTTP)
+
+**File:** `backend/app/test/test_phase5_integration.py`
+- Test full flow: video upload → transcribe with `ASR_PROVIDER=openrouter` → verify text
+- Mock `httpx.AsyncClient.post` to return valid OpenRouter STT response
+
+### Task 5.7: Acceptance test (real OpenRouter)
+
+**File:** `backend/app/test/acceptance/test_acceptance_phase5_openrouter.py`
+- Real OpenRouter API call with a short test audio file
+- Verify transcription quality
+- Marked `@pytest.mark.acceptance` and `@pytest.mark.slow`
+
+---
+
+## 6. Realtime ASR (Out of Scope — Future Consideration)
+
+OpenRouter has no WebSocket STT endpoint. For realtime streaming, we have two possible future approaches:
+
+1. **Chunked REST**: Send accumulated audio chunks to OpenRouter REST endpoint periodically. Latency ~2-5s per chunk. Not true realtime.
+2. **Stay DashScope-only**: Realtime WebSocket stays on DashScope. Users who need realtime must have a DashScope key.
+
+For Phase 5, realtime streaming **stays DashScope-only**. The `ws_asr.py` WebSocket endpoint is not modified. If `ASR_PROVIDER=openrouter` is set, it affects only the batch transcription path. The WebSocket endpoint continues to check `dashscope_api_key` independently.
+
+---
+
+## 7. Test Plan
+
+| Test File | What It Covers | Mock Strategy |
+|-----------|---------------|---------------|
+| `test_phase5_config.py` | Config validation, invalid provider rejection | No mocks (pure config) |
+| `test_phase5_openrouter_provider.py` | OpenRouterASRProvider unit tests | Mock `httpx.AsyncClient` |
+| `test_phase2_asr_client.py` (updated) | ASRClient with both providers | Mock DashScope + OpenRouter |
+| `test_phase5_integration.py` | Full video→transcribe with OpenRouter | Mock `httpx` (TestClient) |
+| `test_acceptance_phase5_openrouter.py` | Real OpenRouter API | None (real API) |
+
+**Test-first rule:** Write tests BEFORE implementation (per AGENTS.md convention). Each implementation task references its test file.
+
+---
+
+## 8. Acceptance Criteria
+
+- [ ] `ASR_PROVIDER=openrouter` in `.env` → batch transcription uses OpenRouter STT
+- [ ] `ASR_PROVIDER=dashscope` (default) → same behavior as before (backward compat)
+- [ ] Invalid `ASR_PROVIDER` value → clear error at startup
+- [ ] Realtime WebSocket ASR is unaffected (still DashScope-only, independent of `ASR_PROVIDER`)
+- [ ] OpenRouter transcription returns traditional Chinese (same `_to_traditional` conversion)
+- [ ] Error handling: network errors, HTTP errors, empty responses → clear error messages
+- [ ] All existing tests pass unchanged (with `ASR_PROVIDER=dashscope`)
+- [ ] New tests pass
+- [ ] Acceptance test returns valid transcription from real OpenRouter
+
+---
+
+## 9. Dependencies & Risks
+
+| Risk | Mitigation |
+|------|-----------|
+| OpenRouter STT latency > DashScope | Acceptable tradeoff; OpenRouter is cheaper and uses existing API key |
+| OpenRouter STT not as accurate for Cantonese | Need acceptance testing to verify; may need different model |
+| `transcribe_full` sync→async refactor could break callers | Only one caller (`video.py`); minimal blast radius |
+| No streaming/WebSocket for OpenRouter | Documented as limitation; realtime stays DashScope |
+| OpenRouter 60s timeout for long videos | Document limitation; large files may need chunking (future) |
+
+---
+
+## 10. Estimated Effort
+
+| Task | Est. Time |
+|------|-----------|
+| 5.1 Config | 15 min |
+| 5.2 OpenRouter provider | 30 min |
+| 5.3 Refactor ASRClient | 20 min |
+| 5.4 Update video router | 5 min |
+| 5.5 Update .env.example | 5 min |
+| 5.6 Integration test | 20 min |
+| 5.7 Acceptance test | 15 min |
+| **Total** | **~2 hours** |
+
+---
+
+## 11. Implementation Notes (2026-05-19)
+
+### Decisions During Implementation
+
+- **`_to_traditional` moved to `asr_providers.py`** — original plan placed it in `asr_client.py` with a cross-import, but this caused a circular import (`asr_client` → `asr_providers` → `asr_client`). Moved to `asr_providers.py`; `asr_client.py` re-exports for backward compatibility with `ws_asr.py`.
+- **Separate `OPENROUTER_API_KEY`** — per user preference for independent accounting.
+- **`DashScopeASRProvider` wraps sync OpenAI call in `loop.run_in_executor()`** — avoids blocking the event loop without rewriting the existing DashScope client.
+
+### Files Changed
+
+| File | Action | Details |
+|------|--------|---------|
+| `backend/app/core/config.py` | Modified | 3 new settings + validation in `get_settings()` |
+| `backend/app/services/asr_providers.py` | **New** | `ASRProvider` ABC, `DashScopeASRProvider`, `OpenRouterASRProvider` (with tenacity retry), `create_asr_provider()` factory |
+| `backend/app/services/asr_client.py` | Refactored | Thin wrapper; `transcribe_full` now async; utility functions retained |
+| `backend/app/routers/video.py` | Modified | `await transcribe_full()`; provider-aware API key validation |
+| `backend/.env.example` | Modified | Phase 5 vars with usage comments |
+| `backend/requirements.txt` | Modified | Added `tenacity>=8.0.0` |
+
+### Test Files
+
+| File | Tests | Status |
+|------|-------|--------|
+| `test_phase5_config.py` | 6 | ✅ |
+| `test_phase5_openrouter_provider.py` | 14 | ✅ |
+| `test_phase5_integration.py` | 4 | ✅ |
+| `test_phase2_asr_client.py` | 19 (3 updated) | ✅ |
+| `test_phase2_full_transcript.py` | 6 (updated fixtures) | ✅ |
+| `test_integration_phase2.py` | 7 (updated fixtures) | ✅ |
+
+### Pre-existing Test Failures (Unrelated)
+- Phase 3: `test_phase3_history_service.py`, `test_phase3_prompt_injection.py`, `test_phase3_prompt_service.py`, `test_phase3_prompts_router.py` — pre-existing failures in SQLite/prompt tests unrelated to ASR changes.
+- Phase 1: 1 config test — pre-existing, unrelated.