docs: add sub-phase development rules and acceptance test structure
This commit is contained in:
parent
3c2d647943
commit
be48b1d8c7
59
AGENTS.md
59
AGENTS.md
|
|
@ -105,14 +105,69 @@ test_phase<N>_<module_or_feature>.py
|
||||||
- Use `tmp_path` fixture for ChromaDB test instances
|
- Use `tmp_path` fixture for ChromaDB test instances
|
||||||
- Each test file must have a module-level docstring describing coverage
|
- Each test file must have a module-level docstring describing coverage
|
||||||
|
|
||||||
|
## SUB-PHASE DEVELOPMENT
|
||||||
|
|
||||||
|
**Workflow**: Plan → Implement → Acceptance Test → Commit
|
||||||
|
|
||||||
|
### Sub-Phase Plan Template
|
||||||
|
|
||||||
|
Each sub-phase plan (stored in `.plans/`) must include:
|
||||||
|
1. **Objective** — What this sub-phase delivers
|
||||||
|
2. **Acceptance Criteria** — List of behaviors that must work
|
||||||
|
3. **Acceptance Tests** — `test_acceptance_<subphase>.py` file(s) with real environment
|
||||||
|
4. **Implementation Tasks** — Atomic steps to complete
|
||||||
|
|
||||||
|
### Acceptance Testing Rules
|
||||||
|
|
||||||
|
**Unit tests** (`test_phase*.py`) — mocked, fast, CI-safe
|
||||||
|
**Acceptance tests** (`test_acceptance_*.py`) — real environment, actual LLM/ASR calls
|
||||||
|
|
||||||
|
**Acceptance test requirements**:
|
||||||
|
- Run against real services (ChromaDB instance, actual LLM API, ASR if applicable)
|
||||||
|
- Name format: `test_acceptance_<subphase>_<feature>.py`
|
||||||
|
- Location: `backend/app/test/acceptance/`
|
||||||
|
- Use `pytest` markers: `@pytest.mark.acceptance` and `@pytest.mark.slow`
|
||||||
|
- Each acceptance test file must have docstring describing real environment setup
|
||||||
|
- Acceptance tests run manually before sub-phase completion, not in CI
|
||||||
|
|
||||||
|
**Example acceptance test**:
|
||||||
|
```python
|
||||||
|
"""Acceptance test: Phase 1 RAG query with real Qwen LLM.
|
||||||
|
|
||||||
|
Prerequisites:
|
||||||
|
- ChromaDB running (local or docker)
|
||||||
|
- .env configured with valid LLM_BASE_URL and LLM_API_KEY
|
||||||
|
- Test documents ingested via /api/v1/ingest
|
||||||
|
"""
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
@pytest.mark.acceptance
|
||||||
|
@pytest.mark.slow
|
||||||
|
def test_query_with_real_llm():
|
||||||
|
"""Query should return bullet-point answer from actual LLM."""
|
||||||
|
# Real HTTP call to LLM provider
|
||||||
|
# Real ChromaDB retrieval
|
||||||
|
pass
|
||||||
|
```
|
||||||
|
|
||||||
|
**Sub-phase completion checklist**:
|
||||||
|
- [ ] All unit tests pass (`pytest app/test/test_phase*.py -v`)
|
||||||
|
- [ ] All acceptance tests pass (`pytest app/test/acceptance/ -v -m acceptance`)
|
||||||
|
- [ ] Code reviewed (self or peer)
|
||||||
|
- [ ] Sub-phase plan marked complete in `.plans/`
|
||||||
|
- [ ] Git commit with clear message referencing sub-phase plan
|
||||||
|
|
||||||
## COMMANDS
|
## COMMANDS
|
||||||
```bash
|
```bash
|
||||||
# Dev
|
# Dev
|
||||||
backend: uvicorn app.main:app --reload --port 8000
|
backend: uvicorn app.main:app --reload --port 8000
|
||||||
frontend: npm run dev
|
frontend: npm run dev
|
||||||
|
|
||||||
# Test
|
# Unit tests (mocked, CI-safe)
|
||||||
backend: cd backend && pytest app/test/ -v
|
backend: cd backend && pytest app/test/test_phase*.py -v
|
||||||
|
|
||||||
|
# Acceptance tests (real LLM/ASR/ChromaDB)
|
||||||
|
backend: cd backend && pytest app/test/acceptance/ -v -m acceptance
|
||||||
|
|
||||||
# Prod
|
# Prod
|
||||||
docker-compose up -d
|
docker-compose up -d
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,22 @@
|
||||||
|
"""Acceptance test: Phase 1 document ingestion with real ChromaDB.
|
||||||
|
|
||||||
|
Prerequisites:
|
||||||
|
- ChromaDB running with persistent storage
|
||||||
|
- Test PDF and TXT files available in test fixtures
|
||||||
|
- Embedding model accessible (local or remote)
|
||||||
|
"""
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.acceptance
|
||||||
|
@pytest.mark.slow
|
||||||
|
def test_ingest_pdf_with_real_embedding():
|
||||||
|
"""Should ingest PDF and create embeddings in real ChromaDB."""
|
||||||
|
pass # TODO: implement with real ChromaDB instance
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.acceptance
|
||||||
|
@pytest.mark.slow
|
||||||
|
def test_ingest_metadata_extraction():
|
||||||
|
"""Should extract and store metadata with real embedding pipeline."""
|
||||||
|
pass # TODO: implement with real metadata extraction service
|
||||||
|
|
@ -0,0 +1,15 @@
|
||||||
|
"""Acceptance test: Phase 1 RAG query with real Qwen LLM.
|
||||||
|
|
||||||
|
Prerequisites:
|
||||||
|
- ChromaDB running (local or docker)
|
||||||
|
- .env configured with valid LLM_BASE_URL and LLM_API_KEY
|
||||||
|
- Test documents ingested via /api/v1/ingest
|
||||||
|
"""
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.acceptance
|
||||||
|
@pytest.mark.slow
|
||||||
|
def test_query_with_real_llm():
|
||||||
|
"""Query should return bullet-point answer from actual LLM."""
|
||||||
|
pass # TODO: implement with real HTTP call to LLM provider and real ChromaDB retrieval
|
||||||
|
|
@ -0,0 +1,23 @@
|
||||||
|
"""Acceptance test: Phase 2 ASR transcription with real Qwen ASR model.
|
||||||
|
|
||||||
|
Prerequisites:
|
||||||
|
- ASR model endpoint accessible (local vLLM or cloud)
|
||||||
|
- Test audio file available
|
||||||
|
- WebSocket server running
|
||||||
|
"""
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.acceptance
|
||||||
|
@pytest.mark.slow
|
||||||
|
def test_asr_transcribe_real_audio():
|
||||||
|
"""Should return accurate transcript from real ASR model."""
|
||||||
|
pass # TODO: implement with real ASR API call
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.acceptance
|
||||||
|
@pytest.mark.slow
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_ws_asr_streaming():
|
||||||
|
"""Should stream audio chunks and receive transcripts via WebSocket."""
|
||||||
|
pass # TODO: implement with real WebSocket connection
|
||||||
|
|
@ -0,0 +1,22 @@
|
||||||
|
"""Acceptance test: Phase 2 video upload with real file storage.
|
||||||
|
|
||||||
|
Prerequisites:
|
||||||
|
- Backend server running (uvicorn)
|
||||||
|
- uploads/ directory writable
|
||||||
|
- Test video file < 300MB available
|
||||||
|
"""
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.acceptance
|
||||||
|
@pytest.mark.slow
|
||||||
|
def test_upload_video_mp4():
|
||||||
|
"""Should upload MP4 and return playable video URL."""
|
||||||
|
pass # TODO: implement with real HTTP POST to /api/v1/upload-video
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.acceptance
|
||||||
|
@pytest.mark.slow
|
||||||
|
def test_upload_size_rejection():
|
||||||
|
"""Should reject files over 300MB with proper error."""
|
||||||
|
pass # TODO: implement with real file upload attempt
|
||||||
|
|
@ -0,0 +1,15 @@
|
||||||
|
"""Acceptance test: End-to-end Phase 1 — text query → RAG → answer.
|
||||||
|
|
||||||
|
Prerequisites:
|
||||||
|
- Full backend running (uvicorn)
|
||||||
|
- ChromaDB initialized with test documents
|
||||||
|
- LLM provider configured and accessible
|
||||||
|
"""
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.acceptance
|
||||||
|
@pytest.mark.slow
|
||||||
|
def test_e2e_phase1_text_query():
|
||||||
|
"""Should ingest document and answer question with source attribution."""
|
||||||
|
pass # TODO: implement full flow: ingest → query → verify bullet answer
|
||||||
|
|
@ -0,0 +1,16 @@
|
||||||
|
"""Acceptance test: End-to-end Phase 2 — video → ASR → RAG → answer.
|
||||||
|
|
||||||
|
Prerequisites:
|
||||||
|
- Full backend running (uvicorn)
|
||||||
|
- ChromaDB initialized with test documents
|
||||||
|
- LLM and ASR providers configured and accessible
|
||||||
|
- Test video file with known audio content
|
||||||
|
"""
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.acceptance
|
||||||
|
@pytest.mark.slow
|
||||||
|
def test_e2e_phase2_video_query():
|
||||||
|
"""Should upload video, transcribe, and answer from transcript."""
|
||||||
|
pass # TODO: implement full flow: upload → ASR → query → verify answer
|
||||||
Loading…
Reference in New Issue