woody
/
legco_ai_assistant
1
1
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs: update testing rules to integration-first approach 

- Prefer integration tests (TestClient + real DB) over mocked unit tests
- Never mock database or internal services; only mock external APIs (LLM/ASR)
- Rename 'unit tests' to 'integration tests' throughout for consistency
This commit is contained in:
Woody 2026-04-27 10:55:43 +08:00
parent a72d5b0773
commit 7069f15b95
1 changed files with 12 additions and 10 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
AGENTS.md
View File

@ -125,11 +125,13 @@ test_phase<N>_<module_or_feature>.py
- `test_integration_phase1.py` — End-to-end text → RAG → answer - `test_integration_phase1.py` — End-to-end text → RAG → answer
- `test_integration_phase2.py` — End-to-end video → ASR → RAG → answer - `test_integration_phase2.py` — End-to-end video → ASR → RAG → answer
**Rules**: **Testing Rules (Python Backend)**:
- Use `pytest` + `pytest-asyncio` for async tests - **Prefer integration tests** over unit tests with mocks for all backend features and API routes.
- Mock all external LLM/ASR calls (do not hit live APIs in tests) - Use real application via `TestClient` (FastAPI). Never mock the database or internal services.
- Use `tmp_path` fixture for ChromaDB test instances - Use existing test database fixtures and `conftest.py`. Only mock truly external third-party APIs (LLM, ASR).
- Each test file must have a module-level docstring describing coverage - Match the exact style and imports of existing tests in the `tests/` directory.
- Always run `pytest` after writing tests and iterate until they pass against the real system.
- Each test file must have a module-level docstring describing coverage.
## SUB-PHASE DEVELOPMENT ## SUB-PHASE DEVELOPMENT
@ -154,7 +156,7 @@ Every sub-phase follows **test-driven delivery**:
1. **Write test first** — Before writing implementation code, write the test that defines "done" 1. **Write test first** — Before writing implementation code, write the test that defines "done"
2. **Implement** — Write the minimum code to make the test pass 2. **Implement** — Write the minimum code to make the test pass
3. **Run test** — Verify test passes (both unit and acceptance where applicable) 3. **Run test** — Verify test passes (both integration and acceptance where applicable)
4. **Commit** — Only commit after tests pass. Never commit broken tests. 4. **Commit** — Only commit after tests pass. Never commit broken tests.
5. **Next sub-phase** — Only start next sub-phase after current is committed 5. **Next sub-phase** — Only start next sub-phase after current is committed
@ -174,7 +176,7 @@ Each sub-phase plan (stored in `.plans/`) must include:
### Acceptance Testing Rules ### Acceptance Testing Rules
**Unit tests** (`test_phase*.py`) — mocked, fast, CI-safe **Integration tests** (`test_phase*.py`) — TestClient + real DB, only external APIs mocked, CI-safe
**Acceptance tests** (`test_acceptance_*.py`) — real environment, actual LLM/ASR calls **Acceptance tests** (`test_acceptance_*.py`) — real environment, actual LLM/ASR calls
**Acceptance test requirements**: **Acceptance test requirements**:
@ -206,8 +208,8 @@ def test_query_with_real_llm():
``` ```
**Sub-phase completion checklist**: **Sub-phase completion checklist**:
- [ ] All unit tests written BEFORE implementation - [ ] All integration tests written BEFORE implementation
- [ ] All unit tests pass (`pytest app/test/test_phase*.py -v`) - [ ] All integration tests pass (`pytest app/test/test_phase*.py -v`)
- [ ] All acceptance tests pass (`pytest app/test/acceptance/ -v -m acceptance`) - [ ] All acceptance tests pass (`pytest app/test/acceptance/ -v -m acceptance`)
- [ ] Code reviewed (self or peer) - [ ] Code reviewed (self or peer)
- [ ] Sub-phase plan marked complete in `.plans/` - [ ] Sub-phase plan marked complete in `.plans/`
@ -219,7 +221,7 @@ def test_query_with_real_llm():
backend: uvicorn app.main:app --reload --port 8000 backend: uvicorn app.main:app --reload --port 8000
frontend: npm run dev frontend: npm run dev
# Unit tests (mocked, CI-safe) # Integration tests (TestClient, real DB, only external APIs mocked)
backend: cd backend && pytest app/test/test_phase*.py -v backend: cd backend && pytest app/test/test_phase*.py -v
# Acceptance tests (real LLM/ASR/ChromaDB) # Acceptance tests (real LLM/ASR/ChromaDB)