Commit Graph

5 Commits

Author SHA1 Message Date
Woody f9dda7bd18 feat(backend): rename keywords to extracted_questions in query pipeline (sub-phase 2.3)
Change QueryDecomposer prompt to generate 2-5 sub-questions instead of keywords. Rename API field from keywords to extracted_questions across models, service, and router.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-04-24 16:23:53 +08:00
Woody 33b960f786 fix(backend): extract JSON from markdown code blocks in LLM responses
The LLM (Qwen3.5 via OpenRouter) returns JSON wrapped in markdown code blocks:

```json

["project manager", "limits", ...]

```

But the code was trying to parse this directly with json.loads(), causing:

- QueryDecomposer to return empty keywords

- RelevanceFilter to fail with "Expecting value: line 1 column 1"

Changes:

- Added _extract_json_from_markdown() helper function to both modules

- Strips markdown code block markers (```json and ```) before JSON parsing

- Added unit tests for markdown code block handling

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus \u003cclio-agent@sisyphuslabs.ai\u003e
2026-04-23 16:28:07 +08:00
Woody f5cfe44183 feat(backend): add LLM monitoring with step names, timing, and prompt logging
- LLMClient.complete() now accepts step_name parameter to identify processing step

- Logs prompt preview (first 100 + last 100 chars) at INFO level

- Logs processing time in milliseconds with token usage stats

- Updated QueryDecomposer, RelevanceFilter, and RAGService to pass step names

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-04-23 14:51:57 +08:00
Woody f4d78b0b77 refactor(backend): update query decomposer, relevance filter, and RAG service
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-04-23 13:26:56 +08:00
Woody 181f4eca5b feat: Phase 1.3 query pipeline with decomposition, relevance filter, and response
- Add QueryDecomposer: extracts keywords from question via LLM JSON response
- Add RelevanceFilter: batch scores chunks 0-10, filters by threshold
- Add POST /api/v1/query endpoint with full 3-step pipeline:
  1. QueryDecomposer.decompose() → keywords
  2. RAGService.retrieve() → chunks from ChromaDB
  3. RelevanceFilter.filter() → score and filter chunks
  4. RAGService.generate_response() → bullet-point answer
- Fix SourceMetadata.upload_date type from datetime to str for flexibility
- Test-first: 13 new tests pass (5 decomposer, 5 relevance filter, 3 query endpoint)
- All Phase 1 tests: 41 passed, 2 skipped
2026-04-22 17:19:21 +08:00