Log the complete prompt, schema, extra_body content, full API response,
token counts, and full parsed JSON output. Add exc_info=True tracebacks
on all failure paths.
vLLM servers support JSON schema enforcement via extra_body (guided_json
or structured_outputs), not OpenAI's response_format protocol. LangChain's
with_structured_output(method='json_schema') sends response_format which
vLLM ignores, causing NoneType not iterable parsing errors.
- vLLM path: direct OpenAI SDK call with extra_body={guided_json|structured_outputs}
- OpenRouter path: unchanged with_structured_output(method='json_schema')
- Try new 'structured_outputs' format first, fall back to legacy 'guided_json'
- Update _SEED_DECOMPOSE with explicit JSON array instruction
- Add diagnostic logging: exc_info=True, schema preview, prompt template preview
- Add logging in _parse_legacy_json for fallback failure debugging
vLLM's chat_template_kwargs leaked into LangChain's AsyncCompletions.parse()
via _get_langchain_model's model_kwargs, causing structured decomposition
to fail on vLLM backends. Skip vLLM-specific params when building the
LangChain model — only provider-agnostic params (OpenAI reasoning) pass through.
- Add highlight_prompt, highlight_response, highlight_time_ms to QueryHistoryDetail type
- Add 'Highlights' bar segment with pink color in TimingBar component
- Pass highlightTimeMs to TimingBar in HistoryCard expanded view
- Add collapsible sections for highlight prompt and response in HistoryCard detail
- Mark Phase 5.4 complete with actual commit log
- Add Phase 5.4 completion checklist (15 items all checked)
- Add production notes (Vite proxy, port conflicts, cache location)
- Update test counts to current (108 backend, 45 frontend, 153 total)
- Update Decision #12 to reflect inline citation link upgrade
- Vite dev server doesn't proxy /api/v1/v2/ paths to backend
- Changed fetch URL and getHighlightUrl to use http://localhost:8000
- Fixed inline citation highlight URLs in buildCitationUrl
- Cleaned up debug code
- Added sub_question_text to frontend SourceMetadata type
- SubQuestionSection enriches sources with parent sub-question text
- buildCitationUrl routes to highlight page when sub_question_text present
- processCitations threads highlightReadyKeys through inline citations
- citationParser.ts: extractCitedSources() parses answer text for [citations],
resolves against SourceMetadata, returns deduplicated cited sources
- ResponsePanel.tsx: useEffect fires POST /api/v1/v2/highlights/batch after
answer renders; View PDF link upgrades in-place to highlighted HTML when
batch completes; stays as raw PDF on failure
- Updated plan: LLM-based relevance detection, eager background computation,
single batched LLM call, sqlite cache, regex sentence splitter
- 45 frontend tests: 28 citationParser + 17 ResponsePanel (including 4 new
sub-question highlight tests)
- POST /api/v1/v2/highlights/batch: compute and cache highlights for cited chunks
- GET /api/v1/v2/highlights: serve cached highlighted HTML pages
- chunks.py router registered in main.py
- Dynamic DB path computation (prompts.db -> highlights.db), no Settings changes
- 7 endpoint tests: POST 200/422, GET 200/404, mock service verification
- ChunkHighlightService.compute_highlights_batch(): single LLM call across
all cited chunks, grouped by sub-question, with structured output
- render_highlight_html(): self-contained HTML page with yellow-highlighted
relevant sentences, LLM reason annotations, and View Original PDF footer
- Per-target error isolation, ChromaDB miss handling, graceful degradation
- 14 tests: 7 batch service + 7 HTML rendering
DOCX and TXT ingestion now produces chunk_file_path + per-chunk PDF files matching the PDF ingestion flow. Uses reportlab to render chunk text as simple PDFs with automatic text wrapping.
- Add reportlab==4.2.5 to requirements.txt
- New utils/text_to_pdf.py: generate_text_pdf() renders chunk text as PDF
- Ingest router DOCX/TXT branches: generate chunk_N.pdf per chunk, store in chunk_file_paths
- Graceful degradation: chunk_file_path stays None if PDF generation fails
- Update test_phase1_ingest_page_aware.py assertions: DOCX chunks now HAVE chunk_file_path
- New test_phase5_docx_pdf_generation.py: 5 tests (DOCX PDF gen, TXT PDF gen, PDF regression, file count, graceful degradation)
- 361 backend tests pass (4 pre-existing embedding failures unrelated)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
LLMs may cite chunks from one sub-question's context inside another sub-question's answer section. Previously, processCitationsForSubq only looked up the current sub-question's sources, leaving cross-referenced citations unlinked. Now SubQuestionSection passes all sub-question sources and uses processCitations with a combined flat lookup.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Multi-stage Dockerfile: Node builds frontend, Python serves both API
and static files. docker-compose.yml with named volumes for ChromaDB,
chunks, and SQLite data. nginx.conf as reverse proxy with 350M upload
limit and 300s LLM proxy timeout. README with dev setup, deploy steps,
env vars table, and architecture diagram.
Backend main.py: add catch-all route to serve frontend/dist/static
files in production. Only activates when dist/ exists.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Add INFO log in get_settings() to print the actual model names
after merging .env and class defaults. Confirms pydantic-settings
priority: env values override class defaults as expected.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
The per-sub-question filter was all-or-nothing: if the LLM returned
9 scores for 10 chunks (common with qwen3.5-35b), every chunk was
discarded and the user got 'no relevant information found'.
Now: fewer scores → pad with 0.0; more scores → truncate. Changed
from error→warning since this is recoverable.
Also improve LTT page UI: sources collapsed by default in per-sub-q
sections, and the 'Your question' text now shows the full question
instead of being truncated.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Replace mocked DB/internal-services with real ChromaDB/SQLite via tmp_path.
Only mock truly external APIs (LLM, embedding for deterministic vectors).
13 test files rewritten (314 pass, 0 fail):
- Route tests: use TestClient + real ChromaDB, seed test data
- Service tests: use real PersistentClient/SQLite instances
- Pipeline tests: TestClient hits SSE /query endpoint, verify history
- Converted unittest.TestCase to pytest where applicable
Plus: fix metadata.py to filter None values from ChromaDB metadata
(pre-existing bug caught by real-DB ingestion tests)
Break the hardcoded per-sub-q filter prompt into 3 editable PromptService templates (filter_intro, filter_section, filter_outro) with placeholders for the for-loop iteration pattern. Refactor RelevanceFilter._build_per_subq_prompt() to compose them at runtime, falling back to built-in defaults when PromptService is unavailable.
Fix two latent bugs from Package 4:
- generate_per_subq was called by rag.py but never added to _VALID_STEPS or DB seed (would ValueError at runtime)
- _SEED_GENERATE placeholder mismatch: flat generate_response() expects {question}/{context} but Package 4 changed it to {context_sections}. Restored flat template; generate_per_subq now holds {context_sections}.
Add database backfill migration in seed_default_profiles() to INSERT OR IGNORE missing steps into existing profile rows, ensuring all 7 steps exist on restart.
Restructure System Prompts UI: remove unused flat filter/generate steps, replace with Step 2.1-2.3 (filter_intro/section/outro) and Step 3 (generate_per_subq). Update PlaceholderDocs with {context_sections}, {subq_idx}, {subq_question}.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
- Prefer integration tests (TestClient + real DB) over mocked unit tests
- Never mock database or internal services; only mock external APIs (LLM/ASR)
- Rename 'unit tests' to 'integration tests' throughout for consistency
Replace flat 3-step LLM workflow with per-sub-question architecture diagram. Document per-sub-question retrieval, filtering (single LLM call with sub-q grouping), and response generation with ## headers. Update CODE MAP to reflect completed implementation status. Add SSE event sequence with generating_subquestion events, history XML format with <sub_q> wrappers, and sources as list-of-lists JSON.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
6 stream state tests for completed, backward compat, generating_subquestion, null sources, lifecycle, and reset. 7 type tests for SubQuestionSources shape, discriminated union narrowing, and QueryResponse. 8 ResponsePanel tests for per-subq sections, source scoping, backward compat flat rendering, copy button, skeletons, and empty states. 7 citationParser tests for per-subq lookup, cross-section isolation, and processCitationsForSubq. 2 e2e tests for per-subq SSE display and progressive generation.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Redesign ResponsePanel with SubQuestionSections component that parses answer markdown on ## Sub-question N: boundaries and renders per-sub-question cards with headers, ReactMarkdown body, and collapsible sources scoped to each section. Extract FlatResponse for backward compatibility when subQuestionSources is null. Add buildCitationLookupForSubq and processCitationsForSubq for per-sub-question citation lookup scope isolation. Add anchor links in ExtractedQuestionsDisplay that smooth-scroll to matching ResponsePanel sections. Pass subQuestionSources through LTTPage.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Add SubQuestionSources interface for grouped per-sub-question sources. Convert QueryStreamEvent from flat interface to discriminated union with 7 variants including generating_subquestion and completed with sub_question_sources. Add subQuestionSources to QueryStreamState. Update completed event handler to populate subQuestionSources. Make sub_question_sources optional for backward compatibility with old SSE format.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Add 6 tests for retrieve_per_subquestion and generate_response_per_subquestion to Phase 1 rag service tests. Add 4 tests for filter_per_subquestion to Phase 1 relevance filter tests. Add 2 tests for new {context_sections} generate template to Phase 3 prompt injection tests. Add TestPerSubQPipelineHistory class with 3 per-sub-q pipeline simulation tests to Phase 3 integration tests. Add generate_per_subq template seed to conftest mock_prompt_service fixture.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
5 integration tests simulating full per-sub-question pipeline with mocked services covering 2-sub-q, empty decomposition fallback, single sub-q, all-filtered, and partial retrieval. 2 acceptance tests (manual run) for real LLM verification of per-sub-question organized answers with grouped sources and ## Sub-question headers.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
9 tests for generate_response_per_subquestion() and answer format validation covering multi-sub-q, empty, prompt construction, and markdown format. 8 tests for new history XML/JSON formats (sources as list-of-lists, <sub_q> wrappers in XML) and new {context_sections} prompt template.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Woody