Commit Graph

23 Commits

Author SHA1 Message Date
Woody 67d2bddeb6 fix: use relative /api/v1 fallback instead of hardcoded localhost:8000
API URLs now resolve relative to the page origin, working for both local dev (via Vite proxy) and remote production deployments.

Also fixes useFullTranscript which had a double /api/v1 path bug when VITE_API_BASE_URL was set.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-05-18 17:27:28 +08:00
Woody a54d688867 fix: use VITE_API_BASE_URL for highlight endpoints instead of hardcoded localhost
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-05-18 16:31:16 +08:00
Woody c8a9c857f7 Merge branch 'Highlight-Response' 2026-05-15 12:05:17 +08:00
Woody 62db325f02 fix: add rehype-raw to ReactMarkdown so ==term== <mark> HTML renders
Without rehype-raw, ReactMarkdown escaped the raw <mark> HTML injected
by highlightTerms(), showing literal tags instead of yellow highlights.
Now 30 marks render with correct bg-yellow-200 (#FEF08A) background.
2026-05-15 12:05:07 +08:00
Woody 29b4713f22 Merge branch 'Highlight-Response' 2026-05-15 11:23:02 +08:00
Woody 787c6b1692 fix: vLLM highlight batch failure — replace guided_json with response_format + add debug logging
Root cause: guided_json removed in vLLM v0.12.0, and the two-attempt
loop (structured_outputs → guided_json) merged chat_template_kwargs
into the extra_body, potentially causing param conflicts.

Changes:
- llm_client.py: Replace _complete_structured_vllm() with two-tier
  approach — response_format (Tier 1, v0.6.4+) then structured_outputs
  (Tier 2, v0.8+). Remove dead guided_json path. Add _strip_markdown_fence().

- chunk_highlight_service.py: Add complete() fallback as defense-in-depth
  when structured output fails. Strip markdown fences before parsing.

- chunks.py: Add request/response logging at router level.

- chunk_highlight_service.py: Add full logging chain — entry, ChromaDB
  fetch, LLM call, fallback, cache results, exit.

- ResponsePanel.tsx: Add console logging for request payload, response
  status/errors/timing. Handle status=failed explicitly (was silently
  ignored). Track round-trip timing via performance.now().
2026-05-15 11:08:36 +08:00
Woody e78f53b687 feat: Phase 7.2 — wire highlightTerms into ResponsePanel + mark CSS
- Add HighlightMark component rendering <mark class="bg-yellow-200...">
- Call highlightTerms() in SubQuestionSection and FlatResponse before ReactMarkdown
- Add mark: HighlightMark to ReactMarkdown components in both paths
- Add .prose mark CSS rule (yellow-200 bg, rounded, px-0.5)
- Tests: 56/56 pass (citation + highlight + ResponsePanel)
2026-05-15 10:51:08 +08:00
Woody 41f59b396f feat: track highlight generation prompt, response, and timing in history (Phase 5.5)
- Add 3 columns to query_history: highlight_prompt, highlight_response, highlight_time_ms
- HistoryService.update_highlights() updates existing row after batch LLM call
- ChunkHighlightService measures timing, captures prompt and structured JSON response
- SSE completed event includes history_id for frontend to pass back
- Frontend captures historyId, passes as ?history_id= query param in batch POST
- Highlight time tracked separately (excluded from total_time_ms)
- All 153 tests pass (108 backend + 45 frontend)
2026-04-29 11:18:21 +08:00
Woody b47e37f39b fix: use absolute backend URL for highlight API calls
- Vite dev server doesn't proxy /api/v1/v2/ paths to backend
- Changed fetch URL and getHighlightUrl to use http://localhost:8000
- Fixed inline citation highlight URLs in buildCitationUrl
- Cleaned up debug code
2026-04-29 10:39:01 +08:00
Woody bcf4a853bf feat: add highlight status toast notification (Phase 5.4)
- Shows 'Preparing highlights...' (amber spinner) while LLM batch runs
- Shows 'Highlights ready' (green) for 4 seconds when batch completes
- Fixed position top-left corner, auto-dismisses
2026-04-29 10:00:54 +08:00
Woody 1c490ce2fa fix: inline citations now upgrade to highlighted view (Phase 5.4)
- Added sub_question_text to frontend SourceMetadata type
- SubQuestionSection enriches sources with parent sub-question text
- buildCitationUrl routes to highlight page when sub_question_text present
- processCitations threads highlightReadyKeys through inline citations
2026-04-29 09:54:40 +08:00
Woody c632b9ea3b feat: cited source extraction, background batch trigger, and View PDF link upgrade (Phase 5.4.6-5.4.8)
- citationParser.ts: extractCitedSources() parses answer text for [citations],
  resolves against SourceMetadata, returns deduplicated cited sources
- ResponsePanel.tsx: useEffect fires POST /api/v1/v2/highlights/batch after
  answer renders; View PDF link upgrades in-place to highlighted HTML when
  batch completes; stays as raw PDF on failure
- Updated plan: LLM-based relevance detection, eager background computation,
  single batched LLM call, sqlite cache, regex sentence splitter
- 45 frontend tests: 28 citationParser + 17 ResponsePanel (including 4 new
  sub-question highlight tests)
2026-04-29 09:27:04 +08:00
Woody 4058c7dffe fix(citations): use all sub-question sources for citation lookup
LLMs may cite chunks from one sub-question's context inside another sub-question's answer section. Previously, processCitationsForSubq only looked up the current sub-question's sources, leaving cross-referenced citations unlinked. Now SubQuestionSection passes all sub-question sources and uses processCitations with a combined flat lookup.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-04-28 17:10:02 +08:00
Woody c43cb372e9 feat: integrate bullet points in ResponsePanel with CSS list-style
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-04-28 16:43:30 +08:00
Woody a7a22f1494 fix(relevance): tolerate LLM score count mismatches via padding instead of discarding
The per-sub-question filter was all-or-nothing: if the LLM returned
9 scores for 10 chunks (common with qwen3.5-35b), every chunk was
discarded and the user got 'no relevant information found'.

Now: fewer scores → pad with 0.0; more scores → truncate. Changed
from error→warning since this is recoverable.

Also improve LTT page UI: sources collapsed by default in per-sub-q
sections, and the 'Your question' text now shows the full question
instead of being truncated.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-04-27 14:31:18 +08:00
Woody 7d072e5ea1 feat(frontend): redesign ResponsePanel for per-sub-question sections with grouped sources
Redesign ResponsePanel with SubQuestionSections component that parses answer markdown on ## Sub-question N: boundaries and renders per-sub-question cards with headers, ReactMarkdown body, and collapsible sources scoped to each section. Extract FlatResponse for backward compatibility when subQuestionSources is null. Add buildCitationLookupForSubq and processCitationsForSubq for per-sub-question citation lookup scope isolation. Add anchor links in ExtractedQuestionsDisplay that smooth-scroll to matching ResponsePanel sections. Pass subQuestionSources through LTTPage.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-04-26 23:29:51 +08:00
Woody 3b741c1844 feat(query): stream extracted questions immediately via SSE
Convert /query endpoint from synchronous JSON to Server-Sent Events (SSE)
streaming. The frontend now receives extracted_questions as soon as the
first LLM call completes, without waiting for retrieval, filtering, and
answer generation.

Backend:
- Add StreamingQueryEvent union type (Decomposed, Retrieving, Filtering,
  Generating, Completed, Error)
- Convert /query to return StreamingResponse with SSE format
- Yield events after each pipeline phase

Frontend:
- Add queryDocumentStream() using fetch + ReadableStream
- Add useQueryDocumentStream() hook with phase-aware state
- Update LTTPage to use streaming instead of mutation
- Update ResponsePanel to show phase messages (Searching documents...,
  Filtering passages..., Generating answer...)
- Update ExtractedQuestionsDisplay to accept null

Tests:
- Update query_flow e2e test to mock queryDocumentStream
- 84/85 tests pass (1 pre-existing failure from removed file-input)
2026-04-25 18:29:22 +08:00
Woody f07e14aafd feat(frontend): render inline citations as clickable PDF links (sub-phase 2.6)
ResponsePanel now calls processCitations() on answer text before rendering.
Custom ReactMarkdown 'a' component adds target="_blank" to all citation links.
Adds tests for citation link rendering and unmatched citation fallback.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-04-24 17:53:25 +08:00
Woody a3028df4e1 feat(frontend): update View PDF links to open in-browser viewer (sub-phase 2.5)
ResponsePanel and ChunkList View PDF links now open /pdf-viewer page instead of raw PDF download.
Update ChunkList test mock from getChunkPdfUrl to getPdfViewerUrl.
Add DOMMatrix polyfill to test setup for react-pdf/jsdom compatibility.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-04-24 17:09:50 +08:00
Woody d49756f374 feat: add chunk PDF serving endpoint and frontend clickable source links (1.5.6)
- Add page_number and chunk_file_path to SourceMetadata model and query router
- Add GET /chunks/{file_path}/pdf endpoint with path traversal protection
- Add View PDF links in ResponsePanel source cards and ChunkList component
- Update TypeScript types and API helper for chunk PDF URLs
- Add backend tests (5) and frontend ChunkList tests (7)
- Update enhancement plan: all 3 features complete
2026-04-24 11:49:39 +08:00
Woody 52c09b86cb feat(frontend): add nav bar with routing, markdown rendering, and enhancement plan
- Add react-router-dom with NavBar component (LTT + RAG Database tabs)
- Extract AppContent into LTTPage, add RAGDatabasePage placeholder
- Refactor App.tsx to BrowserRouter + Routes layout
- Switch ResponsePanel to react-markdown for rich formatting
- Fix ResponsePanel test for markdown rendering
- Update RAG prompt to cite source name instead of number
- Save Phase 1 enhancement plan (.plans/phase1_enhancement_plan.md)
2026-04-23 18:37:30 +08:00
Woody f6618fd57e feat(frontend): polish ResponsePanel with collapsible sources, copy button, and enhanced skeletons
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-04-23 11:46:40 +08:00
Woody 3d76b894cb feat(frontend): add ResponsePanel component with bullet-point rendering and tests
Displays bullet-point answer with source metadata cards. Handles empty, loading, error, and success states.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-04-23 11:23:54 +08:00