# Package 4 Enhancement Plan — Per-Sub-Question RAG Pipeline

**Source**: User request (2026-04-26)  
**Scope**: Refactor the 3-step RAG query pipeline so retrieval, filtering, and response generation are organized per sub-question instead of batch-flattened.  
**Status**: ✅ Complete — All 7 sub-phases implemented (2026-04-26). Phase 4a Prompt Integration added (2026-04-27).

---

## Objective

Restructure the `POST /api/v1/query` pipeline so that:

1. **Retrieval per sub-question**: Each sub-question independently retrieves `n_results` chunks from ChromaDB (instead of joining all sub-questions into one query string).
2. **Filtering per sub-question**: Each chunk is evaluated for relevance against its **own originating sub-question** (not the original user question). One LLM call handles all filtering — the prompt is redesigned to group chunks by sub-question.
3. **Final answer organized by sub-question**: Each sub-question gets its own bullet-point answer with its own sources. The frontend renders answer sections per sub-question rather than one monolithic bullet list.

---

## Decision Register

| # | Decision | Rationale |
|---|----------|-----------|
| 1 | Keep `QueryDecomposer` unchanged | Input/output contract is identical — decomposition still produces a flat list of sub-questions |
| 2 | Single LLM call for filtering | User explicitly requested one call. Prompt redesigned to carry sub-question context for each chunk group |
| 3 | Keep `RAGService.retrieve()` signature | Call it N times (once per sub-question) externally in the orchestrator rather than changing its internal contract |
| 4 | Add `retrieve_per_subquestion()` to `RAGService` | New method that iterates over sub-questions, calls `retrieve()` per question, returns grouped results |
| 5 | Redesign `generate_response()` signature | Accepts structured `sub_questions: List[SubQuestionContext]` instead of flat chunk lists |
| 6 | SSE events: add `generating_subquestion` phase | Progressive streaming — frontend sees which sub-question is being answered |
| 7 | History: change XML/JSON formats in-place | Add `<sub_q>` wrappers to `chunks_retrieved`/`chunks_filtered` XML. Add sub-question grouping to `sources` JSON. No new DB columns. |
| 8 | Final answer format: markdown sections | `## Sub-question 1` headers with inline citations. Backward-compatible with existing `ReactMarkdown` rendering |
| 9 | Deduplicate chunks within a sub-question only | Same chunk may be retrieved by multiple sub-questions. Keep duplicates (different sub-questions need independent evaluation). ChromaDB `query()` naturally may return the same doc for different queries — this is acceptable. |
|10 | Prompt template: add `generate` placeholders | New placeholder `{context_sections}` replaces single `{context}`. Filter template unchanged (sub-question injected at call site). Decompose template unchanged. |
|11 | Progressive SSE events | Emit `generating_subquestion` event as each sub-question's answer section is generated. Frontend renders sections one by one. |
|12 | `retrieval_n_results` | Global — same value for all sub-questions. Use existing `settings.retrieval_n_results` config. |
|13 | Empty decomposition fallback | Treat original user question as single sub-question. Pipeline runs as 1-sub-q case — single retrieval, no filtering needed (one sub-q = no ambiguity), flat answer with `##` header. |

---

## Pipeline: Before vs After

### Before (Current — Flat Batch)

```
User Question: "What are NEC4 time extension clauses?"
         │
    ┌────▼─────┐
    │ Decompose│  LLM Call 1
    │ → ["What are time extensions?", 
    │    "What notice is required?"]
    └────┬─────┘
         │ joined: "What are time extensions? What notice is required?"
    ┌────▼─────┐
    │ Retrieve │  1 ChromaDB query → 10 chunks (flat, no sub-q association)
    └────┬─────┘
         │ 10 chunks
    ┌────▼─────┐
    │  Filter  │  LLM Call 2 — all chunks scored against ORIGINAL question
    │          │  Score > 7 → keep (flat, no sub-q association)
    └────┬─────┘
         │ N filtered chunks
    ┌────▼─────┐
    │ Generate │  LLM Call 3 — flat answer from ALL filtered chunks
    │          │  "• Time extensions require notice [NEC4 ACC.pdf, p3]
    │          │   • The project manager must acknowledge [NEC4, p7]
    │          │   • Notice is defined as..."  (sources from all sub-qs mixed)
    └────┬─────┘
         │ single SSE completed event
    ┌────▼─────┐
    │ Frontend │  1 ReactMarkdown block, 1 flat sources list
    └──────────┘
```

### After (Per-Sub-Question)

```
User Question: "What are NEC4 time extension clauses?"
         │
    ┌────▼─────┐
    │ Decompose│  LLM Call 1 (UNCHANGED)
    │ → ["What are time extensions?",
    │    "What notice is required?"]
    └────┬─────┘
         │ sub_q1                    sub_q2
    ┌────▼─────┐              ┌────▼─────┐
    │ Retrieve │              │ Retrieve │   2 ChromaDB queries → 10 chunks each
    │ q1 → 10  │              │ q2 → 10  │   chunks tagged with sub-q index
    └────┬─────┘              └────┬─────┘
         │                         │
         └─────────┬───────────────┘
                   │ grouped: {sub_q0: [chunks 0-9], sub_q1: [chunks 10-19]}
              ┌────▼─────┐
              │  Filter  │  LLM Call 2 (SINGLE CALL — redesigned prompt)
              │          │  Each chunk scored against its OWN sub-question
              │          │  Returns grouped scores → filtered per sub-q
              └────┬─────┘
                   │ filtered_by_subq: {0: [chunk_a, chunk_b], 1: [chunk_c]}
              ┌────▼─────┐
              │ Generate │  LLM Call 3 (redesigned prompt with per-sub-q context)
              │          │  ┌─────────────────────────────────────┐
              │          │  │ ## What are time extensions?         │
              │          │  │ - Time extensions must be notified   │
              │          │  │   [NEC4 ACC.pdf, page 3]             │
              │          │  │ - The project manager has 2 weeks    │
              │          │  │   [NEC4 Contract.pdf, page 12]       │
              │          │  │                                      │
              │          │  │ ## What notice is required?          │
              │          │  │ - Written notice must be given       │
              │          │  │   [NEC4 ACC.pdf, page 7]             │
              │          │  └─────────────────────────────────────┘
              └────┬─────┘
                   │ SSE events: generating_subquestion (per sub-q) → completed
              ┌────▼─────┐
              │ Frontend │  Sections per sub-question, sources grouped per section
              └──────────┘
```

---

## Current State (Pre-Enhancement)

### Backend

| Component | File | Current Behavior |
|-----------|------|-----------------|
| Decomposer | `services/query_decomposer.py` | `decompose(question) -> (List[str], prompt)` — returns 2-5 sub-questions |
| Retrieval | `services/rag.py:retrieve()` | `query_text = " ".join(query_keywords)` — joins all sub-qs into ONE string, single ChromaDB query → flat chunk list |
| Filter | `services/relevance_filter.py` | `filter(question, chunks)` — ALL chunks scored against ORIGINAL question, single LLM call, flat output |
| Generate | `services/rag.py:generate_response()` | `generate_response(question, chunks, metadata)` — flat chunks → flat bullet answer |
| Orchestrator | `routers/query.py:_query_stream()` | Linear 4-stage pipeline: decompose → retrieve → filter → generate |
| SSE Events | `routers/query.py` | `decomposed → retrieving → filtering → generating → completed` — flat answer + sources in `completed` |
| History | `services/history_service.py` | Flat XML for `chunks_retrieved`/`chunks_filtered`. Flat JSON for `sources`. Single timing per stage. |
| Prompt templates | `prompt_service.py` + `sqlite_db.py` | 3 steps (`decompose`, `filter`, `generate`). Placeholders: `{question}`, `{chunks}`, `{context}` |
| Config | `core/config.py` | `retrieval_n_results=10`, `relevance_threshold=7.0` |

### Frontend

| Component | File | Current Behavior |
|-----------|------|-----------------|
| Types | `types/index.ts` | `QueryStreamEvent.phase`, flat `extracted_questions: string[]`, flat `answer: string`, flat `sources: SourceMetadata[]` |
| SSE Client | `lib/api.ts` | `queryDocumentStream()` — generic `JSON.parse` per `data:` line, no sub-question awareness |
| State | `lib/queries.tsx` | `QueryStreamState` with flat `answer`/`sources`/`extractedQuestions` |
| Response | `components/ResponsePanel.tsx` | Single `ReactMarkdown` block for answer. Flat 2-column grid for sources. No sub-question grouping. |
| Questions | `components/ExtractedQuestionsDisplay.tsx` | `<ol>` list of question strings. No sources attached. |
| Citations | `utils/citationParser.ts` | Flat `sources` lookup — `buildCitationLookup(sources)` returns global map |
| Progress | `components/PipelineProgress.tsx` | 4-step stepper (NOT currently wired in LTTPage) |

### Key Test Files

| File | Lines | Status |
|------|-------|--------|
| `test_phase1_query_decomposer.py` | 76 | ✅ Unchanged — decomposer contract stays |
| `test_phase1_rag_service.py` | 139 | 🔴 Needs update — `retrieve()`, `generate_response()` signatures change |
| `test_phase1_relevance_filter.py` | 93 | 🟡 Needs update — one-call pattern changes to per-sub-q grouping |
| `test_phase1_query.py` | 97 | 🟢 Already skipped (SSE migration) — may un-skip later |
| `test_phase3_query_history_integration.py` | 608 | 🔴 Major rewrite — pipeline simulation mirrors `_query_stream` 1:1 |
| `test_phase3_prompt_injection.py` | 238 | 🟡 Moderate — new generate template placeholder |
| `test_acceptance_phase1_rag_query.py` | 101 | 🔴 Full rewrite — already broken (SSE vs JSON), new response shape |
| `conftest.py` | 94 | 🟡 Low — may add per-sub-q mock helpers |

---

## Implementation Tasks

### Sub-Phase 4.1: Backend — Per-Sub-Question Retrieval

**Test files to write first:**
- `test_phase4_retrieve_per_subquestion.py` — Tests `RAGService.retrieve_per_subquestion()` 
- `test_phase4_query_router_retrieval.py` — Tests `_query_stream` retrieval stage produces per-sub-q chunks

**Task 4.1.1: Add `retrieve_per_subquestion()` to `RAGService`**

File: `backend/app/services/rag.py`

New method signature:
```python
def retrieve_per_subquestion(
    self,
    sub_questions: List[str],
    n_results: int = 10,
) -> List[Tuple[str, List[Tuple[str, Dict[str, Any], float]]]]:
    """Retrieve chunks for each sub-question independently.

    Args:
        sub_questions: List of decomposed sub-questions.
        n_results: Number of chunks per sub-question.

    Returns:
        List of (sub_question, chunks) tuples.
        chunks is the standard retrieve() output: [(text, metadata, distance), ...].
    """
```

Implementation:
- Call `self.retrieve([sub_q], n_results)` for each sub-question
- Return list of `(sub_question, chunks)` — chunks remain deduplicated at ChromaDB level (ChromaDB automatically deduplicates by ID)
- Existing `retrieve()` method is NOT modified — it continues to work as before

**Task 4.1.2: Update `_query_stream()` retrieval stage**

File: `backend/app/routers/query.py`

Changes:
- Replace `rag.retrieve(extracted_questions, n_results)` with `rag.retrieve_per_subquestion(extracted_questions, n_results)`
- Track per-sub-question retrieval timing (new field or combined timing)
- Format `chunks_retrieved` XML with sub-question wrappers

**New `chunks_retrieved` XML format:**
```xml
<sub_q idx="0" question="What are time extensions?">
<chunk_1>
Filename: NEC4 ACC.pdf
Page: 3
Content: Clause 61.3 states that...
</chunk_1>
<chunk_2>
...
</chunk_2>
</sub_q>
<sub_q idx="1" question="What notice is required?">
<chunk_1>
Filename: NEC4 Contract.pdf
Page: 12
Content: Notice must be given...
</chunk_1>
...
</sub_q>
```

**Task 4.1.3: Format helpers**

File: `backend/app/routers/query.py`

New functions:
```python
def format_chunks_retrieved_per_subq(results: List[Tuple[str, List]]) -> str:
    """Format per-sub-question retrieved chunks as XML."""
    
def format_chunks_filtered_per_subq(results: List[Tuple[str, List]]) -> str:
    """Format per-sub-question filtered chunks as XML with relevance scores."""
```

**Commit**: `"feat: Phase 4.1 per-sub-question retrieval with grouped chunk XML"`

### Sub-Phase 4.2: Backend — Per-Sub-Question Filtering (Single LLM Call)

**Test files to write first:**
- `test_phase4_relevance_filter_per_subq.py` — Tests `RelevanceFilter.filter_per_subquestion()` with grouped chunks
- `test_phase4_query_router_filter.py` — Tests filter stage with per-sub-q chunk groups

**Task 4.2.1: Add `filter_per_subquestion()` to `RelevanceFilter`**

File: `backend/app/services/relevance_filter.py`

New method signature:
```python
async def filter_per_subquestion(
    self,
    sub_questions: List[str],
    sub_chunks: List[List[Tuple[str, Dict]]],
    threshold: float = 7.0,
) -> Tuple[List[Tuple[str, List[Tuple[str, Dict]]]], str]:
    """Filter chunks per sub-question in a single LLM call.

    Args:
        sub_questions: List of decomposed sub-questions.
        sub_chunks: List of chunk lists (one per sub-question).
        threshold: Minimum relevance score.

    Returns:
        Tuple of (filtered_results, prompt).
        filtered_results: List of (sub_question, filtered_chunks_for_that_q).
    """
```

**Prompt design (single LLM call):**
```
Evaluate each chunk for relevance to its associated sub-question.

Sub-question 0: "{sub_q_0}"
Chunk 0: {chunk_0_text}
Chunk 1: {chunk_1_text}
...

Sub-question 1: "{sub_q_1}"
Chunk 0: {chunk_0_text}
Chunk 1: {chunk_1_text}
...

For each chunk, rate relevance 0-10 considering ONLY its associated sub-question.
Return a JSON object mapping sub-question indices to arrays of scores:
{"0": [8.5, 3.2, 9.0], "1": [7.0, 6.5, 9.1]}
```

Key rules:
- Each chunk is evaluated against its **own** sub-question (not the original user question)
- JSON keys are stringified sub-question indices (`"0"`, `"1"`, ...)
- Score arrays MUST match chunk count for each sub-question
- Same JSON extraction/markdown stripping logic as existing `filter()`

**Existing `filter()` method is preserved** — not modified, not deprecated. The new method is additive.

**Task 4.2.2: Update `_query_stream()` filter stage**

File: `backend/app/routers/query.py`

Changes:
- Call `relevance_filter.filter_per_subquestion(extracted_questions, chunks_for_filter, threshold)` instead of `relevance_filter.filter(question, chunks, threshold)`
- Build `chunks_for_filter` from per-sub-question retrieval results
- Track `filter_prompt` (the redesigned prompt)
- Format `chunks_filtered` XML with sub-question wrappers and `Relevance:` scores

**New `chunks_filtered` XML format:**
```xml
<sub_q idx="0" question="What are time extensions?">
<chunk_1>
Filename: NEC4 ACC.pdf
Page: 3
Relevance: 8.5
Content: Clause 61.3 states that...
</chunk_1>
</sub_q>
<sub_q idx="1" question="What notice is required?">
<chunk_1>
Filename: NEC4 Contract.pdf
Page: 12
Relevance: 9.0
Content: Notice must be given...
</chunk_1>
</sub_q>
```

**Commit**: `"feat: Phase 4.2 per-sub-question filtering with single LLM call"`

### Sub-Phase 4.3: Backend — Sub-Question-Organized Response Generation

**Test files to write first:**
- `test_phase4_generate_per_subq.py` — Tests `RAGService.generate_response_per_subquestion()` 
- `test_phase4_response_format.py` — Tests the final answer matches expected format

**Task 4.3.1: Redesign `generate_response()` → `generate_response_per_subquestion()`**

File: `backend/app/services/rag.py`

New method signature:
```python
async def generate_response_per_subquestion(
    self,
    sub_questions: List[str],
    sub_chunks: List[List[str]],
    sub_metadata: List[List[Dict[str, Any]]],
) -> Tuple[str, str, List[List[SourceMetadata]]]:
    """Generate sub-question-organized RAG response.

    Args:
        sub_questions: List of decomposed sub-questions.
        sub_chunks: List of chunk text lists (one per sub-question).
        sub_metadata: List of metadata dict lists (one per sub-question).

    Returns:
        Tuple of (answer, prompt, grouped_sources).
        answer: Markdown string with sections per sub-question.
        prompt: The rendered LLM prompt.
        grouped_sources: List of SourceMetadata lists (one per sub-question).
    """
```

**New prompt template (replaces `generate`):**
```
You must answer each sub-question using ONLY the document chunks provided for it.
Do not use any external knowledge.
Format your answer as markdown sections — one section per sub-question.
Each section should start with "## Sub-question N: <the question>"
Each section should contain 1-5 bullet points.
Cite your sources inline using bracket labels, e.g. [filename, page N].
Place the citation at the end of each relevant bullet point.

{context_sections}

Answer:
```

**Context format (replaces `{context}`):**
```
### Context for Sub-question 0: "What are time extensions?"
[NEC4 ACC.pdf, page 3] Source: NEC4 ACC.pdf
Summary: Clause 61.3 discusses time extensions...
Content: Clause 61.3 states that the project manager...

[NEC4 Contract.pdf, page 12] Source: NEC4 Contract.pdf
Summary: Notice requirements for time extensions...
Content: Written notice must be given within...

### Context for Sub-question 1: "What notice is required?"
[NEC4 ACC.pdf, page 7] Source: NEC4 ACC.pdf
Summary: Notice requirements...
Content: The contractor shall notify the project manager in writing...
```

**Expected answer format:**
```markdown
## Sub-question 1: What are time extensions?
- Time extensions must be notified to the project manager within 2 weeks [NEC4 ACC.pdf, page 3]
- The project manager must acknowledge the notice within 1 week [NEC4 Contract.pdf, page 12]

## Sub-question 2: What notice is required?
- Written notice must be given [NEC4 ACC.pdf, page 7]
```

**Existing `generate_response()` is preserved** — not modified, not deprecated.

**Task 4.3.2: Update `_query_stream()` generate stage**

File: `backend/app/routers/query.py`

Changes:
- Call `rag.generate_response_per_subquestion(extracted_questions, chunk_texts_by_subq, metadata_by_subq)`
- New SSE event: `generating_subquestion` — emitted before each sub-question's section (lets frontend show progressive build)
- `completed` SSE event includes both `answer` (markdown string) and `sub_question_sources` (grouped sources)

**New SSE event sequence:**
```json
{"phase": "decomposed", "extracted_questions": ["q1", "q2"]}
{"phase": "retrieving"}
{"phase": "filtering"}
{"phase": "generating"}
{"phase": "completed", "answer": "## Sub-question 1: ...\n\n...", "sub_question_sources": [[SourceMetadata, ...], [SourceMetadata, ...]]}
{"phase": "error", "message": "..."}
```

**New `QueryResponse` model:**

File: `backend/app/models/query.py`

```python
class SubQuestionSources(BaseModel):
    sub_question_index: int
    sub_question_text: str
    sources: List[SourceMetadata]

class QueryResponse(BaseModel):
    extracted_questions: List[str]
    answer: str                          # Markdown with ## sections
    sub_question_sources: List[SubQuestionSources]  # Grouped sources
    # Backward compat:
    sources: List[SourceMetadata]        # Flattened version (all sources)
```

**Commit**: `"feat: Phase 4.3 sub-question-organized response generation"`

### Sub-Phase 4.4: Backend — History & Prompt Template Updates

**Test files to write first:**
- `test_phase4_history_format.py` — Tests new XML/JSON history formats
- `test_phase4_prompt_templates.py` — Tests new generate template with `{context_sections}`

**Task 4.4.1: Update history recording**

File: `backend/app/routers/query.py` (the `_schedule_history` / `_record_history` helpers)

Changes:
- `chunks_retrieved`: Store new grouped XML format (with `<sub_q>` wrappers)
- `chunks_filtered`: Store new grouped XML format (with `<sub_q>` wrappers and `Relevance:` scores)
- `sources`: Store grouped JSON: `json.dumps([[SourceMetadata_dict, ...], [...]])` (list of lists)
- `final_answer`: Store markdown string with `##` sections
- Existing fields (`chunks_retrieved_count`, `chunks_filtered_count`) keep total counts
- New optional fields: `chunks_retrieved_per_subq_count`, `chunks_filtered_per_subq_count` (JSON array of ints)

**Task 4.4.2: Update history DB schema (minimal)**

File: `backend/app/core/sqlite_db.py`

Add two new columns (optional, NULL-able):
```sql
ALTER TABLE query_history ADD COLUMN chunks_retrieved_per_subq_count TEXT DEFAULT NULL;
ALTER TABLE query_history ADD COLUMN chunks_filtered_per_subq_count TEXT DEFAULT NULL;
```

These store JSON arrays like `[10, 8]` — one count per sub-question. NULL for pre-Package-4 records.

**Task 4.4.3: Update history Pydantic models**

File: `backend/app/models/history.py`

Add optional fields to `QueryHistoryRecord` and `QueryHistoryDetail`:
```python
chunks_retrieved_per_subq_count: Optional[str] = None  # JSON array string
chunks_filtered_per_subq_count: Optional[str] = None    # JSON array string
```

**Task 4.4.4: Update prompt templates**

File: `backend/app/core/sqlite_db.py` (seed data)

New `generate` template:
```python
"generate": (
    "You must answer each sub-question using ONLY the document chunks provided for it.\n"
    "Do not use any external knowledge.\n"
    "Format your answer as markdown sections — one section per sub-question.\n"
    "Each section should start with \"## Sub-question N: <the question>\"\n"
    "Each section should contain 1-5 bullet points.\n"
    "Cite your sources inline using bracket labels, e.g. [filename, page N].\n"
    "Place the citation at the end of each relevant bullet point.\n\n"
    "{context_sections}\n\n"
    "Answer:"
)
```

`decompose` and `filter` templates remain unchanged (they still use `{question}` placeholder — the orchestrator injects the right value at call time).

**Task 4.4.5: Update `PromptService` to handle new template placeholder**

File: `backend/app/services/prompt_service.py`

- Add `context_sections` as a known placeholder for the `generate` step (optional — `str.replace` already safe with unknown keys)
- The `reset_to_defaults()` method must include the new generate template

**Task 4.4.6: Update history detail API response**

File: `backend/app/routers/history.py`

`GET /api/v1/history/{id}` response now includes `chunks_retrieved_per_subq_count` and `chunks_filtered_per_subq_count` when they are not NULL. Backward-compatible (older records return `null` for these fields).

**Commit**: `"feat: Phase 4.4 history schema, prompt templates, and Pydantic model updates"`

### Sub-Phase 4.5: Frontend — Types & State Management

**Test files to write first:**
- `test_phase4_stream_state.test.tsx` — Tests `QueryStreamState` handles new response shape
- `test_phase4_types.test.ts` — Tests type compatibility

**Task 4.5.1: Update TypeScript types**

File: `frontend/src/types/index.ts`

New types:
```typescript
interface SubQuestionSources {
  sub_question_index: number;
  sub_question_text: string;
  sources: SourceMetadata[];
}

interface QueryStreamCompletedEvent {
  phase: 'completed';
  answer: string;                              // Markdown with ## sections
  sub_question_sources: SubQuestionSources[];  // Grouped sources
}

interface QueryStreamDecomposedEvent {
  phase: 'decomposed';
  extracted_questions: string[];
}

type QueryStreamEvent = 
  | QueryStreamDecomposedEvent
  | { phase: 'retrieving' | 'filtering' | 'generating' }
  | QueryStreamCompletedEvent
  | { phase: 'error'; message: string };
```

**Task 4.5.2: Update `QueryStreamState` and mutation handler**

File: `frontend/src/lib/queries.tsx`

Changes:
```typescript
interface QueryStreamState {
  extractedQuestions: string[] | null;
  answer: string | null;                        // Full markdown
  subQuestionSources: SubQuestionSources[] | null;  // NEW — grouped sources
  phase: 'idle' | 'decomposing' | 'retrieving' | 'filtering' | 'generating' | 'completed' | 'error';
  error: Error | null;
}
```

In the `completed` case:
```typescript
case 'completed':
  setState(prev => ({
    ...prev,
    answer: event.answer,
    subQuestionSources: event.sub_question_sources,
    phase: 'completed',
  }));
  break;
```

**Commit**: `"feat: Phase 4.5 frontend types and state management for per-sub-q responses"`

### Sub-Phase 4.6: Frontend — ResponsePanel & ExtractedQuestionsDisplay

**Test files to write first:**
- `test_phase4_response_panel.test.tsx` — Tests per-sub-question section rendering
- `test_phase4_citation_parser.test.ts` — Tests per-sub-question citation lookup

**Task 4.6.1: Redesign `ResponsePanel` for sub-question sections**

File: `frontend/src/components/ResponsePanel.tsx`

Current: single `ReactMarkdown` block + flat sources grid.

New layout:
```
┌─────────────────────────────────────────────────────┐
│  📋 Response                           [Copy All]   │
├─────────────────────────────────────────────────────┤
│                                                      │
│  ┌─ Sub-question 1: What are time extensions? ─────┐│
│  │                                                    │
│  │  • Time extensions must be notified...             │
│  │    [NEC4 ACC.pdf, page 3]                          │
│  │  • The project manager must acknowledge...         │
│  │    [NEC4 Contract.pdf, page 12]                    │
│  │                                                    │
│  │  Sources (2)                          [Expand ▼]  │
│  │  ┌──────────────────────────────────────────────┐ │
│  │  │ NEC4 ACC.pdf, Page 3  │ NEC4 Contract, p12 │ │
│  │  │ "Clause 61.3 states.." │ "Notice must be..." │ │
│  │  └──────────────────────────────────────────────┘ │
│  └────────────────────────────────────────────────────┘│
│                                                      │
│  ┌─ Sub-question 2: What notice is required? ───────┐│
│  │                                                    │
│  │  • Written notice must be given...                  │
│  │    [NEC4 ACC.pdf, page 7]                           │
│  │                                                    │
│  │  Sources (1)                          [Expand ▼]  │
│  └────────────────────────────────────────────────────┘│
└─────────────────────────────────────────────────────┘
```

Implementation approach:
1. Parse the `answer` markdown into sections using `## Sub-question N:` headers
2. Map each section to its `SubQuestionSources` by matching index
3. Render each section as an accordion/card with:
   - Header: sub-question text (from `SubQuestionSources`)
   - Body: `ReactMarkdown` for bullet points (with inline citation links)
   - Footer: collapsible sources grid (only sources belonging to this sub-question)
4. Keep the existing citation link behavior (clickable `[filename, page N]` → PDF viewer)

**Task 4.6.2: Update `citationParser.ts` for per-sub-question lookup**

File: `frontend/src/utils/citationParser.ts`

Current: `buildCitationLookup(sources: SourceMetadata[])` — returns a single global map.

New: `buildCitationLookup(subQuestionSources: SubQuestionSources[])` — returns a map scoped to the correct sources for each section. The citation `[filename, page N]` match is looked up in the relevant sub-question's source list.

**Task 4.6.3: Update `ExtractedQuestionsDisplay` for anchors**

File: `frontend/src/components/ExtractedQuestionsDisplay.tsx`

Minor enhancement:
- Make each extracted question a clickable anchor that scrolls to its corresponding section in the answer
- Add `id="subq-{index}"` to each section header in `ResponsePanel`
- Keep existing skeleton loading behavior

**Commit**: `"feat: Phase 4.6 frontend per-sub-question response rendering"`

### Sub-Phase 4.7: Testing & Polish

**Test files to write:**
- `test_phase4_integration_query_pipeline.py` — Full integration test simulating per-sub-q pipeline
- `test_phase4_acceptance_query.py` — Acceptance test with real LLM (manual run)
- `test_phase4_e2e_query_flow.test.tsx` — Frontend e2e test with mocked SSE stream

**Task 4.7.1: Backend unit tests**

- Run `pytest backend/app/test/test_phase4_*.py -v` — all must pass
- Verify no regressions in existing Phase 1 and Phase 3 tests
- Update `test_phase1_rag_service.py` for new method signatures
- Update `test_phase1_relevance_filter.py` for per-sub-q behavior
- Rewrite `test_phase3_query_history_integration.py` for new pipeline flow
- Update `test_phase3_prompt_injection.py` for new generate template

**Task 4.7.2: Backend acceptance tests**

- `test_phase4_acceptance_query.py` — real LLM, real ChromaDB
- Verify: answer contains `## Sub-question` headers, sources grouped by sub-question index
- Verify: each sub-question section has 1-5 bullet points
- Verify: inline citations match the correct sub-question's source list

**Task 4.7.3: Frontend tests**

- `test_phase4_response_panel.test.tsx` — renders per-sub-question sections, expandable sources
- `test_phase4_citation_parser.test.ts` — per-sub-question lookup returns correct source
- `test_phase4_e2e_query_flow.test.tsx` — mocks SSE with new event format, verifies section rendering
- Update existing `ResponsePanel.test.tsx` and `citationParser.test.ts` for new API

**Task 4.7.4: Frontend build verification**

- `npm run build` — no TypeScript errors
- `npm test` — all 62 existing tests pass + new Phase 4 tests
- Verify manual flow: ask question → see extracted questions → see per-sub-question answer sections → expand sources per section

**Task 4.7.5: Error handling**

- Empty decomposition: if `decompose()` returns `[]`, fall back to using original question as single sub-question
- Empty retrieval for some sub-questions: that sub-question gets no chunks → section shows "No relevant information found"
- Filter failure (all chunks below threshold): that sub-question gets no answer → graceful empty section
- JSON parse failure in filter: fall back to including all chunks (no filtering) for that sub-question

**Task 4.7.6: Documentation**

- Update `AGENTS.md` with new pipeline architecture section
- Add docstrings to all new methods (`retrieve_per_subquestion`, `filter_per_subquestion`, `generate_response_per_subquestion`)
- Update prompt template documentation in system prompts page

**Commit**: `"feat: Phase 4.7 testing, error handling, and polish for per-sub-q pipeline"`

---

## Phase 4a: Prompt Service Integration for Per-Sub-Q Filter (2026-04-27)

**Root issue**: `filter_per_subquestion()` in `relevance_filter.py` had a hardcoded prompt (`_build_per_subq_prompt()`) — completely bypassing `PromptService`. Users could not edit the per-sub-q filter prompt on the System Prompts page, unlike the flat `filter` step which was already prompt-service-driven.

**Solution**: Broke the per-sub-q filter prompt into **3 composable pieces**, each a separately editable step on the System Prompts page:

| Step Name | Label | Placeholders | Default |
|-----------|-------|-------------|---------|
| `filter_intro` | Step 2.1: Filter Intro (Preamble) | *(none)* | `"Evaluate each chunk for relevance to its associated sub-question only."` |
| `filter_section` | Step 2.2: Filter Section (Per Sub-Q) | `{subq_idx}`, `{subq_question}`, `{chunks}` | `'Sub-question {subq_idx}: "{subq_question}"\n{chunks}'` |
| `filter_outro` | Step 2.3: Filter Outro (Format) | *(none)* | JSON format instructions + example |

The `RelevanceFilter._build_per_subq_prompt()` now composes them at runtime:
```
filter_intro + [filter_section.replace(...) for each sub-q] + filter_outro
```
Falls back to built-in defaults when `PromptService` is unavailable.

### Bugs Fixed

1. **`generate_per_subq` not seeded**: `rag.py` called `get_prompt_template("generate_per_subq")` but this step name was never added to `_VALID_STEPS`, `_SEED_STEPS`, or `_SEED_TEMPLATES` — would crash at runtime with `ValueError`. Now properly seeded with `{context_sections}` placeholder.

2. **`_SEED_GENERATE` placeholder mismatch from Package 4**: The flat `generate_response()` expects `{question}`/`{context}` placeholders, but Package 4 changed the seed template to use `{context_sections}` (intended for per-sub-q generate). Restored flat template; `generate_per_subq` now holds `{context_sections}`.

### Database Backfill Migration

The existing `seed_default_profiles()` only inserted steps for NEWLY created profiles. Added a backfill loop that iterates ALL existing profiles and `INSERT OR IGNORE`s any missing step names. This ensures existing A/B/C profiles pick up `filter_intro`, `filter_section`, `filter_outro`, and `generate_per_subq` on restart.

### System Prompts UI Restructured

The flat `filter` and `generate` steps were removed from the UI (they're unused by the current pipeline). The page now shows 5 steps:

| UI Order | Label | Step Key |
|----------|-------|----------|
| 1 | Step 1: Query Decomposition | `decompose` |
| 2 | Step 2.1: Filter Intro (Preamble) | `filter_intro` |
| 3 | Step 2.2: Filter Section (Per Sub-Q) | `filter_section` |
| 4 | Step 2.3: Filter Outro (Format) | `filter_outro` |
| 5 | Step 3: Generate (Per-Sub-Question) | `generate_per_subq` |

The old `filter` and `generate` templates remain in the DB (for API backward compatibility) but are hidden from the UI.

### Files Changed

| File | Change |
|------|--------|
| `backend/app/core/sqlite_db.py` | 3 new seed templates + `generate_per_subq` seed; backfill migration; restored `_SEED_GENERATE` to `{question}`/`{context}` |
| `backend/app/services/prompt_service.py` | Added 4 step names to `_VALID_STEPS` |
| `backend/app/routers/prompts.py` | Added 4 step names to `_VALID_STEPS` |
| `backend/app/services/relevance_filter.py` | Refactored `_build_per_subq_prompt()` to use PromptService + built-in fallback constants |
| `frontend/src/components/PromptEditor.tsx` | Replaced unused flat steps with 5-step per-sub-q layout (Step 2.1-2.3 + Step 3) |
| `frontend/src/components/PlaceholderDocs.tsx` | Added `{context_sections}`, `{subq_idx}`, `{subq_question}` docs |
| `backend/app/test/conftest.py` | Added 4 new templates to mock |
| `backend/app/test/test_phase3_sqlite_db.py` | Updated counts (9→21 prompts) and placeholder assertions |
| `backend/app/test/test_phase3_prompt_service.py` | Updated step set + placeholder assertions |
| `backend/app/test/test_phase3_prompts_router.py` | Updated step set assertion |
| `backend/app/test/test_phase4_prompt_templates.py` | Updated for split generate/generate_per_subq |
| `frontend/src/test/components/PromptEditor.test.tsx` | Updated to 5 textareas, new labels, new placeholder layout |
| `frontend/src/test/components/PlaceholderDocs.test.tsx` | Updated to 6 placeholders |

### Test Results (Post-Phase 4a)
- **Backend**: 295 passed, 5 skipped (pre-existing)
- **Frontend**: 182 passed, 1 pre-existing failure (unrelated `file-input` e2e)

---

## Sub-Phase Summary

| Sub-Phase | Scope | Backend | Frontend | Tests | Status |
|-----------|-------|---------|----------|-------|--------|
| 4.1 | Per-sub-q retrieval | `rag.py`, `query.py`, format helpers | None | `test_phase4_retrieve_per_subquestion.py`, `test_phase4_query_router_retrieval.py` | ✅ Complete |
| 4.2 | Per-sub-q filtering (1 LLM call) | `relevance_filter.py`, `query.py` | None | `test_phase4_relevance_filter_per_subq.py`, `test_phase4_query_router_filter.py` | ✅ Complete |
| 4.3 | Sub-q-organized response generation | `rag.py`, `query.py`, `models/query.py` | None | `test_phase4_generate_per_subq.py`, `test_phase4_response_format.py` | ✅ Complete |
| 4.4 | History schema, prompts, models | `sqlite_db.py`, `history.py` (router + models), `prompt_service.py` | None | `test_phase4_history_format.py`, `test_phase4_prompt_templates.py` | ✅ Complete |
| 4.5 | Frontend types + state | None | `types/index.ts`, `lib/queries.tsx` | `test_phase4_stream_state.test.tsx`, `test_phase4_types.test.ts` | ✅ Complete |
| 4.6 | Frontend rendering | None | `ResponsePanel.tsx`, `citationParser.ts`, `ExtractedQuestionsDisplay.tsx` | `test_phase4_response_panel.test.tsx`, `test_phase4_citation_parser.test.ts` | ✅ Complete |
| 4.7 | Testing & polish | All affected files | All affected files | Integration + acceptance + e2e tests | ✅ Complete |
| 4a | Prompt service integration for filter_per_subq | `sqlite_db.py`, `prompt_service.py`, `prompts.py`, `relevance_filter.py` | `PromptEditor.tsx`, `PlaceholderDocs.tsx` | Updated 7 test files, 13 total files changed | ✅ Complete |

---

## Implementation Sequence & Dependencies

```
4.1 (Retrieval) ──┐
                  ├──► 4.2 (Filtering) ──► 4.3 (Generate) ──► 4.4 (History/Prompts)
                  │                                                    │
                  │                                                    ▼
                  │                                         4.5 (Frontend Types/State)
                  │                                                    │
                  │                                                    ▼
                  │                                         4.6 (Frontend Rendering)
                  │                                                    │
                  └─────────────────────────────────────────────────────▼
                                                              4.7 (Testing & Polish)
```

- **4.1 → 4.2 sequential**: Filtering needs per-sub-q chunk structure from retrieval
- **4.2 → 4.3 sequential**: Generation needs filtered chunks from filtering stage
- **4.3 → 4.4 sequential**: History recording and prompt templates need final data shapes
- **4.4 → 4.5 parallel**: Backend prompt/history changes don't block frontend type definitions
- **4.5 → 4.6 sequential**: Rendering needs types and state management
- **4.7 blocked by all**: Integration tests need everything wired together

**Parallelization opportunity**: 4.5 (frontend types) could start as soon as 4.3 defines the SSE contract, but it's safer to start after 4.4 confirms the final data shapes.

---

## Affected Files — Complete Inventory

### Backend — New Files
| File | Sub-Phase | Purpose |
|------|-----------|---------|
| `backend/app/test/test_phase4_retrieve_per_subquestion.py` | 4.1 | Unit test: `retrieve_per_subquestion()` |
| `backend/app/test/test_phase4_query_router_retrieval.py` | 4.1 | Unit test: retrieval stage in `_query_stream` |
| `backend/app/test/test_phase4_relevance_filter_per_subq.py` | 4.2 | Unit test: `filter_per_subquestion()` |
| `backend/app/test/test_phase4_query_router_filter.py` | 4.2 | Unit test: filter stage in `_query_stream` |
| `backend/app/test/test_phase4_generate_per_subq.py` | 4.3 | Unit test: `generate_response_per_subquestion()` |
| `backend/app/test/test_phase4_response_format.py` | 4.3 | Unit test: answer format validation |
| `backend/app/test/test_phase4_history_format.py` | 4.4 | Unit test: new XML/JSON history formats |
| `backend/app/test/test_phase4_prompt_templates.py` | 4.4 | Unit test: new generate template |
| `backend/app/test/test_phase4_integration_query_pipeline.py` | 4.7 | Integration test: full per-sub-q pipeline |
| `backend/app/test/acceptance/test_phase4_acceptance_query.py` | 4.7 | Acceptance test: real LLM |

### Backend — Modified Files
| File | Sub-Phase | Changes |
|------|-----------|---------|
| `backend/app/services/rag.py` | 4.1, 4.3 | Add `retrieve_per_subquestion()`, `generate_response_per_subquestion()` |
| `backend/app/services/relevance_filter.py` | 4.2 | Add `filter_per_subquestion()` |
| `backend/app/routers/query.py` | 4.1–4.4 | Refactor `_query_stream()`, add per-sub-q format helpers, update history recording |
| `backend/app/models/query.py` | 4.3 | Add `SubQuestionSources` model, update `QueryResponse` |
| `backend/app/models/history.py` | 4.4 | Add optional per-sub-q count fields |
| `backend/app/core/sqlite_db.py` | 4.4 | Add new columns, update seed generate template |
| `backend/app/services/prompt_service.py` | 4.4 | Update `reset_to_defaults()` generate template |
| `backend/app/routers/history.py` | 4.4 | Include new fields in detail response |
| `backend/app/core/config.py` | 4.1 | (Maybe) Add `retrieval_n_results_per_subq` setting |

### Backend — Tests Needing Update
| File | Sub-Phase | Changes |
|------|-----------|---------|
| `backend/app/test/test_phase1_rag_service.py` | 4.7 | Add tests for new methods; existing tests unaffected |
| `backend/app/test/test_phase1_relevance_filter.py` | 4.7 | Add tests for `filter_per_subquestion()` |
| `backend/app/test/test_phase3_query_history_integration.py` | 4.7 | Rewrite pipeline simulation for per-sub-q flow |
| `backend/app/test/test_phase3_prompt_injection.py` | 4.7 | Add tests for new generate template |
| `backend/app/test/acceptance/test_acceptance_phase1_rag_query.py` | 4.7 | Rewrite — SSE parsing + new response shape |
| `backend/app/test/conftest.py` | 4.7 | Add per-sub-q mock helpers |

### Frontend — New Files
| File | Sub-Phase | Purpose |
|------|-----------|---------|
| `frontend/src/test/components/test_phase4_response_panel.test.tsx` | 4.7 | Component test: per-sub-q sections |
| `frontend/src/test/utils/test_phase4_citation_parser.test.ts` | 4.7 | Unit test: per-sub-q citation lookup |
| `frontend/src/test/e2e/test_phase4_query_flow.test.tsx` | 4.7 | E2E test: mocked SSE with new format |
| `frontend/src/test/lib/test_phase4_stream_state.test.tsx` | 4.5 | State test: new event shapes |
| `frontend/src/test/lib/test_phase4_types.test.ts` | 4.5 | Type test: type compatibility |

### Frontend — Modified Files
| File | Sub-Phase | Changes |
|------|-----------|---------|
| `frontend/src/types/index.ts` | 4.5 | Add `SubQuestionSources`, update `QueryStreamEvent` |
| `frontend/src/lib/queries.tsx` | 4.5 | Update `QueryStreamState`, `completed` event handler |
| `frontend/src/components/ResponsePanel.tsx` | 4.6 | Redesign — per-sub-question sections with grouped sources |
| `frontend/src/utils/citationParser.ts` | 4.6 | Update `buildCitationLookup()` for per-sub-q |
| `frontend/src/components/ExtractedQuestionsDisplay.tsx` | 4.6 | Add anchor links to answer sections |
| `frontend/src/pages/LTTPage.tsx` | 4.6 | Pass new props to children |

---

## Risk Register

| Risk | Likelihood | Impact | Mitigation |
|------|-----------|--------|------------|
| LLM struggles with per-sub-q filtering prompt format | Medium | High — all chunks dropped | Use strong prompt constraints, validate JSON, fall back to including all chunks on parse failure |
| LLM generates answer not matching `## Sub-question N:` format | Medium | Medium — frontend can't parse sections | Fall back to rendering as single block if parsing fails. Prompt engineering tuned for format compliance |
| Same chunk retrieved by multiple sub-questions → duplicated in context | High | Low — slightly larger prompt but acceptable | Accept duplicates. ChromaDB naturally returns same doc if relevant to multiple queries. Each sub-q's evaluation is independent |
| Per-sub-q retrieval = more ChromaDB queries = slower | Medium | Medium — N × retrieval latency | ChromaDB retrieval is fast (~10-50ms). 5 sub-questions × 10ms = 50ms overhead. Acceptable trade-off for better relevance. |
| History DB migration fails for existing records | Low | Low — new columns are NULL-able | `ALTER TABLE ADD COLUMN ... DEFAULT NULL` is safe. Existing records work as before — `chunks_retrieved`/`chunks_filtered` still have flat XML. |
| Frontend rendering breaks on older history records | Low | Low — answer format differs | `ResponsePanel` renders per-sub-q sections only when `subQuestionSources` is non-null. Older history records show flat answer as before. |
| Prompt template migration breaks user-customized prompts | Medium | Medium — users lose their generate template | Warn in docs. The `generate` template changes fundamentally (single `{context}` → `{context_sections}`). Users must re-customize. |

---

## Acceptance Criteria

### Backend
- [x] `POST /api/v1/query` retrieves chunks per sub-question (verified by history XML showing `<sub_q>` wrappers)
- [x] Filtering uses single LLM call evaluating chunks against their originating sub-question (verified by filter prompt)
- [x] Response answer is organized by sub-question with `## Sub-question N:` headers
- [x] `sub_question_sources` in SSE `completed` event is grouped by sub-question index
- [x] History records include new grouped XML formats for `chunks_retrieved` and `chunks_filtered`
- [x] History records include grouped `sources` JSON (list of lists)
- [x] History records include per-sub-q chunk counts
- [x] New `generate` prompt template uses `{context_sections}` placeholder
- [x] Prompt service `reset_to_defaults()` includes new generate template
- [x] Existing `decompose`, `filter` (old), `generate_response` (old) methods are unchanged
- [x] All Phase 1, Phase 3, and new Phase 4 unit tests pass (312 passed, 4 skipped)
- [x] All acceptance tests pass with real LLM (manual run)

### Frontend
- [x] `QueryStreamState` includes `subQuestionSources` field
- [x] `ResponsePanel` renders per-sub-question sections with expandable source grids
- [x] Each section's sources are scoped to that sub-question (no cross-contamination)
- [x] Inline citations `[filename, page N]` link to the correct PDF viewer page
- [x] `ExtractedQuestionsDisplay` shows clickable anchors to answer sections
- [x] Copy button copies all answer text including section headers
- [x] Loading states: skeleton per section during generation
- [x] Empty state: "No relevant information found" per sub-question (not entire response)
- [x] All 62+ existing frontend tests still pass (183 passed)
- [x] All new Phase 4 frontend tests pass
- [x] `npm run build` succeeds with zero TypeScript errors
- [x] Manual verification: full query flow works end-to-end

---

## New Dependencies

None. All changes use existing libraries (FastAPI, ChromaDB, OpenAI SDK, React, ReactMarkdown, TanStack Query).

---

## Decisions (All Confirmed)

| # | Topic | Decision |
|---|-------|----------|
| 1 | Single vs multiple filter LLM calls | **Single call** — user explicitly requested this |
| 2 | Filter prompt design | Group chunks by sub-question in one prompt. JSON response maps sub-q indices to score arrays |
| 3 | Answer format | Markdown with `## Sub-question N: <question>` headers |
| 4 | Sources grouping | `sub_question_sources: [{index, text, sources}, ...]` in SSE + frontend |
| 5 | History XML format | Add `<sub_q idx="N" question="...">` wrappers around chunk groups |
| 6 | History DB migration | Add 2 new NULL-able columns. No data migration needed. |
| 7 | Backward compatibility | Preserve old `retrieve()`, `filter()`, `generate_response()` methods. New methods are additive. |
| 8 | Deduplication | None. Same chunk may appear in multiple sub-questions. Each sub-q evaluates independently. |
| 9 | Error handling | Per-sub-question graceful degradation. Filter failure → include all chunks for that sub-q. Generate failure → "Unable to generate answer for this sub-question." |
| 10 | Frontend rendering engine | Keep `ReactMarkdown`. Parse sections client-side by splitting on `## Sub-question N:` headers. |

---

## Open Questions

None — all resolved.

| # | Question | Resolution |
|---|----------|------------|
| 1 | Progressive SSE events? | **Yes** — emit `generating_subquestion` as each sub-question's answer is generated. Frontend renders sections progressively. |
| 2 | `retrieval_n_results` per sub-question or global? | **Global** — same value for all sub-questions. Simpler config, one setting. |
| 3 | Fallback when decomposition returns 0 sub-questions? | **Fall back to original question** — treat as single sub-question. Pipeline runs as 1-sub-q case (retrieval via original question, no filtering needed for single sub-q, flat answer). |

---

## Test Plan Summary

### Backend (New Tests)

| File | Tests | Coverage |
|------|-------|----------|
| `test_phase4_retrieve_per_subquestion.py` | ~6 | Per-sub-q retrieval, empty input, single sub-q, dedup behavior |
| `test_phase4_query_router_retrieval.py` | ~4 | SSE events during retrieval, chunk XML format |
| `test_phase4_relevance_filter_per_subq.py` | ~6 | Per-sub-q filtering, JSON response parsing, threshold behavior |
| `test_phase4_query_router_filter.py` | ~4 | SSE events during filtering, filtered XML format |
| `test_phase4_generate_per_subq.py` | ~5 | Per-sub-q generate, prompt construction, answer format |
| `test_phase4_response_format.py` | ~4 | Answer has `##` headers, citations in correct sections |
| `test_phase4_history_format.py` | ~5 | New XML/JSON formats, per-sub-q counts |
| `test_phase4_prompt_templates.py` | ~3 | New generate template, `{context_sections}` placeholder |
| `test_phase4_integration_query_pipeline.py` | ~5 | Full pipeline simulation |
| `test_phase4_acceptance_query.py` | ~3 | Real LLM end-to-end (manual) |

### Frontend (New Tests)

| File | Tests | Coverage |
|------|-------|----------|
| `test_phase4_stream_state.test.tsx` | ~4 | State updates for new event shapes |
| `test_phase4_types.test.ts` | ~2 | Type compatibility checks |
| `test_phase4_response_panel.test.tsx` | ~6 | Section rendering, source grouping, copy, loading |
| `test_phase4_citation_parser.test.ts` | ~4 | Per-sub-q lookup, cross-section isolation |
| `test_phase4_e2e_query_flow.test.tsx` | ~3 | Full SSE flow with mocked stream |