179 lines
7.2 KiB
Markdown
179 lines
7.2 KiB
Markdown
# Plan: Configurable SubQuestions via System Prompt Page
|
||
|
||
**Source**: User request (2026-05-04)
|
||
**Scope**: Make SubQuestions `description` and `max_length` configurable from the system prompt page. Rename "Step 1: Query Decomposition" → "Step 1.1: Query Decomposition". Add new "Step 1.2: Query Decomposition Format" step storing JSON config.
|
||
**Status**: Draft
|
||
**Depends on**: Package 6 (LLMClientDP) — already implemented
|
||
|
||
---
|
||
|
||
## Objective
|
||
|
||
Allow users to configure `SubQuestions.description` (the format instruction for how sub-questions should be structured) and `SubQuestions.max_length` (1–5) from the system prompt configuration page, rather than hardcoding them in `backend/app/models/decompose.py`.
|
||
|
||
## Design
|
||
|
||
### Step Split
|
||
|
||
The current "Step 1: Query Decomposition" (`step_name="decompose"`) stores the prompt template (e.g. "Given this question... break it down..."). This becomes **Step 1.1**.
|
||
|
||
A new step **Step 1.2: Query Decomposition Format** (`step_name="decompose_format"`) stores a JSON configuration string:
|
||
|
||
```json
|
||
{"description": "<the field description text>", "max_length": 3}
|
||
```
|
||
|
||
The `prompt_template` column in `system_prompts` is reused — it's a `TEXT NOT NULL` field that can hold any string, including JSON.
|
||
|
||
### Dynamic Pydantic Model
|
||
|
||
At runtime, `QueryDecomposer.decompose()` reads `decompose_format` from `PromptService`, parses the JSON, and creates a Pydantic model dynamically using `pydantic.create_model()`:
|
||
|
||
```python
|
||
from pydantic import create_model, Field
|
||
|
||
def create_subquestions_model(description: str, max_length: int) -> type[BaseModel]:
|
||
return create_model(
|
||
"SubQuestions",
|
||
questions=(list[str], Field(description=description, min_length=1, max_length=max_length)),
|
||
)
|
||
```
|
||
|
||
The static `SubQuestions` model in `decompose.py` is kept as a fallback and for the `_BUILTIN_DECOMPOSE_TEMPLATE` path.
|
||
|
||
### Data Flow
|
||
|
||
```
|
||
system_prompts DB
|
||
├── step_name="decompose" → "Given this question: '{question}'..." (Step 1.1)
|
||
└── step_name="decompose_format" → '{"description": "...", "max_length": 3}' (Step 1.2)
|
||
|
||
QueryDecomposer.decompose(question):
|
||
template = prompt_service.get_prompt_template("decompose") # Step 1.1
|
||
prompt = template.replace("{question}", question)
|
||
|
||
format_json = prompt_service.get_prompt_template("decompose_format") # Step 1.2
|
||
format_config = json.loads(format_json)
|
||
DynamicSubQuestions = create_subquestions_model(
|
||
description=format_config["description"],
|
||
max_length=format_config["max_length"],
|
||
)
|
||
|
||
result = await llm_client.complete_structured(
|
||
prompt=prompt,
|
||
pydantic_model=DynamicSubQuestions,
|
||
)
|
||
```
|
||
|
||
---
|
||
|
||
## Files to Modify
|
||
|
||
### 1. `backend/app/core/sqlite_db.py`
|
||
|
||
| Change | Detail |
|
||
|--------|--------|
|
||
| Add `_SEED_DECOMPOSE_FORMAT` | Default JSON string with current description + max_length=3 |
|
||
| Update `_SEED_STEPS` | Add `"decompose_format"` |
|
||
| Update `_SEED_TEMPLATES` | Add `"decompose_format": _SEED_DECOMPOSE_FORMAT` |
|
||
| Update `_SEED_DECOMPOSE` | Change "2-5" → "1-3" (match current defaults) |
|
||
|
||
### 2. `backend/app/services/prompt_service.py`
|
||
|
||
| Change | Detail |
|
||
|--------|--------|
|
||
| `_VALID_STEPS` | Add `"decompose_format"` |
|
||
|
||
### 3. `backend/app/routers/prompts.py`
|
||
|
||
| Change | Detail |
|
||
|--------|--------|
|
||
| `_VALID_STEPS` | Add `"decompose_format"` |
|
||
|
||
### 4. `backend/app/models/decompose.py`
|
||
|
||
| Change | Detail |
|
||
|--------|--------|
|
||
| Add `create_subquestions_model()` | Takes `description: str`, `max_length: int`, returns dynamic model |
|
||
| Keep static `SubQuestions` | As fallback with current defaults |
|
||
| Add `parse_decompose_format()` | Parses JSON string into `(description, max_length)` tuple with validation |
|
||
|
||
### 5. `backend/app/services/query_decomposer.py`
|
||
|
||
| Change | Detail |
|
||
|--------|--------|
|
||
| Read `decompose_format` | From `prompt_service.get_prompt_template("decompose_format")` |
|
||
| Create dynamic model | Call `create_subquestions_model()` with parsed config |
|
||
| Fallback | If `decompose_format` unavailable or parse fails, use static `SubQuestions` |
|
||
| `_BUILTIN_DECOMPOSE_TEMPLATE` | Update "2-5" → "1-3" |
|
||
|
||
### 6. `frontend/src/components/PromptEditor.tsx`
|
||
|
||
| Change | Detail |
|
||
|--------|--------|
|
||
| `STEPS[0]` | `label`: "Step 1: Query Decomposition" → "Step 1.1: Query Decomposition" |
|
||
| Add new entry | After `decompose`, insert `{ key: 'decompose_format', label: 'Step 1.2: Query Decomposition Format', placeholders: [], type: 'format_config' }` |
|
||
| `VALID_PLACEHOLDERS` | Add `decompose_format: []` |
|
||
| Render `decompose_format` differently | Two fields: **textarea** for `description` + **number input** for `max_length` (min=1, max=5). Combined to/from JSON `{"description": "...", "max_length": N}` on read/write. Other steps render as single textarea (unchanged). |
|
||
| No auto-sync | Step 1.1 and Step 1.2 are independent — user edits both manually. No `{max_length}` placeholder in Step 1.1 template. |
|
||
|
||
### 7. `frontend/src/test/components/PromptEditor.test.tsx`
|
||
|
||
| Change | Detail |
|
||
|--------|--------|
|
||
| Update assertion | "Step 1: Query Decomposition" → "Step 1.1: Query Decomposition" |
|
||
|
||
### 8. Tests (new/updated)
|
||
|
||
| File | Type | Coverage |
|
||
|------|------|----------|
|
||
| `test_phase6_decompose_format.py` | Integration | Dynamic model creation, DB config read, fallback to static model |
|
||
| Update `test_phase1_query.py` | Integration | Mock `decompose_format` step in mock prompt service |
|
||
| Update `test_phase3_query_history_integration.py` | Integration | Mock `decompose_format` step |
|
||
| Update `test_phase4_integration_query_pipeline.py` | Integration | Mock `decompose_format` step |
|
||
|
||
---
|
||
|
||
## DB Migration
|
||
|
||
No schema changes needed. The `decompose_format` step uses existing `system_prompts` table columns. The existing `seed_default_profiles()` backfill logic (`INSERT OR IGNORE`) will automatically add the new step to existing profiles on next startup.
|
||
|
||
---
|
||
|
||
## Acceptance Criteria
|
||
|
||
- [ ] User can configure `description` and `max_length` on the system prompt page under "Step 1.2: Query Decomposition Format"
|
||
- [ ] Changing the format JSON updates sub-question output format in real decompose calls
|
||
- [ ] `max_length` correctly limits the number of sub-questions returned
|
||
- [ ] Existing profiles (A/B/C) get the new `decompose_format` step with defaults on restart
|
||
- [ ] Import/export includes `decompose_format`
|
||
- [ ] Fallback to static `SubQuestions` when `decompose_format` is missing or invalid
|
||
- [ ] All existing tests pass
|
||
- [ ] Step 1 label renamed to "Step 1.1" in frontend
|
||
|
||
---
|
||
|
||
## Implementation Order
|
||
|
||
```
|
||
Task A: Backend model + service (decompose.py, query_decomposer.py)
|
||
↓
|
||
Task B: DB seed + valid_steps (sqlite_db.py, prompt_service.py, prompts.py)
|
||
↓
|
||
Task C: Frontend (PromptEditor.tsx, tests)
|
||
↓
|
||
Task D: Update all integration tests
|
||
```
|
||
|
||
Tasks B and C can run in parallel after Task A.
|
||
|
||
---
|
||
|
||
## Risk: Existing DB Profiles Missing `decompose_format`
|
||
|
||
Existing SQLite databases won't have the `decompose_format` row. Mitigation:
|
||
1. The `seed_default_profiles()` backfill loop adds missing steps on startup
|
||
2. `PromptService.get_prompt_template("decompose_format")` may raise `RuntimeError` if missing
|
||
3. `QueryDecomposer` catches this and falls back to static `SubQuestions`
|
||
4. A manual profile save from the frontend will write the new step
|