From 63e4c1a385e1e53d61c4e1f5bc7272b892e4b0f9 Mon Sep 17 00:00:00 2001 From: Woody Date: Mon, 4 May 2026 17:22:38 +0800 Subject: [PATCH] docs: add plan for configurable SubQuestions format --- .plans/package6_subquestions_config_plan.md | 178 ++++++++++++++++++++ 1 file changed, 178 insertions(+) create mode 100644 .plans/package6_subquestions_config_plan.md diff --git a/.plans/package6_subquestions_config_plan.md b/.plans/package6_subquestions_config_plan.md new file mode 100644 index 0000000..e4909c6 --- /dev/null +++ b/.plans/package6_subquestions_config_plan.md @@ -0,0 +1,178 @@ +# Plan: Configurable SubQuestions via System Prompt Page + +**Source**: User request (2026-05-04) +**Scope**: Make SubQuestions `description` and `max_length` configurable from the system prompt page. Rename "Step 1: Query Decomposition" → "Step 1.1: Query Decomposition". Add new "Step 1.2: Query Decomposition Format" step storing JSON config. +**Status**: Draft +**Depends on**: Package 6 (LLMClientDP) — already implemented + +--- + +## Objective + +Allow users to configure `SubQuestions.description` (the format instruction for how sub-questions should be structured) and `SubQuestions.max_length` (1–5) from the system prompt configuration page, rather than hardcoding them in `backend/app/models/decompose.py`. + +## Design + +### Step Split + +The current "Step 1: Query Decomposition" (`step_name="decompose"`) stores the prompt template (e.g. "Given this question... break it down..."). This becomes **Step 1.1**. + +A new step **Step 1.2: Query Decomposition Format** (`step_name="decompose_format"`) stores a JSON configuration string: + +```json +{"description": "", "max_length": 3} +``` + +The `prompt_template` column in `system_prompts` is reused — it's a `TEXT NOT NULL` field that can hold any string, including JSON. + +### Dynamic Pydantic Model + +At runtime, `QueryDecomposer.decompose()` reads `decompose_format` from `PromptService`, parses the JSON, and creates a Pydantic model dynamically using `pydantic.create_model()`: + +```python +from pydantic import create_model, Field + +def create_subquestions_model(description: str, max_length: int) -> type[BaseModel]: + return create_model( + "SubQuestions", + questions=(list[str], Field(description=description, min_length=1, max_length=max_length)), + ) +``` + +The static `SubQuestions` model in `decompose.py` is kept as a fallback and for the `_BUILTIN_DECOMPOSE_TEMPLATE` path. + +### Data Flow + +``` +system_prompts DB +├── step_name="decompose" → "Given this question: '{question}'..." (Step 1.1) +└── step_name="decompose_format" → '{"description": "...", "max_length": 3}' (Step 1.2) + +QueryDecomposer.decompose(question): + template = prompt_service.get_prompt_template("decompose") # Step 1.1 + prompt = template.replace("{question}", question) + + format_json = prompt_service.get_prompt_template("decompose_format") # Step 1.2 + format_config = json.loads(format_json) + DynamicSubQuestions = create_subquestions_model( + description=format_config["description"], + max_length=format_config["max_length"], + ) + + result = await llm_client.complete_structured( + prompt=prompt, + pydantic_model=DynamicSubQuestions, + ) +``` + +--- + +## Files to Modify + +### 1. `backend/app/core/sqlite_db.py` + +| Change | Detail | +|--------|--------| +| Add `_SEED_DECOMPOSE_FORMAT` | Default JSON string with current description + max_length=3 | +| Update `_SEED_STEPS` | Add `"decompose_format"` | +| Update `_SEED_TEMPLATES` | Add `"decompose_format": _SEED_DECOMPOSE_FORMAT` | +| Update `_SEED_DECOMPOSE` | Change "2-5" → "1-3" (match current defaults) | + +### 2. `backend/app/services/prompt_service.py` + +| Change | Detail | +|--------|--------| +| `_VALID_STEPS` | Add `"decompose_format"` | + +### 3. `backend/app/routers/prompts.py` + +| Change | Detail | +|--------|--------| +| `_VALID_STEPS` | Add `"decompose_format"` | + +### 4. `backend/app/models/decompose.py` + +| Change | Detail | +|--------|--------| +| Add `create_subquestions_model()` | Takes `description: str`, `max_length: int`, returns dynamic model | +| Keep static `SubQuestions` | As fallback with current defaults | +| Add `parse_decompose_format()` | Parses JSON string into `(description, max_length)` tuple with validation | + +### 5. `backend/app/services/query_decomposer.py` + +| Change | Detail | +|--------|--------| +| Read `decompose_format` | From `prompt_service.get_prompt_template("decompose_format")` | +| Create dynamic model | Call `create_subquestions_model()` with parsed config | +| Fallback | If `decompose_format` unavailable or parse fails, use static `SubQuestions` | +| `_BUILTIN_DECOMPOSE_TEMPLATE` | Update "2-5" → "1-3" | + +### 6. `frontend/src/components/PromptEditor.tsx` + +| Change | Detail | +|--------|--------| +| `STEPS[0]` | `label`: "Step 1: Query Decomposition" → "Step 1.1: Query Decomposition" | +| Add new entry | After `decompose`, insert `{ key: 'decompose_format', label: 'Step 1.2: Query Decomposition Format', placeholders: [], type: 'format_config' }` | +| `VALID_PLACEHOLDERS` | Add `decompose_format: []` | +| Render `decompose_format` differently | Two fields: **textarea** for `description` + **number input** for `max_length` (min=1, max=5). Combined to/from JSON `{"description": "...", "max_length": N}` on read/write. Other steps render as single textarea (unchanged). | +| No auto-sync | Step 1.1 and Step 1.2 are independent — user edits both manually. No `{max_length}` placeholder in Step 1.1 template. | + +### 7. `frontend/src/test/components/PromptEditor.test.tsx` + +| Change | Detail | +|--------|--------| +| Update assertion | "Step 1: Query Decomposition" → "Step 1.1: Query Decomposition" | + +### 8. Tests (new/updated) + +| File | Type | Coverage | +|------|------|----------| +| `test_phase6_decompose_format.py` | Integration | Dynamic model creation, DB config read, fallback to static model | +| Update `test_phase1_query.py` | Integration | Mock `decompose_format` step in mock prompt service | +| Update `test_phase3_query_history_integration.py` | Integration | Mock `decompose_format` step | +| Update `test_phase4_integration_query_pipeline.py` | Integration | Mock `decompose_format` step | + +--- + +## DB Migration + +No schema changes needed. The `decompose_format` step uses existing `system_prompts` table columns. The existing `seed_default_profiles()` backfill logic (`INSERT OR IGNORE`) will automatically add the new step to existing profiles on next startup. + +--- + +## Acceptance Criteria + +- [ ] User can configure `description` and `max_length` on the system prompt page under "Step 1.2: Query Decomposition Format" +- [ ] Changing the format JSON updates sub-question output format in real decompose calls +- [ ] `max_length` correctly limits the number of sub-questions returned +- [ ] Existing profiles (A/B/C) get the new `decompose_format` step with defaults on restart +- [ ] Import/export includes `decompose_format` +- [ ] Fallback to static `SubQuestions` when `decompose_format` is missing or invalid +- [ ] All existing tests pass +- [ ] Step 1 label renamed to "Step 1.1" in frontend + +--- + +## Implementation Order + +``` +Task A: Backend model + service (decompose.py, query_decomposer.py) + ↓ +Task B: DB seed + valid_steps (sqlite_db.py, prompt_service.py, prompts.py) + ↓ +Task C: Frontend (PromptEditor.tsx, tests) + ↓ +Task D: Update all integration tests +``` + +Tasks B and C can run in parallel after Task A. + +--- + +## Risk: Existing DB Profiles Missing `decompose_format` + +Existing SQLite databases won't have the `decompose_format` row. Mitigation: +1. The `seed_default_profiles()` backfill loop adds missing steps on startup +2. `PromptService.get_prompt_template("decompose_format")` may raise `RuntimeError` if missing +3. `QueryDecomposer` catches this and falls back to static `SubQuestions` +4. A manual profile save from the frontend will write the new step