Add chunk_pages() to TokenChunkingStrategy: one chunk per page with 200-token overlap from adjacent pages. Uses original page text for main content, decoded tokens for overlap. Never splits a page regardless of size. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai> |
||
|---|---|---|
| .. | ||
| core | ||
| models | ||
| routers | ||
| services | ||
| test | ||
| utils | ||
| __init__.py | ||
| main.py | ||