PRODUCTION SCENARIO
Sundridge Legal retrieves statutes from a vector store built with the default chunking settings. Clauses that straddle a chunk boundary come back missing their second half.
Which chunking settings give retrieval the most cross-boundary context?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: max_chunk_size_tokens 1,600 with chunk_overlap_tokens 800, twice the default chunk
The defaults are max_chunk_size_tokens 800 with chunk_overlap_tokens 400, already the ceiling because chunk_overlap_tokens must not exceed max_chunk_size_tokens / 2. More cross-boundary context therefore means raising both together, inside the 100-to-4,096-token range. Overlap of 600 against an 800-token chunk is rejected outright, keeping overlap at 400 while doubling the chunk lowers the ratio, and halving both produces more boundaries than the default did.
Official Source: https://developers.openai.com/api/docs/guides/retrieval ↗