Chunking divides a document into units that can be retrieved. The useful size depends on the document and question. A short product description and a policy with several exceptions should not automatically use the same boundaries.
A practical scenario
A travel policy may allow a booking only when a later paragraph lists approval conditions. Splitting the allowance from its exceptions creates a misleading retrieval result. Preserve section headings and meaningful paragraph groups. For tables, carry column names and units with each relevant row.
Design the first version
Create a small set of questions with known source passages. Compare at least two chunking strategies while keeping the model and retrieval settings fixed. Record whether each retrieved chunk contains sufficient context to answer without borrowing facts from neighboring material that was not retrieved.
What to test and measure
Look for redundant chunks, lost definitions, and citations that point to an entire document rather than the actual evidence. Overlap can help at boundaries but can also fill the result set with near duplicates. Choose settings based on measured evidence coverage and answer quality, not a supposedly universal token count.
Questions to resolve before commissioning
- Which source system owns the facts used in this workflow?
- Who reviews exceptions and corrects inaccurate output?
- What baseline and acceptance criteria will determine whether the pilot is useful?
- What should the user do when a source, tool, or device is unavailable?
Explore the implementation
This is a planning guide, not a report of measured client results. Examples are illustrative. Explore the related SyntaxLab demo to discuss the interaction, then use your own records and acceptance criteria for a production pilot. Discuss a scoped project or review our AI automation services.
Further reading
Read Microsoft guidance on evaluating RAG answers for technical background. Continue with A RAG Evaluation Scorecard for Business Knowledge Assistants.