Chunking Strategy Selection and Reranking Plan
Choose between fixed, semantic and model-based chunking for a corpus and pair it with an overfetch-and-rerank retrieval plan.
A chunking decision with trade-off reasoning, the chosen parameters, and an overfetch-plus-rerank retrieval plan with latency accounting and an A/B test design.
Role
You are optimising the retrieval layer of an existing RAG system.
Task
Given {{corpus_characteristics}}, evaluate {{candidate_chunk_methods}} across three axes: semantic coherence of the resulting units, compute cost at ingestion, and retrieval precision. Recommend one method and give its parameters, using {{fixed_chunk_params}} if fixed-size is chosen or {{semantic_threshold}} if similarity-based splitting is chosen. Then design retrieval: overfetch {{overfetch_k}} candidates, rerank with {{reranker_choice}} scoring each query-candidate pair, and pass {{final_k}} to generation. Account for the added latency against {{latency_budget}} and specify an A/B test that isolates chunking changes from reranking changes.
Context
Chunking and reranking both change which evidence reaches the model, so evaluating them together without isolation makes regressions untraceable.
Inputs
- {{corpus_characteristics}}
- {{candidate_chunk_methods}}
- {{fixed_chunk_params}}
- {{semantic_threshold}}
- {{overfetch_k}}
- {{final_k}}
- {{reranker_choice}}
- {{latency_budget}}
Constraints
- State the compute cost of the chosen chunking method explicitly
- Overfetch count must exceed the final count
- Reranker latency must be accounted against the budget
- The A/B design must vary one factor at a time
Output Format
Markdown: comparison table, recommendation with parameters, retrieval plan, latency table, A/B test design.
Quality Criteria
- Recommendation follows from the corpus characteristics
- Parameters concrete and justified
- Latency accounting arithmetic shown
- Experiment isolates each change