PRODUCTION SCENARIO
Your due-diligence assistant sends three contracts totaling 60,000 tokens in each request, and the answers have to compare terms across all three, so the documents cannot be separated. The prompt opens with the question and detailed instructions and pastes the contracts underneath, untagged. Where the assistant does find a clause it reasons about it correctly; the failure is that it misses clauses that are plainly present.
Which prompt restructuring should you try first?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Put the contracts above the question, each in its own tagged document block
Above roughly 20,000 tokens, long documents belong near the top of the prompt, each wrapped in a tagged block with its source metadata, with the query and instructions last; putting the query at the end has improved response quality by up to 30 percent in testing, most of all on multidocument inputs. Splitting into three requests is right when each document can be answered alone, and this comparison spans all three. Moving instructions to the system prompt is right when they are being read as part of the documents, and it leaves the documents below the query. Raising effort is right when the model finds the clauses and reasons badly, and it reasons correctly on what it finds.