PRODUCTION SCENARIO
A research archive needs 18,000 documents classified by tomorrow morning. Each classification is independent of the others and needs no tool call, nobody is waiting on a screen, and cost is what the archive is optimizing. No rate limit or downstream consumer requires the work to be paced. Past runs have had a few hundred documents fail on oversized inputs, and those have to be identified and resent without redoing the rest.
Which approach fits this workload best?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Submit them as one Message Batches request, matched by custom_id
Large, independent, latency-tolerant work is what the Message Batches API is for, each request in a batch is processed independently, and custom_id is what matches a result back to its document, so the few hundred failures can be resent on their own. Parallel synchronous calls are right when the deadline is an hour rather than tomorrow, and they pay full price for latency nobody needs. One long conversation is right when a label depends on documents already seen, and these are independent, so shared context inflates every request. Serial batches are right when a rate limit or a consumer requires pacing, and nothing here does.