PRODUCTION SCENARIO
A media archive needs alt-text and keyword tags generated once for 4 million photographs, with results due in three days. No user is waiting on any individual response.
Which processing option fits this job?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Batch inference
Batch inference exists for exactly this: asynchronous, high-throughput work priced at a 50% discount to real-time inference, with most jobs finishing within 24 hours of starting. Nothing here is waiting on a single response, so there is no reason to pay for immediacy. Provisioned Throughput reserves capacity to guarantee latency on real-time serving - a lane nobody in this scenario needs. Context caching discounts repeated input tokens, which is tempting because the alt-text instruction repeats, but four million distinct photographs share almost nothing worth caching and caching does nothing for throughput. Real-time inference does the job at twice the price for a deadline three days out.