PRODUCTION SCENARIO
Users upload contract PDFs of up to 400 MB at unpredictable times. Each inference run takes about eight minutes, and users accept receiving results by notification rather than in the upload response. The team wants no charge while no uploads are arriving.
Which SageMaker inference option meets these requirements?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: An asynchronous endpoint, which queues each request and scales to zero when idle
Asynchronous inference queues each request, accepts payloads up to 1 GB and processing up to one hour, and scales the instance count to zero when the queue drains. Serverless inference also scales to zero but targets short requests between traffic spurts, not eight-minute jobs. A one-instance floor never reaches zero cost, and a batch job per upload starts a cluster per file.