Serving an on-premises fine-tuned Llama through Bedrock
MLA-C02
Deployment and Orchestration of ML and AI Workflows
Medium
Question 5 of 30
A company fine-tuned a Llama 3.1 8B model on premises with the Hugging Face Transformers library. The company wants applications to call the model through the Amazon Bedrock InvokeModel API with on-demand throughput, so that the company does not manage any inference instances. The model weights are 16 GB.
Which solution will meet these requirements?
Show answer and explanation
DUpload the safetensors, config.json, and tokenizer files to S3 and run a Custom Model Import job
Amazon Bedrock Custom Model Import meets the requirements. It imports models customized in other environments, supports the Llama 3.1 architecture, and reads Hugging Face format files (.safetensors weights, config.json and the tokenizer files) from Amazon S3. The imported model runs with on-demand throughput and is called with InvokeModel. A SageMaker AI real-time endpoint works, but it runs on an instance the company manages and is not called through Bedrock InvokeModel. A fine-tuning job takes example records as training data, not another model's weights. Bedrock does not serve models from a container image that a customer selects in the console.
Source: docs.aws.amazon.com/
Anonymous, no account needed. Your tally stays in this browser.
Practice by topic: Amazon Bedrock questions