PRODUCTION SCENARIO
A pipeline asks an Amazon Bedrock model to return a JSON object and then parses the response. The model produces valid JSON but appends an explanatory paragraph after it, and the parser fails.
Which inference parameter will resolve this?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: A stop sequence
A stop sequence specifies characters that stop the model from generating further tokens, so generation ends as soon as the JSON is closed and nothing follows it. Top-p and temperature shape which tokens are selected rather than where output ends, and raising the length limit only gives the extra paragraph more room.