PRODUCTION SCENARIO
A pipeline asks an Amazon Bedrock model to return a JSON object and then parses the response. The model produces valid JSON but appends an explanatory paragraph after it, and the parser fails.
Which inference parameter will resolve this?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: Stop sequences
A stop sequence specifies characters that stop the model from generating further tokens, so generation ends as soon as the JSON closes and nothing follows it. Response length only caps how long the output may run, which gives the trailing paragraph room rather than removing it. Temperature and Top P change which tokens the model selects, not where generation ends.