PRODUCTION SCENARIO
In the Foundry playground, a GPT model continues the first line of a well-known novel word for word, but invents details when the same deployment is asked about the company's internal expense policy.
What accounts for the difference?
Answering here is anonymous. Nothing is saved unless you sign in.
Show answer and explanation
Answer: The novel is in the training data and the expense policy is not
The model produces 'the next series of words that are most likely to follow from the previous text', responding with 'what it determines is most likely (given its training data and training targets)'. A famous opening continues accurately because it is in that data; a private policy the model never saw produces a plausible guess. Temperature is a per-request sampling parameter the playground does not vary by topic, a short question does not exhaust the context window, and the docs state there is no separate factual code path.