Would you join a company where 85% of employees stay longer than two years? Or one where 15% of employees leave within two years?
Most of us like the first option better, even though the numbers are identical. Reality can be framed by description, and that framing affects our decision-making in very real ways. Psychologists call this the framing effect.
Decision Frame and Framing Effect in Humans
First described by Tversky and Kahneman in 1981, a decision frame is the acts and outcomes of a particular choice. The framing effect occurs when equivalent choices are presented in different ways, dramatically altering the majority response.
In the original study, participants were split into two groups. One group saw a gain frame (lives saved) and the other saw a loss frame (lives lost). Each frame contained the exact same mathematical outcomes, with only the descriptions changed.
In this setup, scenarios A and C are identical, and B and D are identical. However, in the gain frame participants overwhelmingly chose the certain outcome (A) while in the loss frame they overwhelmingly chose the risky option (D). That flip is the framing effect in action.
While the framing effect was originally demonstrated in humans, recent observations indicate that large language models (LLMs) — the technology behind systems like ChatGPT and Perplexity — are also susceptible to the framing.
How LLMs Work and Why They’re Frame-Sensitive
Large language models are prediction engines. They generate the next word based on everything that came before. That’s why they’re called language models — they’re not reasoning in a human sense, they’re simply extending patterns.
On top of that foundation, today’s models are tuned specifically to follow instructions. Using a process called reinforcement learning from human feedback (RLHF), humans will score sample outputs, and the model then learns to prefer the ones people find helpful and safe.
These dynamics make LLMs sensitive to framing. Because every output depends directly on the words in the prompt, and because the model has been trained to satisfy the implied intent of your prompt. Even small changes in how a question is asked can lead to different answers.
The Framing Effect in LLMs
Let’s get one thing out of the way right now: LLMs are not people, they do not think, and they do not have “biases” in the human sense of the word. But they are created by humans, and they essentially parse information and create their replies in a predictable fashion. That means that they are susceptible to the framing effect.
What we mean by framing effect in LLMs is slightly different than what it means for humans. For LLMs, the framing effect is less about bias or thought process, and more about systematic, frame-driven shifts in outputs.
Mechanics of Framing in LLMs
The framing effect in LLMs can be easily demonstrated with a few consistent patterns of queries:
- Leading the LLM: The way you frame a question will influence the answer. Starting with “assuming X is true…” will generally have the model do just that, even if X isn’t in reality true.
- Framing Risk: Similar to the original example for humans, if you frame a question around potential gains or losses, the models will potentially struggle with a recommendation, even when the math is the same.
- Numerical Anchors: If you introduce a number, such as “experts estimate 75% of…,” then the models may take that as reality, even if it isn’t.
- Authority and Priming: Telling the LLM that they are to act as a specific authority (“you are a CFO,” or “you are an expert at domestic planning”) will shift the output to that particular tone or expertise.
To be clear, none of these are bugs in the system, but they are clear examples of a framing effect.
Experiments You Can Try
Like MediaFeed’s content? Be sure to follow us.
This article originally appeared on Rebelmouse.com and was syndicated by MediaFeed.org
