I. What Is LLM Hallucination?
LLM hallucination refers to a large language model generating content that does not conform to objective facts, input evidence, or task constraints, while presenting it in a fluent, confident, or even professional manner.
For example, ask the model:
Please introduce an academic paper that does not actually exist, including its authors, publication year, and core conclusions.
The model may generate:
- A seemingly real paper title.
- Several fictitious authors.
- A specific publication year.
- A journal or conference that does not exist.
- A logically complete research conclusion.
This information may be very natural in linguistic form, but it has no reliable factual basis.
1. Typical Types of LLM Hallucination
| Type | Manifestation | Example |
|---|---|---|
| Factual hallucination | Generates incorrect facts | Fabricates historical events or people's experiences |
| Citation hallucination | Fabricates sources and citations | Generates non-existent papers, books, or URLs |
| Numerical hallucination | Incorrect values, dates, or statistical data | Incorrectly describes market size |
| Entity hallucination | Confuses people, organizations, or products | Mixes product information from different companies |
| Logical hallucination | There are errors in the reasoning process | Correct premises but an invalid conclusion |
| Contextual hallucination | Ignores key information in the input | Answer contradicts the document provided by the user |
| Code hallucination | Generates non-existent APIs or interfaces | Calls a function that does not actually exist |
| Over-inference | Draws definite conclusions from limited evidence | Interprets correlation as causation |
These types may occur simultaneously.
For example, when answering a technical question, the model may first fabricate an API, then invent parameter descriptions, and finally provide code that cannot run.
2. Hallucination Is Not the Same as Random Nonsense
Hallucination is fundamentally different from completely random generation.
LLM hallucinations usually have the following characteristics:
- The language structure conforms to natural language conventions.
- The content is somewhat relevant to the topic of the question.
- Details may have superficial logical consistency among themselves.
- Some content is correct, and some is incorrect.
- The model may express incorrect conclusions with high confidence.
This is also why hallucinations are difficult for ordinary users to identify.
The fluency of content generated by a large model does not necessarily correspond to its factual accuracy.
II. Root Cause: The Training Objective of LLMs Is Not Factual Verification
To understand hallucination, we must first understand how large language models are actually trained.
Mainstream decoder-only large language models typically use an autoregressive language modeling objective, learning from large amounts of text to predict the next Token.
1. Next Token Prediction
Suppose the training text is:
Artificial intelligence is changing the way software is developed.
The tokenizer converts the text into a Token sequence:
The model learns to predict the next Token based on the preceding Tokens:
The joint probability of the entire sequence can be expressed as:
During training, the model learns to increase the predicted probability of the true training Tokens by minimizing the negative log-likelihood loss:
Where:
- \theta: model parameters.
- x_t: the true Token in the training text.
- x_{<t}: the context before the current position.
- P_\theta: the conditional probability distribution determined by the model parameters.
This training objective mainly requires the model to learn statistical patterns in text sequences, rather than establish an independent truth verification mechanism for each fact.
2. Language Modeling Objectives Differ from Factual Verification Objectives
Suppose the training data contains many similar expressions:
A certain technology was proposed by a certain research team in a certain year.
The model can learn that:
- Technology introductions usually include the proposer.
- Technology histories usually include dates.
- Academic expressions usually include the research team.
- Technology descriptions usually include background and conclusions.
When the model encounters a question with insufficient information, it may still generate an answer that conforms to the above language patterns.
However, conforming to language patterns does not mean that the specific people, years, and events generated actually exist.
Therefore:
What the model learns is a conditional probability distribution, not a database that naturally possesses factual verification capabilities.
3. Predicted Probability Is Not Factual Probability
The model outputs:
It represents the conditional probability of the model predicting a certain Token under the current context.
It does not directly represent:
The two are not the same probability concept.
For example, the model may be very familiar with the expression structure of a certain type of academic paper and therefore be able to generate a citation that is highly plausible in form.
However, the model's high predicted probability for this text does not prove that the paper actually exists.
3. Training Data Problems: Where Does Erroneous Knowledge Come From?
Large language models are pre-trained on large-scale corpora.
This data may come from:
- Web page text.
- News reports.
- Books.
- Academic papers.
- Technical documentation.
- Open-source code.
- Q&A communities.
- Public datasets.
- Synthetic data.
The quality and distribution of training data directly affect the knowledge representations the model learns.
1. Training Data Itself May Contain Errors
The internet is not a knowledge base that has undergone rigorous fact-checking.
It may contain:
- Articles with factual errors.
- Outdated technical documentation.
- Contradictory reports.
- Unverified opinions.
- Fake news.
- Incorrect code examples.
- Misinformation that has been repeatedly propagated.
If the model learns this content during training, it may encode the misinformation into its parameters.
Even if an error appears very frequently in the training data, that does not mean it is true in the real world.
2. Imbalanced Data Distribution
Different knowledge domains are covered to varying degrees in the training data.
For example:
- Mainstream programming languages have abundant materials.
- Niche frameworks have fewer materials.
- Newly released software versions have limited materials.
- Some regional events lack public reporting.
- The latest research in specialized fields may not yet have entered the training data.
When the model encounters topics with insufficient coverage in the training data, it may rely more on existing knowledge patterns to fill in the gaps.
This can easily lead to:
- Confusing different versions of an API.
- Mixing up the features of similar products.
- Describing obscure technologies incorrectly.
- Fabricating details without training evidence.
3. Outdated Data
A model's pre-training data usually has a cutoff point in time.
Suppose the model learned the API documentation for a piece of software from 2024, while a user asks about the latest version from 2026.
The model may still answer based on its old knowledge.
For example:
- Deprecated APIs are treated as current interfaces.
- Old configuration is described as the latest configuration.
- Product features that have been adjusted are introduced incorrectly.
- Prices and policies that have already changed are still cited.
This situation does not necessarily mean the model is fabricating; it may be that the model is using outdated knowledge.
4. Data Conflicts and Source Reliability
The training data may contain contradictory information.
For example, different websites give different descriptions of when an event occurred.
The model must learn from a large number of textual patterns, but it does not inherently possess a reliable mechanism for ranking all sources by authority at the factual level.
When different pieces of information conflict, the model may form vague or even incorrect knowledge representations.
5. The Relationship Between Data Quality and Hallucination
flowchart TB
A["Large-scale training data"] --> B["Data collection"]
B --> C["Data cleaning and filtering"]
C --> D{"Is data quality reliable?"}
D -->|"Yes"| E["Enter the training corpus"]
D -->|"No"| F["Erroneous, outdated, or conflicting information"]
F --> E
E --> G["Model pre-training"]
G --> H["Parameter learning"]
H --> I["Form internal knowledge representation"]
I --> J["Reasoning and text generation"]
J --> K["May produce factual errors"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style C fill:#EAF3EC,stroke:#83A995,color:#315D49
style D fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style E fill:#EAF3EC,stroke:#83A995,color:#315D49
style F fill:#F7EAE7,stroke:#C98D7C,color:#754438
style G fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style H fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style I fill:#EAF3EC,stroke:#83A995,color:#315D49
style J fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style K fill:#F7EAE7,stroke:#C98D7C,color:#754438
It should be emphasized that even if the training data is rigorously cleaned, there is no guarantee that hallucinations will be completely eliminated. Hallucinations are also related to the model's objectives, parameterized representations, contextual understanding, and reasoning mechanisms.
4. Parametric Knowledge: Why Does the Model Get Facts Wrong?
Large language models generally do not have structured fact tables like traditional databases do.
They mainly learn and store language patterns, knowledge associations, and pattern representations through neural network parameters.
1. Knowledge Is Encoded in Model Parameters
A model can be abstractly represented as:
Where:
- x: input context.
- \theta: model parameters.
- f_\theta: the model's computation process.
- y: model output.
The model's knowledge is not simply stored in some parameter location that can be directly queried.
It is usually distributed across a large number of weights and neural network computation paths.
For example, the model may learn:
- A person is associated with a company.
- A technology is related to a research direction.
- A concept usually appears in a certain type of context.
- An API typically has certain parameters.
These associations participate in model prediction in a distributed manner.
2. Parametric knowledge is not equivalent to database records
A traditional database can store:
| Person | Company | Position |
|---|---|---|
| A | Company X | CEO |
During queries, retrieval can be performed based on fields and conditions.
Neural network parameters do not provide such an explicit structure.
The model needs to compute the output through a multi-layer neural network based on the input context.
Therefore, the model may:
- Remember a person but confuse the company they belong to.
- Remember a technology but confuse when it was proposed.
- Remember a product but confuse the version.
- Remember a concept but be unable to accurately trace it back to its original source.
3. Confusion may occur between similar pieces of knowledge
Suppose the training data contains all of the following:
- Technology A: developed by Team X.
- Technology B: developed by Team Y.
- Technology C: developed by Team Z.
These technologies may belong to the same field and have similar names and descriptions.
When the model generates content, it may incorrectly combine attributes across different entities.
For example:
Technology A + Team Y + Technology C's release time
Ultimately, it generates a combination of facts that looks complete but does not actually exist.
This type of problem can be understood as incorrect knowledge association or confusion of entity attributes.
4. Differences between parametric memory and factual retrieval
| Dimension | Parametric knowledge | Structured database |
|---|---|---|
| Storage form | Distributed neural network parameters | Explicit records |
| Query method | Neural network forward computation | Query statements or indexes |
| Update method | Fine-tuning or retraining, etc. | Updating records |
| Traceability | Usually weak | Usually strong |
| Factual constraints | Not naturally guaranteed | Can be achieved through constraints and validation |
| Information conflicts | May form ambiguous representations | Can explicitly preserve sources and versions |
Parametric knowledge is suitable for learning complex patterns and semantic associations, but it cannot replace factual databases that require strict accuracy.
V. Context understanding problems: Why do models misunderstand users?
The generation results of large models are highly dependent on the current context.
Even if the model possesses relevant knowledge, it may still produce incorrect answers because the input information is incomplete, instructions conflict, or the context is too long.
1. Insufficient input information
For example, a user asks:
What features are supported by the latest version of this framework?
But does not provide:
- The framework name.
- The specific version.
- The runtime environment.
- The query time.
- The official documentation source.
The model may guess which framework the user means based on the context, and then generate inaccurate content.
When a question has multiple reasonable interpretations, if the model does not proactively clarify, it may choose the wrong interpretation path.
2. Unclear referential relationships
For example:
Both Python and Go support concurrency. What is the difference between its scheduler and the other's?
Here, "its" may refer to Python or Go.
If the model incorrectly resolves the referential relationship, it may answer with the scheduling mechanism of the other language.
3. Excessively long context leads to insufficient use of information
A larger Context Window does not mean that the model can use information at all positions equally effectively.
In long-context scenarios, the following may occur:
- Important information is drowned out by large amounts of irrelevant text.
- Early constraints are not fully utilized.
- Information scattered across different paragraphs is not correctly associated.
- Key information in middle positions is ignored.
For example, at the beginning of the conversation, the user explicitly specifies:
The project uses Java 17 and does not allow the use of Java 21 features.
After a large amount of subsequent conversation, the model may generate code that depends on Java 21.
This shows that although the model received the early constraint, it may not always use it correctly.
4. Instruction conflicts
The model may simultaneously receive:
- System Prompt.
- Developer Instruction.
- User Prompt.
- Conversation history.
- Retrieved documents.
- Tool outputs.
This information may contain mutually contradictory requirements.
If the system does not reasonably handle priorities and trusted sources, the model may be interfered with by low-credibility content.
For example, a retrieved webpage contains an incorrect configuration, and the user requires strict adherence to official documentation, but the model brings the incorrect configuration from the webpage into the answer.
5. How context problems arise
flowchart TB
A["User input and historical context"] --> B["Context parsing"]
B --> C["Extract entities, instructions, and constraints"]
C --> D{"Is the information complete and consistent?"}
D -->|"Yes"| E["Build task representation"]
D -->|"No"| F["Ambiguity or missing information"]
F --> G["Incorrect association or incomplete understanding"]
E --> H["Model generation"]
G --> H
H --> I["Output may deviate from the actual requirement"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style D fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style E fill:#EAF3EC,stroke:#83A995,color:#315D49
style F fill:#F7EAE7,stroke:#C98D7C,color:#754438
style G fill:#F7EAE7,stroke:#C98D7C,color:#754438
style H fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style I fill:#F7EAE7,stroke:#C98D7C,color:#754438
VI. Decoding Strategies: Why Does the Model Choose the Wrong Answer?
Even if the model has already computed a probability distribution, the final output is still affected by the sampling strategy.
Common generation parameters include:
- Temperature.
- Top-K.
- Top-P.
- Repetition Penalty.
- Frequency Penalty.
- Presence Penalty.
These parameters affect how the model selects the next Token.
1. Highest Probability Does Not Mean Factually Correct
Suppose the model generates the following probability distribution for a question:
| Candidate answer | Model probability |
|---|---|
| Answer A | 0.70 |
| Answer B | 0.20 |
| Answer C | 0.10 |
If Answer A is actually wrong, then even if the model chooses it with a probability of 0.70, that does not prove that Answer A is correct.
Model probability reflects its current predictive distribution, not truth verified by external facts.
2. High Temperature May Increase Unreliable Output
Higher Temperature usually makes the probability distribution flatter.
Candidate Tokens that originally had low probability gain a greater chance of being sampled.
This may increase:
- Uncommon expressions.
- Unexpected content.
- Factually incorrect combinations.
- Unstable response results.
But one cannot simply assume that high Temperature necessarily produces hallucinations.
Even with greedy decoding at Temperature 0, the model may still consistently output incorrect facts.
3. Greedy Decoding Cannot Eliminate Hallucinations Either
Greedy Decoding selects the Token with the highest current probability at each step:
It reduces sampling randomness, but it does not introduce a factual verification mechanism.
If the model considers a wrong answer to be the most likely expression, greedy decoding will still choose it.
Therefore:
Reducing randomness can reduce output fluctuations, but it cannot fundamentally solve factual errors.
VII. Reasoning and Knowledge Boundaries: Why Does the Model Fabricate Nonexistent Content?
Large models often face questions beyond their own knowledge scope.
For example:
- Research results that have not yet been made public.
- The latest news events.
- Internal APIs of niche software.
- Academic papers that do not exist.
- Private databases that cannot be accessed.
- Fictional entities deliberately proposed by users.
The model may not have enough evidence to answer, but it still generates a complete answer.
1. The Model’s Completion Tendency When Reliable Evidence Is Lacking
Language models are trained to generate coherent text based on context.
When a question requires:
- Names of people.
- Specific dates.
- Paper titles.
- Citation sources.
- Technical parameters.
The model may complete them according to the language patterns in which such content usually appears.
For example:
Paper title:
Author:
Publication year:
Research institution:
Core conclusion:
Even if there is no real paper, the model may generate complete field content.
This is a typical kind of unsupported completion.
2. Insufficient Expression of Uncertainty
There is no natural one-to-one correspondence between the model’s internal predictive distribution and its final natural-language expression.
The model may have substantial candidate uncertainty internally, but still ultimately output:
This technology was formally proposed by a research team in 2022.
What the user sees is a definitive statement, not the state in which the model lacks evidence for the facts.
This makes hallucinations harder to identify.
3. Lack of an Effective Refusal Mechanism
If the model has not learned to refuse to answer when evidence is insufficient, it may tend to generate an apparently reasonable answer.
Ideally, a model should be able to distinguish between:
- Known and well-supported.
- Supported to some extent, but with uncertainty.
- Insufficient information, requiring further retrieval.
- Unverifiable, and no definitive conclusion should be given.
But this capability needs to be achieved through training, evaluation, reasoning strategies, and external tools working together.
8. Errors in Reasoning Chains: Why Do Models Produce Logical Hallucinations?
Large models can generate multi-step reasoning text, but this does not mean every step of the reasoning is correct.
1. False Premises Propagate to Subsequent Conclusions
Suppose the model incorrectly identifies a technical version in the first step.
Subsequent reasoning may continue the analysis based on this incorrect version, ultimately generating a seemingly complete technical solution.
flowchart TB
A["User poses a complex question"] --> B["Model parses the question"]
B --> C["Establish reasoning premises"]
C --> D{"Are the premises correct?"}
D -->|"Yes"| E["Execute subsequent reasoning"]
D -->|"No"| F["Introduce erroneous assumptions"]
F --> G["Continue reasoning based on erroneous premises"]
E --> H["Form a conclusion"]
G --> H
H --> I["Output a potentially incorrect answer"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style D fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style E fill:#EAF3EC,stroke:#83A995,color:#315D49
style F fill:#F7EAE7,stroke:#C98D7C,color:#754438
style G fill:#F7EAE7,stroke:#C98D7C,color:#754438
style H fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style I fill:#F7EAE7,stroke:#C98D7C,color:#754438
2. Error Accumulation in Complex Reasoning
Multi-step tasks usually involve multiple intermediate judgments.
Suppose a task contains five reasoning steps, and each step has some probability of error.
Under a simplifying assumption, if each step is independent and has a correctness probability of p, then the probability that all steps are correct is:
Suppose the correctness probability of each step is 0.95, and there are 10 steps in total:
That is approximately 59.9%.
This is only an idealized model used to illustrate multi-step error accumulation. Errors in real models are usually not independent, and the correctness probabilities of different steps are not the same.
But it reveals an important problem:
Local errors in complex tasks may gradually affect the final conclusion.
3. Reasoning Text Is Not Equivalent to the Actual Internal Computation Trace
The explanation process generated by the model is part of the model's output.
It does not necessarily fully or accurately reflect all computational activity inside the model.
Therefore, one cannot conclude that a model's conclusion has been reliably verified merely because it provides a well-structured reasoning explanation.
9. Why Can't SFT and Preference Alignment Completely Eliminate Hallucinations?
Large models are usually not only pretrained but also undergo instruction fine-tuning and preference alignment.
These training stages can improve the model's interaction capabilities, but they cannot guarantee that all facts are correct.
1. SFT: Supervised Fine-Tuning
SFT (Supervised Fine-Tuning) uses instruction and response data to train the model so that it learns:
- How to understand user instructions.
- How to organize responses.
- How to follow output formats.
- How to conduct multi-turn conversations.
- How to refuse to answer in specific scenarios.
But SFT depends on the quality of the training data.