Back to series
· 15 min read

Document Parsing and Knowledge Extraction: A Complete Technical Pipeline from Unstructured Data to Searchable Knowledge


I. Core Concepts of Document Parsing and Knowledge Extraction

In a RAG system, document parsing and knowledge extraction are two closely related stages with different responsibilities.

Document parsing mainly addresses:

  • What content does the document contain?
  • In what form is the content organized?
  • What relationships exist among text, tables, images, and layout?
  • How can the document's original structure and order be restored?

Knowledge extraction mainly addresses:

  • What key facts does the document contain?
  • Which content has independent knowledge value?
  • How are entities, attributes, relations, and events represented?
  • How can the content be converted into data usable for downstream retrieval and reasoning?

Differences between the two:

Dimension Document Parsing Knowledge Extraction
Core objective Restore document content and structure Extract valuable knowledge
Input PDF, Word, HTML, images, etc. Parsed document content
Processing target Text, layout, tables, images Entities, attributes, relations, facts
Output Structured document Knowledge records, entity relations, metadata
Key technologies OCR, Layout Analysis, Parser NER, Relation Extraction, LLM
Primary use Content restoration and organization Retrieval, question answering, knowledge reasoning

Note: not every RAG system requires complex knowledge extraction. For ordinary document question answering, structure-preserving text parsing plus reasonable chunking may already be sufficient; for enterprise knowledge bases, complex business documents, and cross-document associative retrieval, the value of knowledge extraction becomes more apparent.

II. Overall Processing Pipeline

Document parsing and knowledge extraction typically consist of multiple processing stages.

flowchart TD
    A["Raw Document<br/>PDF / Word / HTML / Images"] --> B["Document Identification & Classification"]
    B --> C["Document Parsing"]
    C --> D["Structure Recovery & Content Normalization"]
    D --> E["Knowledge Extraction"]
    E --> F["Quality Validation & Data Cleaning"]
    F --> G["Structured Knowledge Data"]
    G --> H["Chunking & Index Construction"]
    H --> I["Vector Database / Search Index"]

    style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style B fill:#EAF3EC,stroke:#83A995,color:#315D49
    style C fill:#EAF3EC,stroke:#83A995,color:#315D49
    style D fill:#EAF3EC,stroke:#83A995,color:#315D49
    style E fill:#E5EAF8,stroke:#9AA8D2,color:#3F4F7A
    style F fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
    style G fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style H fill:#EAF3EC,stroke:#83A995,color:#315D49
    style I fill:#E5F0E9,stroke:#83A995,color:#315D49

The whole process can be divided into four core stages:

  1. Document ingestion: identify file type, source, and basic attributes.
  2. Document parsing: extract text, tables, images, and layout structure.
  3. Knowledge extraction: identify entities, relations, attributes, and key facts.
  4. Data governance: validate, deduplicate, standardize, and build retrieval indexes.

III. Document Ingestion and Type Identification

Document ingestion is the entry point of the entire knowledge processing pipeline.

Different document formats have different internal structures and parsing methods, so a single parsing strategy cannot handle every file.

3.1 Common Document Types

Document type Characteristics Primary parsing method
PDF Fixed page layout PDF Parser, Layout Analysis
DOCX Structured Office document XML parsing, Office Parser
XLSX Worksheets, cells, and formulas Spreadsheet Parser
PPTX Slides, text boxes, and shapes Presentation Parser
HTML Tags and DOM structure HTML Parser
Markdown Headings, lists, code blocks Markdown Parser
TXT Plain text Text reading
Images Pixel data, no direct text structure OCR, vision models
Scanned PDF Composed of page images PDF rendering, OCR

Two types of PDF need to be distinguished in particular:

  • Text-based PDF: the page contains extractable text objects that can be read by a PDF text parser.
  • Scanned PDF: the page is essentially an image, and the text must be recognized through OCR.

There is also a hybrid PDF: some pages contain text objects while other pages consist of scanned images. A real-world system needs page-level detection and mixed-processing capability.

3.2 File identification cannot rely on extensions alone

File extensions are not necessarily trustworthy.

For example:

  • A .pdf file may actually be an image container or a corrupted file.
  • A .docx file is essentially a ZIP container holding XML files inside.
  • File extensions may have been modified manually.
  • The same file may have a MIME Type that does not match its actual content.

Production systems usually combine the following information to determine file type:

  • File extension.
  • MIME Type.
  • File header signature (Magic Number).
  • Internal container structure.
  • File content characteristics.

File ingestion also needs to handle size limits, corrupted files, compression bombs, malicious files and parsing resource limits.

3.3 Document metadata

Beyond the body content, the ingestion stage should also record document-level metadata.

For example:

{
  "document_id": "doc_001",
  "file_name": "technical_specification.pdf",
  "file_type": "application/pdf",
  "file_size": 2048000,
  "source": "internal_knowledge_base",
  "department": "engineering",
  "version": "2.1",
  "created_at": "2026-11-20T10:00:00Z",
  "status": "pending"
}

Where:

  • document_id: unique document identifier.
  • file_name: original file name.
  • source: document source.
  • department: the business department it belongs to.
  • version: document version.
  • status: current processing status.

Document metadata and body content should be managed separately.

The body is used for knowledge retrieval, while metadata is used for filtering, permission control, version management and data traceability.

IV. Core Technologies of Document Parsing

Document parsing is not simply converting a file into a string; it is about restoring, as far as possible, the content, hierarchy and spatial relationships of the original document.

4.1 Text Extraction

Text extraction is the most fundamental capability of document parsing.

Different document formats use different extraction methods:

  • PDF: extract the text objects on the page along with their coordinates.
  • DOCX: parse paragraphs, text and tables in the XML.
  • XLSX: extract worksheets, cells, formulas and data ranges.
  • PPTX: extract text boxes, titles and notes from slides.
  • HTML: parse the DOM and extract the body content and structural tags.
  • Markdown: parse headings, lists, quotes and code blocks.

For PDFs, text extraction usually requires not only obtaining the text content but also preserving the position of the text on the page.

For example:

{
  "page": 1,
  "text": "Transformer 是一种基于注意力机制的神经网络架构。",
  "bbox": [100, 120, 520, 160],
  "font_size": 14,
  "font_name": "Arial"
}

bbox represents the bounding box of the text on the page, usually expressed as:

bbox = [x_0,y_0,x_1,y_1]

Preserving coordinates helps with subsequent judgments:

  • Which pieces of text belong to the same paragraph.
  • Which pieces of text belong to a particular table.
  • Which pieces of text are headers or footers.
  • How the reading order should be arranged.
  • The hierarchical relationship between headings and body text.

If only the text is extracted and the coordinates are discarded, it may be impossible to accurately restore the original document structure.

4.2 OCR Optical Character Recognition

OCR (Optical Character Recognition) is used to recognize text from images.

It primarily solves the problem of being unable to extract text directly from scans, images and screenshots.

A typical OCR process:

flowchart TD
    A["Input Image"] --> B["Image Preprocessing"]
    B --> C["Text Region Detection"]
    C --> D["Text Region Cropping"]
    D --> E["Character Recognition"]
    E --> F["Text Post-processing"]
    F --> G["Recognition Result"]

    style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style B fill:#EAF3EC,stroke:#83A995,color:#315D49
    style C fill:#EAF3EC,stroke:#83A995,color:#315D49
    style D fill:#EAF3EC,stroke:#83A995,color:#315D49
    style E fill:#E5EAF8,stroke:#9AA8D2,color:#3F4F7A
    style F fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
    style G fill:#DCEFE5,stroke:#4F8B70,color:#24563F

The main steps of OCR include:

Step 1: Image Preprocessing

Common operations:

  • Grayscale conversion.
  • Binarization.
  • Denoising.
  • Skew correction.
  • Resolution adjustment.
  • Contrast enhancement.

Image quality directly affects the subsequent recognition performance.

Step 2: Text Detection

Locate text regions in the image.

The detection model needs to determine which regions contain text and the boundaries of each text region.

Common methods include:

  • EAST.
  • DBNet.
  • CRAFT.

Step 3: Text Recognition

Perform character or sequence recognition on the detected text regions.

Common technical approaches include:

  • CNN + RNN + CTC.
  • Attention-based Recognition.
  • Transformer-based Recognition.

Step 4: Text Post-processing

Handle formatting issues and errors in the recognition results:

  • Punctuation restoration.
  • Space and line break handling.
  • Character error correction.
  • Language rule validation.
  • Text region order restoration.

OCR output should also usually retain confidence scores and coordinates to facilitate subsequent quality assessment.

For example:

{
  "text": "人工智能技术",
  "confidence": 0.98,
  "bbox": [120, 80, 350, 120],
  "page": 1
}

Note: OCR confidence is the recognition model's estimate of the result and is not equivalent to accuracy verified by humans.

For high-risk fields such as contract amounts, ID card numbers, and equipment numbers, content correctness cannot be judged based on confidence alone.

4.3 Layout Analysis

Layout analysis is used to understand the spatial layout and region types of document pages.

A document contains not only text but also:

  • Titles.
  • Body paragraphs.
  • Tables.
  • Images.
  • Figure captions.
  • Headers and footers.
  • Page numbers.
  • Formulas.
  • Multi-column layouts.

The core task of layout analysis is to determine the type, position, and interrelationships of regions on a page.

For example:

┌────────────────────────────┐
│       Document Title       │
├────────────────────────────┤
│                            │
│       Body Paragraph       │
│       Body Paragraph       │
│                            │
├────────────────────────────┤
│           Table            │
│    ┌─────┬─────┬─────┐     │
│    │ A   │ B   │ C   │     │
│    ├─────┼─────┼─────┤     │
│    │ 10  │ 20  │ 30  │     │
│    └─────┴─────┴─────┘     │
├────────────────────────────┤
│           Image            │
│       Image Caption        │
└────────────────────────────┘

Common layout element categories:

Category Description
Title Document title
Section Header Section title
Text Body text
Table Table
Figure Image or chart
Caption Image or table caption
List List
Formula Mathematical formula
Header Page header
Footer Page footer

Common tools and techniques include:

  • LayoutParser: an ecosystem of tools for layout analysis.
  • Detectron2: an object detection and vision model framework that can be used for layout region detection.
  • LayoutLM: a family of document understanding models that combine text, layout, and visual information.
  • DocLayNet: a dataset for document layout analysis.
  • PP-Structure: a tool suite for structured document parsing.

It is important to distinguish: layout analysis and OCR are not the same task.

OCR focuses on what the text says, while layout analysis focuses on what regions exist on the page, where those regions are, and how they are organized.

4.4 Table Extraction

Tables are the part of document parsing where errors are most likely to occur.

Plain text extraction might produce:

Product Quantity Unit Price
A 10 20
B 15 30

But the original table may contain a complex structure:

Product Sales Data
Name Quantity Unit Price
A 10 20
B 15 30

If only the plain text is retained, the following may be lost:

  • The correspondence between headers and data.
  • Merged cells.
  • Row and column hierarchy.
  • Multi-level headers.
  • Unit information.
  • Table titles and notes.

Therefore, table extraction requires not only recognizing text but also restoring the table structure.

A typical processing pipeline:

flowchart TD
    A["Locate table region"] --> B["Recognize row and column structure"]
    B --> C["Detect cell boundaries"]
    C --> D["Recognize cell text"]
    D --> E["Handle merged cells"]
    E --> F["Associate headers with data"]
    F --> G["Structured table"]

    style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style B fill:#EAF3EC,stroke:#83A995,color:#315D49
    style C fill:#EAF3EC,stroke:#83A995,color:#315D49
    style D fill:#E5EAF8,stroke:#9AA8D2,color:#3F4F7A
    style E fill:#EAF3EC,stroke:#83A995,color:#315D49
    style F fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
    style G fill:#DCEFE5,stroke:#4F8B70,color:#24563F

The key to table extraction lies in restoring the logical structure, not just recognizing the characters in each cell.

For example:

{
  "table_id": "table_001",
  "headers": ["产品", "数量", "单价"],
  "rows": [
    ["A", 10, 20],
    ["B", 15, 30]
  ]
}

For complex tables, you also need to preserve:

  • row_span: row-spanning information.
  • col_span: column-spanning information.
  • cell_bbox: cell coordinates.
  • header_path: the path through multi-level headers.
  • unit: the unit of a numeric value.
  • caption: the table caption.

Pay special attention: numeric values in a table must be bound to their headers, units, and context.

For example, extracting 100 on its own does not mean much; you have to know whether it represents a quantity, an amount, a ratio, or a temperature.

4.5 Image and Chart Parsing

Images in documents may contain important knowledge:

  • Architecture diagrams.
  • Flowcharts.
  • System topology diagrams.
  • Business relationship diagrams.
  • Bar charts and line charts.
  • Product schematics.
  • Screenshots and user interfaces.

Extracting only the text surrounding an image may lose a large amount of information.

Image parsing typically requires:

  1. Locating the image region.
  2. Extracting the image and its original coordinates.
  3. Identifying the image type.
  4. Recognizing the text in the image.
  5. Analyzing the structure and semantics of the image.
  6. Associating the image content with the body text and captions.

For charts, it is also necessary to identify:

  • Coordinate axes.
  • Data series.
  • Legends.
  • Units.
  • Time ranges.
  • Data points.
  • Chart titles.

Vision-language models (VLMs) can help interpret complex images, but the descriptions they generate are the result of model inference and should not be treated directly as established facts from the original document.

From an engineering perspective, the original images, page positions, model descriptions and processing versions should be retained for review.

V. Document Structure Recovery and Standardization

After text, table and image extraction is complete, the logical structure of the document still needs to be recovered.

This is an important step in converting raw content into retrievable knowledge.

5.1 Why is structure recovery needed?

Consider the following content:

Chapter 1 System Architecture

1.1 Overall Design

The system adopts a microservices architecture.

1.2 Service Communication

Services communicate with each other via HTTP and message queues.

If the text is extracted directly, the result may be one continuous block of content.

But in reality, the document has a clear hierarchical relationship:

Chapter 1 System Architecture
    1.1 Overall Design
        The system adopts a microservices architecture.
    1.2 Service Communication
        Services communicate with each other via HTTP and message queues.

Structure recovery requires identifying:

  • Heading levels.
  • Paragraph boundaries.
  • List levels.
  • The chapter a table belongs to.
  • The correspondence between images and captions.
  • Page numbers and reading order.

Among these, reading order is especially important.

Two-column papers, newspapers and complex layouts may be arranged in an interleaved manner according to spatial position. Simply sorting text objects by their coordinates can easily mix content from different columns together.

Therefore, reading order recovery usually needs to take the following into comprehensive consideration:

  • Page columns.
  • Region types.
  • Text block coordinates.
  • Heading levels.
  • Paragraph continuity.
  • Document reading conventions.

5.2 Unified document intermediate representation

A production-grade system should not connect every file format directly to the downstream knowledge extraction module.

A more reasonable approach is to define a unified document intermediate representation (Document IR).

For example:

{
  "document_id": "doc_001",
  "title": "System Technical Design Document",
  "pages": [
    {
      "page_number": 1,
      "blocks": [
        {
          "block_id": "b001",
          "type": "heading",
          "level": 1,
          "text": "Chapter 1 System Architecture",
          "bbox": [80, 60, 500, 100]
        },
        {
          "block_id": "b002",
          "type": "paragraph",
          "text": "The system adopts a microservices architecture.",
          "parent_id": "b001",
          "bbox": [80, 120, 500, 160]
        },
        {
          "block_id": "b003",
          "type": "table",
          "caption": "Service Configuration Table",
          "table_data": {
            "headers": ["Service", "Port"],
            "rows": [
              ["API", "8080"],
              ["Auth", "8081"]
            ]
          }
        }
      ]
    }
  ]
}

The fields here are only examples; the actual schema should be designed according to business requirements.

The value of Document IR lies in:

  • Shielding differences between underlying file formats.
  • Unifying downstream processing interfaces.
  • Preserving original page and region information.
  • Supporting multiple knowledge extraction strategies.
  • Supporting re-segmentation and re-indexing.
  • Supporting problem localization and result traceability.

It is recommended to retain at least the following information:

Field Purpose
document_id Unique document identifier
block_id Unique content block identifier
parent_id Parent content block
block_type Content block type
text Extracted text
page_number Page number
bbox Page coordinates
source Original source
parser_version Parser version
confidence Recognition confidence
metadata Extended metadata

5.3 Content Cleaning and Normalization

Raw parsing results usually contain noise and need cleaning.

Common issues:

  • Extra spaces.
  • Repeated line breaks.
  • Repeated headers and footers.
  • Page numbers mixed into body text.
  • OCR typos.
  • Garbled text.
  • Broken lines.
  • Hyphenation splits.
  • Duplicate paragraphs.
  • Meaningless symbols.

For example, a PDF may contain:

A large language mod
el is a type of model based on the Transformer
architecture.

After cleaning:

A large language model is a type of model based on the Transformer architecture.

However, cleaning must not be excessive.

The following information should not be deleted arbitrarily:

  • Code indentation.
  • Mathematical formulas.
  • Empty values in tables.
  • List numbering.
  • Section numbering.
  • Minus signs and decimal points.
  • Dates and units.
  • Symbols with business meaning.

Especially in technical documents, the formatting of code, configurations, commands, and formulas is itself part of the knowledge.

Cleaning operations should preferably preserve a mapping between the original content and the normalized content to avoid losing traceability to the original text.

6. Core Technologies of Knowledge Extraction

Document parsing produces structured content, while knowledge extraction further extracts knowledge from the content that can be expressed independently, associated, and retrieved.

6.1 Entity Extraction (Named Entity Recognition)

Named entity recognition (NER) is used to identify objects with clear meaning in text.

Common entity types:

  • Person.
  • Organization.
  • Location.
  • Product.
  • Technology.
  • System.
  • Time.
  • Amount.
  • Business object.

For example:

OpenAI released GPT-4 in 2023.

Entity extraction result:

{
  "entities": [
    {
      "text": "OpenAI",
      "type": "ORGANIZATION"
    },
    {
      "text": "2023 年",
      "type": "DATE"
    },
    {
      "text": "GPT-4",
      "type": "MODEL"
    }
  ]
}

Entity recognition focuses not only on finding names but also on determining entity types, boundaries, and context.

Common technical approaches:

  • Rule and dictionary matching.
  • CRF.
  • BiLSTM-CRF.
  • BERT Token Classification.
  • LLM-based structured extraction.

Traditional NER models are suitable for fixed entity types and large-scale stable tasks; LLMs are better suited to complex, open-ended entity definitions but require strict output validation.

6.2 Attribute Extraction

Entity extraction identifies objects, while attribute extraction identifies their characteristics.

For example:

GPT-4 is a large language model released by OpenAI in 2023,
supporting text and image input.

The following can be extracted:

{
  "entity": "GPT-4",
  "attributes": {
    "developer": "OpenAI",
    "release_year": 2023,
    "model_type": "大语言模型",
    "input_modalities": [
      "text",
      "image"
    ]
  }
}

Common attributes include:

  • Name.
  • Type.
  • Time.
  • Status.
  • Version.
  • Specifications.
  • Organization.
  • Technical parameters.
  • Scope of application.

Attribute extraction must take contextual constraints into account.

For example:

The system supports a maximum of 1000 concurrent connections.

You should not extract only:

{
  "value": 1000
}

Instead, you should retain:

{
  "entity": "system",
  "attribute": "maximum number of concurrent connections",
  "value": 1000,
  "unit": "count"
}

6.3 Relation Extraction

Relation extraction is used to identify the semantic relations between entities.

For example:

OpenAI developed GPT-4.

This can be represented as:

(OpenAI, developed, GPT-4)

This is a triple:

(h,r,t)

Where:

  • h: Head, the head entity.
  • r: Relation, the relation.
  • t: Tail, the tail entity.

For example:

(OpenAI,\ developed,\ GPT\text{-}4)

Other common relations:

Head entity Relation Tail entity
OpenAI developed GPT-4
GPT-4 belongs to large language model
Transformer uses Self-Attention
Python supports object-oriented programming
Redis provides caching capability

The difficulties of relation extraction are:

  • Relations may span multiple sentences.
  • Entities may involve coreference relations.
  • Relations are directional.
  • Multiple relations may exist between the same entities.
  • Relations may be constrained by time and conditions.

For example:

System A calls System B.
System B caches user information through Redis.

Extracting only the entities and relations of each sentence is not enough; you also need to identify entity coreference and cross-sentence associations in the context.

6.4 Event Extraction

Event extraction focuses on what happened, as well as which participants and conditions the event involves.

For example:

In March 2026, a company released a new version of its database product.

You can extract:

{
  "event_type": "PRODUCT_RELEASE",
  "time": "2026-03",
  "organization": "a company",
  "product": "database product",
  "action": "release"
}

An event usually includes:

  • Event type.
  • Participating entities.
  • Time of occurrence.
  • Location of occurrence.
  • Trigger words.
  • Event arguments.

Common event types:

  • Product release.
  • Corporate acquisition.
  • System failure.
  • Order creation.
  • User login.
  • Transaction completion.
  • Contract signing.

Event extraction is valuable for news analysis, enterprise knowledge bases, audit systems, and business process analysis.

6.5 Fact Extraction

Compared with simple entity recognition, fact extraction focuses more on complete semantics.

For example:

The default port of Redis is 6379.

You can extract:

{
  "subject": "Redis",
  "predicate": "default port",
  "object": 6379,
  "unit": null,
  "source": "doc_001",
  "page": 12
}

A usable fact record should contain not only the subject, predicate, and object, but should also retain, as far as possible:

  • The source document.
  • The page number or paragraph location.
  • The effective time.
  • The scope of applicability.
  • The confidence level.
  • The document version.

This is especially true for enterprise knowledge bases, where facts are usually not permanently valid.

For example:

The system currently uses Redis 7.0.

And:

The system's historical version used Redis 6.2.

Both facts may be correct, but they correspond to different points in time and different versions.

If time and version information is not retained, the knowledge base may mistake historical facts for current facts.

7. Using Large Language Models for Knowledge Extraction

Traditional information extraction models usually require entity categories, relation types and training data to be defined in advance.

Large language models can perform more flexible knowledge extraction tasks based on natural language instructions.

7.1 Prompt-Based Structured Extraction

For example, the input:

Please extract entities, attributes and relations from the following content.

Text:
Redis is an open-source in-memory data structure storage system,
commonly used for caching, message queues and real-time data processing.

Required output:

{
  "entities": [
    {
      "name": "Redis",
      "type": "SOFTWARE"
    },
    {
      "name": "cache",
      "type": "APPLICATION"
    },
    {
      "name": "message queue",
      "type": "APPLICATION"
    },
    {
      "name": "real-time data processing",
      "type": "APPLICATION"
    }
  ],
  "relations": [
    {
      "subject": "Redis",
      "predicate": "commonly used for",
      "object": "cache"
    },
    {
      "subject": "Redis",
      "predicate": "commonly used for",
      "object": "message queue"
    },
    {
      "subject": "Redis",
      "predicate": "commonly used for",
      "object": "real-time data processing"
    }
  ]
}

This approach does not require training a dedicated model for each new entity type, so it offers good flexibility.

However, note that:

  • The model may generate entities that do not appear in the source text.
  • It may incorrectly infer relations between entities.
  • It may turn uncertain content into definite facts.
  • It may omit important information.
  • It may output JSON that does not conform to the schema.

Therefore, LLM extraction results must be validated and cannot be written directly into an authoritative knowledge base.

7.2 Structured Output and Schema Constraints

In engineering practice, a fixed data structure should be defined wherever possible.

For example:

{
  "type": "object",
  "properties": {
    "entities": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "name": {
            "type": "string"
          },
          "type": {
            "type": "string"
          }
        },
        "required": [
          "name",
          "type"
        ],
        "additionalProperties": false
      }
    },
    "relations": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "subject": {
            "type": "string"
          },
          "predicate": {
            "type": "string"
          },
          "object": {
            "type": "string"
          }
        },
        "required": [
          "subject",
          "predicate",
          "object"
        ],
        "additionalProperties": false
      }
    }
  },
  "required": [
    "entities",
    "relations"
  ],
  "additionalProperties": false
}

The schema mainly addresses structural validity:

  • Whether fields exist.
  • Whether field types are correct.
  • Whether additional fields are allowed.
  • Whether array elements conform to the definition.

However, being schema-valid does not mean the knowledge is correct.

For example:

{
  "subject": "Redis",
  "predicate": "开发",
  "object": "OpenAI"
}

This may fully conform to JSON Schema while being semantically incorrect.

Therefore, further semantic validation is required.

7.3 Evidence-Based Extraction

A more reliable extraction method is to require the model to provide source-text evidence for each fact.

For example:

{
  "subject": "Redis",
  "predicate": "默认端口",
  "object": "6379",
  "evidence": {
    "text": "Redis 默认端口为 6379。",
    "page": 12,
    "block_id": "b012"
  }
}

Evidence can be used to:

  • Check whether a fact comes from the source text.
  • Locate the original passage.
  • Detect model hallucinations.
  • Support manual review.
  • Generate citations for question answering.
  • Trace the source of knowledge.

Note that the evidence text returned by the model may also be an incorrect quotation or paraphrase, so you cannot merely check whether the fields exist. Production systems should match the evidence against the original content and retain the original location identifiers.

7.4 Layered Extraction Strategy

It is not suitable to feed a complex document into a model all at once and ask it to extract all knowledge.

A more reasonable strategy is to process it in layers:

flowchart TD
    A["Structured document"] --> B["Document-level analysis"]
    B --> C["Section-level analysis"]
    C --> D["Paragraph-level extraction"]
    D --> E["Entity and attribute recognition"]
    E --> F["Relation and event extraction"]
    F --> G["Cross-paragraph association"]
    G --> H["Fact validation and merging"]
    H --> I["Knowledge data"]

    style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style B fill:#EAF3EC,stroke:#83A995,color:#315D49
    style C fill:#EAF3EC,stroke:#83A995,color:#315D49
    style D fill:#EAF3EC,stroke:#83A995,color:#315D49
    style E fill:#E5EAF8,stroke:#9AA8D2,color:#3F4F7A
    style F fill:#E5EAF8,stroke:#9AA8D2,color:#3F4F7A
    style G fill:#EAF3EC,stroke:#83A995,color:#315D49
    style H fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
    style I fill:#DCEFE5,stroke:#4F8B70,color:#24563F

Hierarchical extraction solves several problems:

  1. Reducing the length of a single input.
  2. Preserving section and paragraph context.
  3. Avoiding missed information in long documents.
  4. Separating local extraction from global association.
  5. Making it easy to configure different models for different stages.

Among these, document-level analysis is responsible for identifying topics, sections and the overall context; paragraph-level extraction is responsible for extracting local facts; and cross-paragraph association is responsible for resolving anaphora, entity normalization and relation completion.