I. What Is Structured Output?
Structured output refers to requiring a large language model to generate results according to a predefined data structure, rather than freely generating natural language in an arbitrary format.
Traditional large models mainly return results in text form:
User Zhang San, age 28, from Shanghai, currently a backend engineer.
This format is suitable for reading, but for programs it is difficult to parse:
- The user's name and age need to be extracted from the text.
- The order of fields may change.
- The way fields are described may be inconsistent.
- Extra explanatory text may appear.
- Some fields may be missing.
- Numeric values may be output as strings.
If the model is asked to return JSON:
{
"name": "Zhang San",
"age": 28,
"city": "Shanghai",
"occupation": "Backend Engineer"
}
The program can then parse the data with a JSON parser and access the corresponding content by field name.
The core value of structured output is:
- Stable format: Responses follow the agreed data structure.
- Clear types: Strings, numbers, booleans, arrays, and objects are distinguished.
- Easy to validate: You can check whether fields exist and whether their types are correct.
- Easy to integrate: It can connect directly to databases, business APIs, and workflows.
- Lower parsing cost: It reduces reliance on regular expressions and text cleaning.
Note that structured output mainly addresses data format and structural constraints; it does not automatically guarantee that the facts provided by the model are correct, nor that the business logic is valid.
II. Why Do Large Models Need Structured Output?
The native output of a large language model is a token sequence.
The model typically predicts the next token based on context and ultimately generates a piece of text. It does not naturally return data according to the database fields or interface definitions of a particular business system.
2.1 Problems with Free-Text Output
Suppose you need to extract the name, age, and email address from user input.
User input:
My name is Li Ming, I am 30 years old, and my email is liming@example.com.
The model may return:
Name: Li Ming
Age: 30
Email: liming@example.com
It may also return:
{
"姓名": "李明",
"年龄": "30",
"邮箱地址": "liming@example.com"
}
It may also return:
The extraction result is as follows:
{
"name": "Li Ming",
"age": 30,
"email": "liming@example.com"
}
To humans, these contents express the same information, but to programs, the field names, data types, and response structures are not uniform.
Common problems include:
| Problem | Manifestation | Engineering Impact |
|---|---|---|
| Field names unstable | name, 姓名, and username are mixed |
Field mapping is complex |
| Type inconsistency | Age is sometimes a number and sometimes a string | Type conversion and validation |
| Missing fields | Some fields are not returned | Null value handling |
| Format error | JSON brackets or quotes are invalid | Parsing fails |
| Extra text | Explanatory text is included before or after the JSON | Parsing boundaries are unclear |
| Structural change | An array becomes an object or the field hierarchy changes | Downstream programs fail |
In production systems, these problems can lead to API call failures, data ingestion errors, or workflow interruptions.
2.2 How Structured Output Solves These Problems
Structured output uses a predefined data structure to make the model generate responses according to explicit fields and types.
For example:
{
"name": "Li Ming",
"age": 30,
"email": "liming@example.com"
}
The application can parse it directly:
import json
response = '''
{
"name": "Li Ming",
"age": 30,
"email": "liming@example.com"
}
'''
data = json.loads(response)
print(data["name"])
print(data["age"])
print(data["email"])
Output:
Li Ming
30
liming@example.com
Structured output enables large models to move beyond being mere text-generation components and further become components that can participate in data processing within software systems.
3. What Is JSON?
JSON (JavaScript Object Notation) is a lightweight data interchange format.
Although the name JSON comes from JavaScript, it is an independent data format that can be parsed by mainstream programming languages such as Python, Java, Go, Rust, and C#.
JSON is mainly used for:
- API requests and responses.
- Configuration files.
- Data exchange.
- Data persistence.
- Structured responses from large models.
- Tool-call parameters.
3.1 Basic JSON Data Types
JSON defines six data types.
| Type | Description | Example |
|---|---|---|
| String | Character string | "hello" |
| Number | Number | 18, 3.14 |
| Boolean | Boolean value | true, false |
| Null | Null value | null |
| Object | Object | {"name":"张三"} |
| Array | Array | ["Python","Go"] |
Note that JSON's Number type does not distinguish between integers and floating-point numbers as two separate types, but a specific programming language may map them to different numeric types when parsing.
3.2 JSON Object
An Object is a data structure composed of key-value pairs.
{
"id": 1001,
"name": "张三",
"active": true
}
Here:
idis the field name.1001is the field value.nameis the field name."张三"is the field value.activeis a boolean field.
A JSON Object is represented using curly braces {}.
3.3 JSON Array
An Array is used to represent an ordered collection of data.
{
"skills": [
"Python",
"Java",
"Go"
]
}
Elements in an array can be strings, numbers, booleans, objects, or other arrays.
For example:
{
"users": [
{
"id": 1,
"name": "张三"
},
{
"id": 2,
"name": "李四"
}
]
}
This structure is very suitable for representing list-style business data.
3.4 JSON Syntax Rules
JSON has strict formatting requirements:
- Objects use
{}. - Arrays use
[]. - Strings must use double quotes.
- Field names must be strings.
- A colon is used between a field and its value.
- Commas are used between multiple fields.
- Trailing commas are not allowed.
- Comments are not supported.
- JavaScript special values such as
undefined,NaN, andInfinityare not supported.
For example, the following is valid JSON:
{
"name": "张三",
"age": 28,
"active": true
}
The following, however, is not valid JSON:
{
name: "张三",
age: 28,
active: true,
}
The reasons are:
- The field names do not use double quotes.
- There is an extra comma after the last field.
In large-model applications, it is essential to distinguish JSON format from JavaScript object literals; the latter cannot be directly treated as valid JSON.
4. What Is JSON Schema?
JSON Schema is a specification for describing and validating JSON data structures.
It can define:
- Data types.
- Field names.
- Required fields.
- Optional fields.
- String length.
- Numeric ranges.
- Enumeration values.
- Array element types.
- Nested object structures.
- Field count limits.
- String format constraints.
Simply put:
- JSON describes the actual data.
- JSON Schema describes what the data should look like.
4.1 Differences Between JSON and JSON Schema
JSON:
{
"name": "张三",
"age": 28
}
JSON Schema:
{
"type": "object",
"properties": {
"name": {
"type": "string"
},
"age": {
"type": "integer"
}
},
"required": [
"name",
"age"
],
"additionalProperties": false
}
The relationship between the two:
| Comparison aspect | JSON | JSON Schema |
|---|---|---|
| Core purpose | Store and transport data | Describe and validate data structures |
| Content | Actual fields and values | Types, constraints, and rules |
| Contains business data | Yes | Usually no |
| Defines required fields | No | Can define |
| Restricts field types | No | Can define |
| Can be used for validation | As the object being validated | As validation rules |
4.2 Basic Structure of JSON Schema
A common JSON Schema:
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"title": "User",
"description": "用户信息",
"properties": {
"name": {
"type": "string",
"description": "用户姓名"
},
"age": {
"type": "integer",
"minimum": 0,
"maximum": 150
},
"email": {
"type": "string",
"format": "email"
}
},
"required": [
"name",
"age"
],
"additionalProperties": false
}
Main field descriptions:
| Field | Purpose |
|---|---|
$schema |
Declares the JSON Schema version used |
type |
Specifies the data type |
title |
Schema title |
description |
Describes the purpose of a field or structure |
properties |
Defines object properties |
required |
Specifies required fields |
additionalProperties |
Controls whether additional fields are allowed |
minimum |
Minimum numeric value |
maximum |
Maximum numeric value |
format |
Declares the semantic format of data |
Note: The validation behavior of format depends on the JSON Schema validator used and its configuration. Some implementations treat it as an annotation, while others perform format checking.
5. Core Constraints of JSON Schema
The value of JSON Schema is not limited to describing fields; it can also express relatively complex data constraints.
5.1 type: Data Type
type is used to define the data type of a field.
{
"type": "object",
"properties": {
"name": {
"type": "string"
},
"age": {
"type": "integer"
},
"score": {
"type": "number"
},
"active": {
"type": "boolean"
},
"tags": {
"type": "array"
},
"profile": {
"type": "object"
}
}
}
Common types:
stringintegernumberbooleanobjectarraynull
JSON Schema also supports expressing multiple allowed types through a type array, for example:
{
"type": ["string", "null"]
}
This means the field can be a string or null.
5.2 properties: Field Definitions
properties is used to define the properties that are allowed to be described in an object and their constraints.
{
"type": "object",
"properties": {
"username": {
"type": "string"
},
"age": {
"type": "integer"
}
}
}
Note that being defined in properties does not mean the field must exist.
Whether a field must be present needs to be specified through required.
5.3 required: Required Fields
{
"type": "object",
"properties": {
"name": {
"type": "string"
},
"age": {
"type": "integer"
}
},
"required": [
"name"
]
}
Here:
namemust exist.agemay be absent.
One common point of confusion is:
required constrains whether a field exists, not whether its value is non-empty.
For example:
{
"name": ""
}
In the above Schema, name still meets the required requirement, because the field exists and is of type string.
If you also need to restrict the string so that it cannot be empty, you can add:
{
"type": "string",
"minLength": 1
}
5.4 additionalProperties: Additional Fields
By default, JSON Schema does not automatically forbid an object from containing undefined fields.
For example:
{
"type": "object",
"properties": {
"name": {
"type": "string"
}
}
}
The following data may still pass validation against the basic Schema:
{
"name": "张三",
"age": 28
}
If you want to strictly restrict fields, you can set:
{
"type": "object",
"properties": {
"name": {
"type": "string"
}
},
"additionalProperties": false
}
In this case, the undefined age field will not conform to the Schema.
In strict data APIs, this setting is very important.
5.5 enum: Enumerated Values
enum is used to restrict a field to only the specified values.
{
"type": "object",
"properties": {
"status": {
"type": "string",
"enum": [
"pending",
"processing",
"completed",
"failed"
]
}
},
"required": [
"status"
]
}
Good for:
- Order status.
- User roles.
- Task types.
- Risk levels.
- Review results.
- Category tags.
Enums can reduce the inconsistency problems caused by models freely generating field values.
5.6 Numeric Constraints
JSON Schema can restrict the range of numbers.
{
"type": "object",
"properties": {
"age": {
"type": "integer",
"minimum": 0,
"maximum": 150
},
"score": {
"type": "number",
"minimum": 0,
"maximum": 100
}
}
}
Common constraints:
| Keyword | Effect |
|---|---|
minimum |
Greater than or equal to the specified value |
maximum |
Less than or equal to the specified value |
exclusiveMinimum |
Greater than the specified value |
exclusiveMaximum |
Less than the specified value |
multipleOf |
Must be a multiple of the specified value |
For example:
{
"type": "number",
"minimum": 0,
"maximum": 100,
"multipleOf": 0.5
}
This means the value must be between 0 and 100 and must be a multiple of 0.5.
5.7 String Constraints
{
"type": "object",
"properties": {
"username": {
"type": "string",
"minLength": 3,
"maxLength": 20
},
"email": {
"type": "string",
"format": "email"
},
"phone": {
"type": "string",
"pattern": "^1[3-9][0-9]{9}$"
}
}
}
Common constraints:
| Keyword | Effect |
|---|---|
minLength |
Minimum string length |
maxLength |
Maximum string length |
pattern |
Regular expression constraint |
format |
Format semantic constraint |
Note that in JSON Schema, string length is calculated based on Unicode characters rather than UTF-8 byte count.
The specific behavior of regular expressions may also be affected by the scope of support in the implementation.
5.8 Array Constraints
items is used to define the structure of array elements.
{
"type": "object",
"properties": {
"skills": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"maxItems": 10,
"uniqueItems": true
}
}
}
Common constraints:
| Keyword | Purpose |
|---|---|
items |
Defines rules for array elements |
minItems |
Minimum number of elements |
maxItems |
Maximum number of elements |
uniqueItems |
Whether elements must be unique |
5.9 Nested Objects
Complex business data usually has a multi-level structure.
For example, user information contains an address:
{
"type": "object",
"properties": {
"name": {
"type": "string"
},
"address": {
"type": "object",
"properties": {
"province": {
"type": "string"
},
"city": {
"type": "string"
},
"detail": {
"type": "string"
}
},
"required": [
"province",
"city"
],
"additionalProperties": false
}
},
"required": [
"name",
"address"
],
"additionalProperties": false
}
JSON Schema can recursively describe complex objects.
5.10 Composition Constraints
JSON Schema also supports composition rules:
| Keyword | Purpose |
|---|---|
allOf |
Must satisfy all Schemas simultaneously |
anyOf |
Must satisfy at least one Schema |
oneOf |
Must satisfy exactly one Schema |
not |
Must not satisfy the specified Schema |
$ref |
References another Schema |
For example:
{
"type": "object",
"properties": {
"payment_method": {
"type": "string",
"enum": [
"card",
"cash"
]
}
},
"required": [
"payment_method"
],
"allOf": [
{
"if": {
"properties": {
"payment_method": {
"const": "card"
}
},
"required": [
"payment_method"
]
},
"then": {
"properties": {
"card_number": {
"type": "string"
}
},
"required": [
"card_number"
]
}
}
]
}
This means that when the payment method is a bank card, a bank card number must be provided.
Note that complex combined constraints are not necessarily fully supported by all structured output interfaces of large models. The standard capabilities of JSON Schema and the Schema subset supported by a model service provider are two different concepts.
6. How Structured Output Works
To understand structured output, we need to distinguish three levels:
- How the model generates tokens.
- How to make the generated result follow the data structure.
- How the application validates and uses the final result.
6.1 Standard Text Generation
Standard large language models usually use autoregressive generation.
Given the input context, the model predicts the next token step by step.
Where:
- x_t: the t-th token.
- x_{<t}: the tokens already generated before.
- P(x_t\mid x_{<t}): the probability distribution of the next token under the current context.
In standard text generation, the model can generate any content that fits its language distribution.
Even if the prompt says “return only JSON,” the model may generate explanatory text, omit fields, or output invalid JSON.
6.2 Constraining the Format Through Prompts
The simplest way is to explicitly require JSON format in the prompt.
For example:
Please extract user information and return only JSON.
Fields:
- name: name, string
- age: age, integer
- email: email, string
Do not output any explanatory text.
This method belongs to format constraints at the prompt level.
It can improve format consistency, but it cannot guarantee that every generation meets the requirements.
6.3 Controlling Generation Through Structural Constraints
Stricter structured output mechanisms can use the Schema as a constraint condition during model generation.
Some model services build a constrained decoding process based on the Schema, restricting the model to only choose tokens or token sequences that meet the current structural requirements.
The abstract process is as follows:
flowchart TB
A["User request"] --> B["Application provides JSON Schema"]
B --> C["Model service parses Schema"]
C --> D["Build generation constraints"]
D --> E["Model computes next token"]
E --> F["Constrained decoding"]
F --> G["Generate structured response"]
G --> H["Application parses and validates"]
H --> I["Business processing"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#EAF3EC,stroke:#83A995,color:#315D49
style D fill:#EAF3EC,stroke:#83A995,color:#315D49
style E fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style F fill:#EAF3EC,stroke:#83A995,color:#315D49
style G fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style H fill:#EAF3EC,stroke:#83A995,color:#315D49
style I fill:#DCEFE5,stroke:#4F8B70,color:#24563F
The constrained decoding here is an abstract description of common implementation mechanisms, and it does not mean that all model service providers use exactly the same internal algorithms.
Some implementations may use grammar constraints, finite state machines, Token Masking, or other decoding control techniques.
6.4 The Basic Idea of Token Masking
Suppose the model currently needs to generate a field of a JSON object.
The Schema requires:
{
"type": "object",
"properties": {
"status": {
"type": "string",
"enum": [
"success",
"failed"
]
}
},
"required": [
"status"
]
}
At the appropriate generation stage, the constrained decoder restricts candidate outputs that do not conform to the current grammar or Schema state.
Abstract representation:
Where:
- P(x_i): the model's original token probability.
- V_{\text{allowed}}: the set of candidate tokens allowed in the current state.
- P'(x_i): the probability distribution after constraints.
Practical systems usually also need to renormalize over the allowed probabilities.
Note that constrained decoding is not simply letting the model first generate arbitrary text and then repairing the JSON with a program; it restricts the selectable outputs during generation.
Different implementations may have different constraint granularity. Some constraints apply at the syntax level, while others can further restrict field values.
6.5 Structural Validity Is Not Semantic Correctness
This is the most important boundary of structured output.
Suppose the Schema requires:
{
"type": "object",
"properties": {
"age": {
"type": "integer",
"minimum": 0,
"maximum": 150
}
},
"required": [
"age"
]
}
The model returns:
{
"age": 35
}
This data is completely valid structurally.
But if the user's actual age is 28, the output is still wrong.
Therefore, it is necessary to distinguish:
| Layer | What is checked | Typical method |
|---|---|---|
| Syntax validity | Whether it is valid JSON | JSON Parser |
| Structural validity | Whether it satisfies the Schema | JSON Schema Validator |
| Semantic correctness | Whether field values match the facts | Evidence verification, model evaluation |
| Business correctness | Whether it complies with business rules | Business code and database validation |
Structured output mainly solves the first two layers; the latter two still require additional mechanisms.
7. Structured Output, JSON Mode, and Function Calling
In LLM APIs, you often encounter three related concepts:
- JSON Mode
- Structured Outputs
- Function Calling
They may all involve JSON, but their design goals differ.
7.1 JSON Mode
JSON Mode is usually used to require the model to return content in valid JSON format.
For example:
{
"name": "张三",
"age": 28
}
It mainly addresses JSON syntax formatting issues.
But JSON Mode does not necessarily guarantee:
- That the specified fields must be returned.
- That field types fully meet the requirements.
- That no extra fields will appear.
- That enum values conform to business definitions.
- That nested structures satisfy the Schema.
The specific capabilities depend on the provider's API interface.
7.2 Structured Outputs
Structured Outputs usually means that model responses need to follow a specified structured Schema.
For example:
{
"type": "object",
"properties": {
"name": {
"type": "string"
},
"age": {
"type": "integer"
}
},
"required": [
"name",
"age"
],
"additionalProperties": false
}
Compared with merely requiring JSON output, structured output can further constrain fields, types, and structure.
But note:
- Different providers support different subsets of Schema.
- Some Schema keywords may not be supported.
- Strict mode may require all properties to appear in
required. - Some services may require extra fields to be disallowed.
- Refusal responses, truncated responses, and abnormal termination still need to be handled separately.
7.3 Function Calling
Function Calling, also called Tool Calling, lets the model generate tool invocation requests according to a predefined tool parameter structure.
For example, define a weather query tool:
{
"type": "function",
"function": {
"name": "get_weather",
"description": "查询指定城市的天气",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "城市名称"
}
},
"required": [
"city"
],
"additionalProperties": false
}
}
}
The model might generate:
{
"city": "上海"
}
This parameter can be used to call the weather tool.
But the focus of Function Calling is tool calling, not merely returning data.
After the model generates a tool call request, the application usually also needs to execute the tool and return the result to the model.
7.4 Differences Among the Three
| Comparison dimension | JSON Mode | Structured Outputs | Function Calling |
|---|---|---|---|
| Core goal | Return valid JSON | Follow the specified data structure | Initiate tool calls |
| Main constraint | JSON syntax | Schema structure | Tool parameter structure |
| Whether tools are needed | Not required | Not required | Usually required |
| Whether business fields are described | Not necessarily | Yes | Yes |
| Whether external execution is triggered | No | No | Usually executed by the application |
| Typical scenarios | JSON data return | Information extraction, classification, data generation | Queries, search, operations |
The three are not completely mutually exclusive technologies.
For example, an Agent can request tool execution through Function Calling while requiring the tool parameters to conform to a Schema.
8. How to Use JSON Schema to Build Structured Outputs for Large Models?
The following uses Python to demonstrate a complete engineering implementation.
8.1 Define the Data Structure
Suppose you need to extract user information from a piece of text.
Target data:
{
"name": "李明",
"age": 30,
"email": "liming@example.com",
"skills": [
"Python",
"Go"
]
}
Corresponding JSON Schema:
user_schema = {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "用户姓名"
},
"age": {
"type": "integer",
"minimum": 0,
"maximum": 150
},
"email": {
"type": "string",
"format": "email"
},
"skills": {
"type": "array",
"items": {
"type": "string"
}
}
},
"required": [
"name",
"age",
"email",
"skills"
],
"additionalProperties": False
}
This step defines the data contract.
Both the application and the model service can use this contract to understand the expected data structure.
8.2 Use Pydantic to Define the Structure
In a Python project, you can use Pydantic to define data structures through type annotations.
from pydantic import BaseModel, Field, EmailStr
class UserInfo(BaseModel):
name: str = Field(description="user name")
age: int = Field(ge=0, le=150)
email: EmailStr
skills: list[str]