Structured output and JSON mode without the model going off the rails

LEARN · LLMS & TRANSFORMERS

Modern LLM applications often fail at a surprisingly mundane boundary: the model says something intelligent, but your program cannot safely consume it.

You ask for JSON and get a Markdown fence. You ask for three fields and get four. You expect "priority": "high" and receive "priority": "urgent". Or everything works for weeks until one long response hits an output limit and the JSON ends halfway through a string.

The fix is to separate three ideas that are often incorrectly treated as interchangeable:

Prompting for JSON asks the model to behave.
JSON mode constrains the response to JSON, but not to your application schema.
Structured Outputs constrains the response to a JSON Schema and is the preferred choice when your application depends on exact fields, enums, arrays, and types.

OpenAI’s current documentation recommends Structured Outputs over JSON mode when possible. In the Responses API, structured data is configured through text.format, and the Python SDK can also generate and parse the schema automatically from Pydantic models. For new projects, the current docs recommend gpt-5.6.

This distinction turns structured generation from “please return something my parser likes” into an actual API contract.

The reliability ladder: prompt, JSON mode, schema

Consider an application that classifies incoming support tickets.

You need an object like this:

{
  "category": "bug",
  "priority": "high",
  "summary": "Checkout freezes after applying a coupon",
  "product_area": "checkout",
  "confidence": 0.96
}

A naïve prompt might say:

Classify this ticket. Return JSON with category, priority, summary,
product_area, and confidence. Do not include Markdown.

That is useful instruction, but it is still only instruction. A probabilistic model can decide to produce:

{
  "category": "technical_issue",
  "priority": "urgent",
  "summary": "Checkout freezes after applying a coupon",
  "product_area": "checkout",
  "confidence": "very high"
}

It is JSON. It is also useless to code expecting a fixed enum and a numeric confidence.

JSON mode improves one part of the problem: it is designed to produce valid JSON. It does not guarantee that the output follows a particular schema. OpenAI explicitly documents this distinction and recommends schema-constrained Structured Outputs when your application needs a specific structure.

Structured Outputs moves the contract into the generation process itself. Instead of merely telling the model that priority should have three possible values, you define those values in the schema.

Conceptually, instead of:

Please remember that priority must be low, medium, or high.

you give the API a constraint equivalent to:

{
  "type": "string",
  "enum": ["low", "medium", "high"]
}

That is the central idea of constrained generation: invalid structural choices are not merely discouraged by prose instructions; the requested output format restricts what constitutes a valid response. The current OpenAI documentation describes Structured Outputs as ensuring that model responses adhere to the supplied JSON Schema.

Set up a minimal Python project

Create a virtual environment and install the current OpenAI Python package plus Pydantic:

python -m venv .venv

On macOS or Linux:

source .venv/bin/activate

On Windows PowerShell:

.venv\Scripts\Activate.ps1

Install the dependencies:

python -m pip install --upgrade openai pydantic

Export your API key before running the examples.

On macOS or Linux:

export OPENAI_API_KEY="your-api-key"

On Windows PowerShell:

$env:OPENAI_API_KEY="your-api-key"

The Python SDK reads the API key from the environment when you construct OpenAI().

The easiest current pattern: Pydantic plus `responses.parse()`

For Python applications, a particularly clean approach is to make your application type the source of truth.

The current OpenAI SDK supports passing a Pydantic model through client.responses.parse(..., text_format=...), then exposing the validated result through response.output_parsed. OpenAI specifically recommends native Pydantic or Zod support as a way to prevent your application types and JSON Schema from drifting apart.

Create triage.py:

from typing import Literal

from openai import OpenAI
from pydantic import BaseModel, Field


client = OpenAI()


class TicketTriage(BaseModel):
    category: Literal["bug", "feature_request", "billing", "other"]
    priority: Literal["low", "medium", "high"]
    summary: str
    product_area: str | None
    confidence: float = Field(ge=0.0, le=1.0)


ticket_text = """
After I apply the SUMMER20 coupon, the checkout spinner never stops.
Refreshing removes the coupon and lets me continue.
This happened three times in Chrome today.
"""

response = client.responses.parse(
    model="gpt-5.6",
    input=[
        {
            "role": "system",
            "content": (
                "Classify customer support tickets. "
                "Use product_area=null when the area cannot be determined. "
                "Confidence must reflect how certain the classification is."
            ),
        },
        {
            "role": "user",
            "content": ticket_text,
        },
    ],
    text_format=TicketTriage,
)

ticket = response.output_parsed

if ticket is None:
    raise RuntimeError("No parsed ticket was returned.")

print(ticket.model_dump_json(indent=2))

Run it:

python triage.py

A plausible result is:

{
  "category": "bug",
  "priority": "high",
  "summary": "Checkout hangs after applying the SUMMER20 coupon",
  "product_area": "checkout",
  "confidence": 0.96
}

The exact wording and confidence value can vary because schema enforcement does not make the model semantically deterministic. It constrains structure, not truth. OpenAI’s documentation explicitly warns that Structured Outputs can still contain mistakes in field values even when the structure is valid.

That distinction matters enormously.

A schema can guarantee that:

confidence is a number.
priority belongs to your allowed enum.
required properties exist.
arbitrary surprise properties are excluded.

It cannot guarantee that the model chose the correct priority.

You still need evaluation, representative test cases, and domain logic.

What the Pydantic helper is saving you from

Without the SDK helper, you need to maintain a JSON Schema manually.

That is sometimes desirable—for example, if schemas are stored independently from your Python code—but it creates another artifact that can drift.

Here is the same basic contract written explicitly:

{
  "type": "object",
  "properties": {
    "category": {
      "type": "string",
      "enum": ["bug", "feature_request", "billing", "other"]
    },
    "priority": {
      "type": "string",
      "enum": ["low", "medium", "high"]
    },
    "summary": {
      "type": "string"
    },
    "product_area": {
      "type": ["string", "null"]
    },
    "confidence": {
      "type": "number",
      "minimum": 0,
      "maximum": 1
    }
  },
  "required": [
    "category",
    "priority",
    "summary",
    "product_area",
    "confidence"
  ],
  "additionalProperties": false
}

Notice two details that frequently surprise developers.

First, Structured Outputs requires object fields to be included in required. To represent an optional semantic value, use a nullable type such as "type": ["string", "null"]. Second, objects used with Structured Outputs require additionalProperties: false.

In other words, “optional” usually means:

The key exists, but its value may be null.

Not:

The model may choose whether the key exists.

That produces much nicer downstream code because your object shape remains predictable.

Using JSON Schema directly with the Responses API

Here is the same task without Pydantic.

Create manual_schema.py:

import json

from openai import OpenAI


client = OpenAI()

schema = {
    "type": "object",
    "properties": {
        "category": {
            "type": "string",
            "enum": ["bug", "feature_request", "billing", "other"],
        },
        "priority": {
            "type": "string",
            "enum": ["low", "medium", "high"],
        },
        "summary": {
            "type": "string",
        },
        "product_area": {
            "type": ["string", "null"],
        },
        "confidence": {
            "type": "number",
            "minimum": 0,
            "maximum": 1,
        },
    },
    "required": [
        "category",
        "priority",
        "summary",
        "product_area",
        "confidence",
    ],
    "additionalProperties": False,
}

response = client.responses.create(
    model="gpt-5.6",
    input=(
        "Classify this support ticket:\n\n"
        "After I apply the SUMMER20 coupon, checkout freezes indefinitely."
    ),
    text={
        "format": {
            "type": "json_schema",
            "name": "ticket_triage",
            "strict": True,
            "schema": schema,
        }
    },
    max_output_tokens=500,
)

if (
    response.status == "incomplete"
    and response.incomplete_details is not None
):
    raise RuntimeError(
        f"Incomplete response: {response.incomplete_details.reason}"
    )

for item in response.output:
    if item.type != "message":
        continue

    for content in item.content:
        if content.type == "refusal":
            raise RuntimeError(f"Model refused request: {content.refusal}")

data = json.loads(response.output_text)

print(json.dumps(data, indent=2))

The current Responses API shape puts the structured output configuration under text.format, with type: "json_schema", a schema name, the schema itself, and strict: true. The official guide also demonstrates checking response.status == "incomplete" and inspecting response.incomplete_details.reason when output limits prevent completion.

The manual approach is slightly noisier but teaches an important architectural lesson:

Your schema belongs at the boundary between probabilistic generation and deterministic software.

Everything downstream should be allowed to assume that boundary has been checked.

Supported JSON Schema is deliberately a subset

Do not assume every keyword from the full JSON Schema ecosystem is supported.

The current Structured Outputs documentation supports core types including strings, numbers, booleans, integers, objects, arrays, enums, and nested anyOf. It also supports several useful constraints such as string patterns and formats, numeric bounds, and array size constraints.

For example:

{
  "type": "object",
  "properties": {
    "order_id": {
      "type": "string",
      "pattern": "^ORD-[0-9]{6}$"
    },
    "email": {
      "type": "string",
      "format": "email"
    },
    "items": {
      "type": "array",
      "minItems": 1,
      "maxItems": 20,
      "items": {
        "type": "string"
      }
    }
  },
  "required": ["order_id", "email", "items"],
  "additionalProperties": false
}

The supported subset has boundaries. The current documentation says the root must be an object rather than a top-level anyOf, and keywords such as allOf, not, dependentRequired, dependentSchemas, if, then, and else are not supported.

That means schemas should be designed for generation rather than copied blindly from a large enterprise validation system.

A good model-facing schema is usually:

shallow enough to understand quickly,
explicit about enums,
strict about additional properties,
descriptive about ambiguous fields,
nullable where absence has real meaning,
focused on one job.

JSON mode still has a legitimate use

Structured Outputs should be your default when code requires a known object shape, but JSON mode remains useful when you merely need syntactically valid JSON and the structure is genuinely flexible.

In the Responses API, JSON mode is enabled with:

{
  "format": {
    "type": "json_object"
  }
}

Here is a runnable example:

import json

from openai import OpenAI


client = OpenAI()

response = client.responses.create(
    model="gpt-5.6",
    instructions=(
        "Return JSON only. Produce a JSON object summarizing the supplied text."
    ),
    input=(
        "A small online shop processed 183 orders Monday, "
        "207 Tuesday, and 195 Wednesday."
    ),
    text={
        "format": {
            "type": "json_object",
        }
    },
    max_output_tokens=300,
)

if response.status != "completed":
    raise RuntimeError(f"Generation did not complete: {response.status}")

data = json.loads(response.output_text)
print(json.dumps(data, indent=2))

The word JSON in the instruction is not decorative. OpenAI’s current documentation says JSON mode requires the conversation to explicitly instruct the model to produce JSON; otherwise generation can degenerate into whitespace until the token limit, and the API checks for a JSON instruction in the context.

But notice what this example does not promise.

The model could produce:

{
  "monday": 183,
  "tuesday": 207,
  "wednesday": 195
}

or:

{
  "orders": {
    "Mon": 183,
    "Tue": 207,
    "Wed": 195
  },
  "total": 585
}

or:

{
  "summary": "Order volume was fairly stable.",
  "daily_orders": [183, 207, 195]
}

All three may be valid JSON.

If your code requires one exact structure, JSON mode is the wrong contract.

Cherry on the cake: the “successful” request that quietly ends halfway through JSON

Here is one of the nastiest production failure modes because it is easy to misdiagnose.

A request can reach its output limit before generation finishes. The network request itself may have succeeded, but the generation is incomplete.

If your application checks only whether the API call raised an exception, you can accidentally treat truncation as success.

The current Responses API exposes this explicitly with response.status == "incomplete" and an incomplete_details.reason, including the max_output_tokens case. OpenAI’s JSON-mode documentation separately warns that applications must handle edge cases where output is not a complete JSON object.

You can deliberately reproduce the problem by setting an absurdly small output budget:

from openai import OpenAI


client = OpenAI()

response = client.responses.create(
    model="gpt-5.6",
    instructions=(
        "Return JSON only. Generate a JSON object containing "
        "20 fictional store departments, each with a name, "
        "description, and five example products."
    ),
    input="Generate the catalog now.",
    text={
        "format": {
            "type": "json_object",
        }
    },
    max_output_tokens=20,
)

print("status:", response.status)
print("incomplete_details:", response.incomplete_details)
print("output_text:", repr(response.output_text))

Depending on the exact generation, the output text may look like the beginning of an otherwise ordinary object:

'{"departments":[{"name":"Home","description":"Products for'

The dangerous code is therefore:

response = client.responses.create(
    model="gpt-5.6",
    input="Return a large JSON response.",
    instructions="Return JSON only.",
    text={"format": {"type": "json_object"}},
    max_output_tokens=20,
)

save_to_database(response.output_text)

The safer pattern is:

import json


if response.status != "completed":
    reason = (
        response.incomplete_details.reason
        if response.incomplete_details is not None
        else "unknown"
    )
    raise RuntimeError(f"Generation incomplete: {reason}")

data = json.loads(response.output_text)
save_to_database(data)

This is why “the HTTP request worked” and “the model completed its contract” must be treated as two different conditions.

The equivalent lesson applies to older Chat Completions code as well: the documentation tells applications to inspect the completion reason and explicitly handle length-limited output rather than assuming the payload is complete.

Structured output does not mean “never check errors”

Schema enforcement eliminates a large class of formatting failures. It does not eliminate lifecycle failures.

Your production code still needs to distinguish at least:

completed output,
incomplete generation,
refusal,
transport or API errors,
logically incorrect but schema-valid data.

Structured Outputs makes refusals programmatically distinguishable, and the official examples inspect response content for refusal objects rather than blindly decoding every result as application data.

A useful application boundary looks like this:

import json


def extract_completed_json(response):
    if response.status != "completed":
        reason = (
            response.incomplete_details.reason
            if response.incomplete_details is not None
            else response.status
        )
        raise RuntimeError(f"Response not completed: {reason}")

    for item in response.output:
        if item.type != "message":
            continue

        for content in item.content:
            if content.type == "refusal":
                raise RuntimeError(f"Request refused: {content.refusal}")

    if not response.output_text:
        raise RuntimeError("Completed response contained no output text.")

    return json.loads(response.output_text)

The model output should enter the rest of your application only after this boundary.

Schema correctness and semantic correctness are separate tests

Suppose your schema is:

{
  "type": "object",
  "properties": {
    "sentiment": {
      "type": "string",
      "enum": ["positive", "neutral", "negative"]
    },
    "confidence": {
      "type": "number",
      "minimum": 0,
      "maximum": 1
    }
  },
  "required": ["sentiment", "confidence"],
  "additionalProperties": false
}

And the model returns:

{
  "sentiment": "positive",
  "confidence": 0.99
}

Structurally, this can be perfect.

If the source text was “This update deleted all my saved projects,” the classification is probably wrong.

The schema solved serialization. It did not solve reasoning.

This leads to a useful testing model:

Layer 1: structural tests

Check that:

parsing succeeds,
enums remain inside their permitted sets,
nullable fields behave correctly,
unexpected keys cannot appear,
incomplete responses are rejected.

Layer 2: semantic evals

Check whether:

categories are actually correct,
extracted numbers match source material,
summaries preserve important information,
confidence behaves sensibly,
edge cases produce appropriate nullable or fallback states.

OpenAI’s current structured-output guidance recommends using evals and representative examples to improve quality even after the output structure itself has been constrained.

Do not make the model invent data just to satisfy the schema

Strict schemas create another subtle trap.

Imagine this structure:

{
  "type": "object",
  "properties": {
    "order_number": {
      "type": "string"
    },
    "delivery_date": {
      "type": "string",
      "format": "date"
    }
  },
  "required": ["order_number", "delivery_date"],
  "additionalProperties": false
}

Then the user says:

My package has not arrived yet.

There is no order number or delivery date in the input.

If your schema leaves the model no representation for “unknown,” you have created pressure to fabricate values.

OpenAI warns about exactly this class of problem: because the model tries to adhere to the provided schema, unrelated or insufficient user input can result in hallucinated values unless your application tells the model how to represent cases that cannot be answered.

Design the uncertainty into the schema instead:

{
  "type": "object",
  "properties": {
    "order_number": {
      "type": ["string", "null"]
    },
    "delivery_date": {
      "type": ["string", "null"],
      "format": "date"
    },
    "status": {
      "type": "string",
      "enum": ["complete", "missing_information"]
    }
  },
  "required": ["order_number", "delivery_date", "status"],
  "additionalProperties": false
}

And give the model explicit instructions:

Never invent missing order information.
When the source does not contain a value, return null and set
status to "missing_information".

This combination—schema plus behavior instructions—is considerably stronger than either alone.

Use tool calling when the model needs to act, not merely answer

There is another architectural distinction worth getting right.

Structured response formatting is appropriate when you want the answer itself to have a predictable shape.

Tool calling is appropriate when the model needs to request an action from your application: query inventory, fetch an order, schedule something, update a record, or invoke other deterministic functionality.

OpenAI’s current documentation draws exactly this line: use function calling when connecting the model to functions, tools, or external data; use structured response formatting when you need the model’s final answer itself to conform to a schema.

Suppose an e-commerce assistant may look up an order.

A strict function definition for the Responses API can look like this:

tools = [
    {
        "type": "function",
        "name": "lookup_order",
        "description": "Look up an order by its order ID.",
        "parameters": {
            "type": "object",
            "properties": {
                "order_id": {
                    "type": "string",
                    "description": "Order ID such as ORD-481920.",
                }
            },
            "required": ["order_id"],
            "additionalProperties": False,
        },
        "strict": True,
    }
]

Then make the request:

from openai import OpenAI


client = OpenAI()

tools = [
    {
        "type": "function",
        "name": "lookup_order",
        "description": "Look up an order by its order ID.",
        "parameters": {
            "type": "object",
            "properties": {
                "order_id": {
                    "type": "string",
                    "description": "Order ID such as ORD-481920.",
                }
            },
            "required": ["order_id"],
            "additionalProperties": False,
        },
        "strict": True,
    }
]

response = client.responses.create(
    model="gpt-5.6",
    input="Where is order ORD-481920?",
    tools=tools,
)

for item in response.output:
    if item.type == "function_call":
        print(item.name)
        print(item.arguments)

For function tools, current OpenAI guidance recommends strict: true. Strict mode requires all declared properties to be required and objects to set additionalProperties: false; nullable types can represent optional values.

The model does not execute your Python function by itself. It produces a function-call request, your program executes the relevant logic, and the tool result is then returned to the model as a function_call_output. That multi-step tool loop is the documented Responses API pattern.

A compact tool-execution loop

Here is a neutral runnable example with fake local order data:

import json

from openai import OpenAI


client = OpenAI()

orders = {
    "ORD-481920": {
        "status": "shipped",
        "carrier": "ParcelCo",
    },
    "ORD-120384": {
        "status": "processing",
        "carrier": None,
    },
}

tools = [
    {
        "type": "function",
        "name": "lookup_order",
        "description": "Look up the status of an order.",
        "parameters": {
            "type": "object",
            "properties": {
                "order_id": {
                    "type": "string",
                }
            },
            "required": ["order_id"],
            "additionalProperties": False,
        },
        "strict": True,
    }
]


def lookup_order(order_id):
    return orders.get(
        order_id,
        {
            "status": "not_found",
            "carrier": None,
        },
    )


input_items = [
    {
        "role": "user",
        "content": "What is happening with ORD-481920?",
    }
]

response = client.responses.create(
    model="gpt-5.6",
    tools=tools,
    input=input_items,
)

input_items += response.output

for item in response.output:
    if item.type != "function_call":
        continue

    if item.name != "lookup_order":
        continue

    arguments = json.loads(item.arguments)
    result = lookup_order(arguments["order_id"])

    input_items.append(
        {
            "type": "function_call_output",
            "call_id": item.call_id,
            "output": json.dumps(result),
        }
    )

final_response = client.responses.create(
    model="gpt-5.6",
    tools=tools,
    input=input_items,
)

print(final_response.output_text)

There are now two contracts in play:

The tool’s argument schema controls how the model asks your application to perform work.
A structured response schema can independently control the shape of the final answer if your UI also requires machine-readable output.

Do not force everything through a single giant schema. Separate action boundaries from presentation boundaries.

Production pattern: typed output at one boundary, tools at another

A maintainable architecture often looks like this:

User request
    |
    v
Model
    |
    +--> strict function call
    |        |
    |        v
    |    application code
    |        |
    |        v
    |    function result
    |        |
    +--------+
    |
    v
structured final response
    |
    v
validated application object
    |
    v
UI / API / database

Each boundary has one responsibility.

Tool schemas describe actions.

Response schemas describe outputs.

Application validation describes what your software accepts.

Business rules describe what is actually allowed.

That separation is more robust than asking one prompt to enforce everything.

Five schema design rules that prevent most failures

1. Prefer enums over prose conventions

Weak:

{
  "priority": {
    "type": "string",
    "description": "Use low, medium, or high."
  }
}

Better:

{
  "priority": {
    "type": "string",
    "enum": ["low", "medium", "high"]
  }
}

If the domain has a finite set, encode it.

2. Represent unknown values explicitly

Do not make the model guess just because every key is required.

Use nullable fields:

{
  "product_area": {
    "type": ["string", "null"]
  }
}

Then tell the model when null is appropriate.

3. Keep descriptions semantic

Descriptions should explain what a field means, not repeat its type.

Weak:

{
  "confidence": {
    "type": "number",
    "description": "A number."
  }
}

Better:

{
  "confidence": {
    "type": "number",
    "minimum": 0,
    "maximum": 1,
    "description": "Confidence that the category matches the source ticket."
  }
}

The schema constrains the range; the description explains the judgment.

4. Keep schemas smaller than your internal domain model

Your database may have 140 fields.

Your model probably does not need to generate 140 fields.

Ask it only for values that require model judgment. Populate IDs, timestamps, defaults, authorization data, derived totals, and deterministic metadata in ordinary code.

5. Version the schema like an API

Changing:

{
  "priority": "high"
}

into:

{
  "priority": {
    "level": "high",
    "reason": "Checkout is blocked"
  }
}

is an API change even if no HTTP endpoint changed.

Schema versions should therefore be:

reviewed,
tested,
evaluated,
deployed intentionally,
observed after rollout.

Write tests for failure paths, not only pretty examples

Most tutorials test only the happy path:

assert result.category == "bug"

Production tests should also exercise infrastructure conditions.

For example:

def test_priority_is_restricted(ticket):
    assert ticket.priority in {"low", "medium", "high"}


def test_confidence_range(ticket):
    assert 0.0 <= ticket.confidence <= 1.0


def test_nullable_product_area(ticket):
    assert ticket.product_area is None or isinstance(ticket.product_area, str)

And your integration layer should test incomplete output behavior:

def ensure_completed(response):
    if response.status != "completed":
        reason = (
            response.incomplete_details.reason
            if response.incomplete_details is not None
            else response.status
        )
        raise RuntimeError(f"Response incomplete: {reason}")

You should also maintain a small set of difficult semantic fixtures.

For ticket triage, that might include:

[
  {
    "input": "Can you add keyboard shortcuts to the editor?",
    "expected_category": "feature_request"
  },
  {
    "input": "I was charged twice for one subscription.",
    "expected_category": "billing"
  },
  {
    "input": "The app crashes whenever I import a 20 MB CSV file.",
    "expected_category": "bug"
  },
  {
    "input": "Hello, I like the new dashboard.",
    "expected_category": "other"
  }
]

Those examples contain no sensitive or medical dataset; they are simply deterministic application fixtures that can be run repeatedly as your prompts, schemas, or model selection change.

What to choose in practice

Use prompt-only formatting when the output is primarily for humans and parsing failures do not matter.

Use JSON mode when:

any valid JSON structure is acceptable,
you are interoperating with code that needs JSON syntax,
schema enforcement is unavailable for a particular use case,
you are prepared to validate and retry yourself.

Use Structured Outputs when:

another program consumes the result,
keys must always exist,
enum values must stay inside a known set,
unexpected fields would cause bugs,
your UI depends on predictable nesting,
extraction pipelines feed databases or other APIs.

Use strict tool calling when:

the model needs external information,
the model needs to request an application action,
arguments must obey a known function contract.

The current OpenAI documentation recommends Structured Outputs over JSON mode when supported, recommends strict mode for function calls, and recommends the Responses API for new direct model-generation workflows.

The production checklist

Before shipping a structured-generation feature, verify that you can answer yes to these questions:

Does machine-consumed output use a schema rather than prompt formatting alone?
Are enums represented as actual enums?
Are unknown values explicitly nullable?
Does every strict object reject unspecified properties?
Are incomplete responses checked before parsing or persistence?
Are refusals handled separately from application data?
Can your code tolerate semantically wrong but structurally valid values?
Do you have representative evaluation fixtures?
Are tool arguments separate from final response formatting?
Is the schema generated from application types where practical?
Will schema changes receive the same scrutiny as API changes?

The biggest mindset shift is simple: JSON is a serialization format, not a contract.

JSON mode solves serialization.

A schema solves structure.

Typed parsing connects that structure to your application.

Checks for incomplete generation and refusals protect the lifecycle around it.

Semantic evaluations determine whether the values are actually useful.

Put all five pieces together and the model stops being a text generator that your parser hopes to understand. It becomes a probabilistic component behind a deterministic, typed boundary.

Take one existing prompt in your codebase that says “return JSON,” replace it with a strict schema or a Pydantic responses.parse() model, add an explicit incomplete-response test, and deliberately run it once with a tiny max_output_tokens value. If your application survives that test cleanly, you have already eliminated one of the most common and least obvious failure modes in production LLM integrations.

The reliability ladder: prompt, JSON mode, schema

Set up a minimal Python project

The easiest current pattern: Pydantic plus `responses.parse()`

What the Pydantic helper is saving you from

Using JSON Schema directly with the Responses API

Supported JSON Schema is deliberately a subset

JSON mode still has a legitimate use

Cherry on the cake: the “successful” request that quietly ends halfway through JSON

Structured output does not mean “never check errors”

Schema correctness and semantic correctness are separate tests

Do not make the model invent data just to satisfy the schema

Use tool calling when the model needs to act, not merely answer

A compact tool-execution loop

Production pattern: typed output at one boundary, tools at another

Five schema design rules that prevent most failures

1. Prefer enums over prose conventions

2. Represent unknown values explicitly

3. Keep descriptions semantic

4. Keep schemas smaller than your internal domain model

5. Version the schema like an API

Write tests for failure paths, not only pretty examples

What to choose in practice

The production checklist

Framework Release Watch No11

Vulnerability Watch No42

Your first tool-using agent: function calling done right

Structured output and JSON mode without the model going off the rails

The reliability ladder: prompt, JSON mode, schema

Set up a minimal Python project

The easiest current pattern: Pydantic plus responses.parse()

What the Pydantic helper is saving you from

Using JSON Schema directly with the Responses API

Supported JSON Schema is deliberately a subset

JSON mode still has a legitimate use

Cherry on the cake: the “successful” request that quietly ends halfway through JSON

Structured output does not mean “never check errors”

Schema correctness and semantic correctness are separate tests

Do not make the model invent data just to satisfy the schema

Use tool calling when the model needs to act, not merely answer

A compact tool-execution loop

Production pattern: typed output at one boundary, tools at another

Five schema design rules that prevent most failures

1. Prefer enums over prose conventions

2. Represent unknown values explicitly

3. Keep descriptions semantic

4. Keep schemas smaller than your internal domain model

5. Version the schema like an API

Write tests for failure paths, not only pretty examples

What to choose in practice

The production checklist

Framework Release Watch No11

Vulnerability Watch No42

Your first tool-using agent: function calling done right

The easiest current pattern: Pydantic plus `responses.parse()`