Structured Outputs

Structured outputs let you request JSON-conforming responses from any LLM provider. Martha supports two modes:

• JSON Schema — constrained decoding guaranteeing the response matches your schema
• JSON Object — best-effort JSON output (no schema enforcement)

Configuring on an Agent

In the admin UI, open an agent's LLM config and set the Output Format:

Setting	Effect
Text (default)	Free-text responses
JSON Schema (strict)	Constrained decoding — LLM returns JSON matching your schema exactly

When selecting JSON Schema, provide a schema in the textarea:

{
  "type": "object",
  "properties": {
    "answer": { "type": "string" },
    "confidence": { "type": "number" },
    "sources": {
      "type": "array",
      "items": { "type": "string" }
    }
  },
  "required": ["answer", "confidence", "sources"],
  "additionalProperties": false
}

The agent's final response will always conform to this schema.

!!! note "Tool calls are unaffected" Structured outputs only apply to the agent's final text response. During tool-calling turns, the LLM is free to emit tool calls as normal. The schema constraint applies only after all tools complete.

Configuring via API

Set response_format in the agent's llm_config:

{
  "provider": "claude",
  "model": "claude-sonnet-4-5-20250929",
  "temperature": 0.7,
  "max_tokens": 4096,
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "agent_output",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": {
          "answer": { "type": "string" },
          "confidence": { "type": "number" }
        },
        "required": ["answer", "confidence"],
        "additionalProperties": false
      }
    }
  }
}

Provider Support

All three LLM providers are supported:

Provider	How it works
Claude (Anthropic)	Translated to Claude's native output_config parameter
OpenAI	Passed through as response_format (native support)
LiteLLM (any provider)	LiteLLM handles translation to each provider's format

!!! tip "Use LiteLLM for maximum compatibility" LiteLLM automatically translates structured output requests for 100+ providers. If you're unsure which provider your deployment will use, configure via LiteLLM.

Workflow Nodes

LLM Router

The LLM router node automatically uses structured outputs for classification. When you configure conditions (e.g., "billing", "technical", "sales"), the router builds a JSON Schema with an enum constraint, ensuring the LLM returns exactly one of the configured categories.

No manual configuration needed — this happens automatically.

LLM Node

In the workflow builder, select an LLM node and scroll to Output Format in the config panel. Choose JSON Object or JSON Schema, same as the agent-level config.

Via API, set response_format inside the node's llm_config:

{
  "llm_config": {
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "extraction",
        "strict": true,
        "schema": { "type": "object", "properties": { "key": { "type": "string" } }, "required": ["key"], "additionalProperties": false }
      }
    }
  }
}

Agent Loop Node

In the workflow builder, expand Override agent settings for this step on an agent loop node to find the Output Format selector. This overrides the agent's default format for this workflow step only.

Via API, set response_format in the node's llm_config override, same format as agent-level config.

Limitations

• Streaming: JSON arrives incrementally during streaming. The complete response is only available at the end. No automatic retry on parse failure during streaming.
• Model support: Older models (GPT-3.5, Claude 3) may not support json_schema mode. LiteLLM falls back gracefully.
• Schema complexity: Very complex schemas (recursive, deeply nested) may be rejected by some providers.
• Token limits: Large schemas with low max_tokens settings can cause truncation.