Have you ever received a response from an AI that was brilliant but badly formatted? Or perhaps you’ve built an AI system where the outputs were as unpredictable as British weather? Today, I’ll share how we can bring order to this chaos using structured outputs, demonstrated through a time-parsing AI agent. And as a bonus, I’ll show you a rustic way to give your AI a memory!
The Challenge of Unpredictable AI Outputs
When working with large language models, one of the primary challenges is ensuring consistent and structured output formats. Large language models are designed to generate natural language text with high flexibility, but for practical applications, we often need their responses in specific, structured formats that can be reliably processed by other components of our system.
Enter Structured Outputs
If you are using OpenAI models, the solution to this challenge is to define strict structures for our AI outputs using Pydantic models. Here’s how we do it:
class TimeParsingResponse(BaseModel):
"""Model for the LLM's structured response"""
times: List[str]
is_range: bool
description: str
class AgentResponse(BaseModel):
"""Model for agent responses"""
times: List[str]
is_range: bool
description: str
thread_id: str
Think of these models as templates that the AI must follow. No matter how creative it wants to be, it must provide exactly these fields in exactly these formats, which is rather convenient.
Enforcing the Structure
The real magic happens when we make our API call to the language model. We use OpenAI’s parsing feature to ensure our responses match our defined structure:
response = self.client.beta.chat.completions.parse(
model="gpt-4o-mini",
messages=messages,
response_format=TimeParsingResponse,
)
# Get the parsed response
parsed_response = response.choices[0].message.parsed
This approach has several benefits:
- Type safety throughout our application
- Predictable response formats
- Easy integration with other systems
- Automatic validation of responses
It’s like having a very strict but fair teacher who ensures all homework is submitted in the correct format!
Building the Time Agent
Let’s see how this structured approach works in a complete agent. This is a basic example. Our time-parsing agent understands natural language queries about time and returns structured temporal data:
class TimeAgent:
def process_request(self, prompt: str, thread_id: Optional[str] = None) -> AgentResponse:
# Get current UTC time
current_utc_time = datetime.now(timezone.utc).isoformat()
# Build messages array with system context
messages = [
{
"role": "developer",
"content": f"You are a time parsing assistant that converts natural language time queries into structured data. The current UTC time is: {current_utc_time}."
}
]
try:
# Call OpenAI with structured output
response = self.client.beta.chat.completions.parse(
model="gpt-4o-mini",
messages=messages,
response_format=TimeParsingResponse,
)
# Create final response
return AgentResponse(
times=parsed_response.times,
is_range=parsed_response.is_range,
description=parsed_response.description,
thread_id=current_thread_id
)
except Exception as e:
raise Exception(f"Error processing request: {str(e)}")
A Bonus Feature: Adding Memory
Now, for a bit of fun, let’s look at how we can add memory to our structured system.
We’ve implemented a simple and rustic, but effective I think, file-based memory system. It is good for when you run this locally, not that good if you run these agents on the cloud…
class FileSystemMemoryStore(Generic[T]):
"""A file-system based memory store that can be used by any agent."""
def __init__(self, storage_dir: str, response_model: Type[T]):
self.storage_dir = storage_dir
self.response_model = response_model
if not os.path.exists(storage_dir):
os.makedirs(storage_dir)
The clever bit is that our memory store is generic — it can work with any structured response type we define. It’s like having a filing system that automatically adapts to whatever kind of documents you need to store!
Exposing Our Structured Agent
Finally, we expose our structured agent through a FastAPI server:
@app.get("/metadata")
async def get_metadata():
return {
"role": "time-parsing assistant",
"description": "Converts natural language time queries into structured data",
"response_model": TimeParsingResponse.model_json_schema()
}
@app.post("/agent", response_model=AgentResponse)
async def process_request(request: AgentRequest):
try:
return agent.process_request(request.prompt, request.thread_id)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
Notice how we’re maintaining our commitment to structure right through to the API level. The /metadata endpoint even shares our response model schema, ensuring clients know exactly what to expect.
Conclusion
Structured outputs are like a good hot chocolate — they bring order and comfort to what could otherwise be chaos. By defining clear structures for our AI responses and enforcing them consistently, we can build reliable, predictable AI systems that play nicely with others.
And remember, if you’re ever feeling lost in a sea of unstructured AI outputs, just remember the British approach: keep calm and add structure!
Disclaimer: The perspectives shared here are my own and do not necessarily represent those of my employer. I use GenAI as a tool to help me compose and structure my articles.