'Complete Guide: Integrating OpenAI Agents SDK with Ollama for Local AI Agent [post] deterministic


Complete Guide: Integrating OpenAI Agents SDK with Ollama

This comprehensive guide demonstrates how to integrate the official OpenAI Agents SDK with Ollama to create AI agents that run entirely on local infrastructure. By the end, you'll understand both the theoretical foundations and practical implementation of locally-hosted AI agents.

Table of Contents

1. [Introduction](#introduction) 2. [Understanding the Components](#understanding-the-components) 3. [Setting Up Your Environment](#setting-up-your-environment) 4. [Integrating Ollama with OpenAI Agents SDK](#integrating-ollama-with-openai-agents-sdk) 5. [Building a Document Analysis Agent](#building-a-document-analysis-agent) 6. [Adding Document Memory](#adding-document-memory) 7. [Putting It All Together](#putting-it-all-together) 8. [Troubleshooting](#troubleshooting) 9. [Conclusion](#conclusion)

Introduction

The OpenAI Agents SDK is a powerful framework for building agent-based AI systems that can solve complex tasks through planning and tool use. By integrating it with Ollama, we can run these agents locally, improving privacy, reducing latency, and eliminating API costs.

Understanding the Components

What is the OpenAI Agents SDK?

The OpenAI Agents SDK (agents) is a framework that simplifies the development of AI agents. It provides:

• A structured approach for defining agent behaviors
• Built-in support for tool usage and planning
• Session management for multi-turn conversations
• Memory and state persistence

At its core, this SDK formalizes the agent pattern that emerged from the broader LLM community, giving developers a standard way to implement agents that can plan, reason, and execute complex tasks.

What is Ollama?

Ollama is an open-source framework for running large language models (LLMs) locally. Key features include:

• Easy installation and model management
• Compatible API endpoints that mimic OpenAI's API structure
• Support for many open-source models (Llama, Mistral, etc.)
• Custom model creation and fine-tuning

Why Integrate Them?

Integration provides several benefits:

1. **Data Privacy**: All data stays on your local machine 2. **Cost Efficiency**: No pay-per-token API costs 3. **Customization**: Fine-tune models for specific use cases 4. **Network Independence**: Agents function without internet access 5. **Reduced Latency**: Eliminate network roundtrips

Setting Up Your Environment

Step 1: Install Ollama

First, install Ollama following the instructions for your operating system:

#### For macOS and Linux:

bash curl -fsSL https://ollama.ai/install.sh | sh

#### For Windows:

Download the installer from [Ollama's website](https://ollama.com/download).

Step 2: Download a Model

Pull a capable model that will power your agent. For this guide, we'll use Mistral:

bash ollama pull mistral

Verify that Ollama is working by running:

bash ollama run mistral "Hello, are you running correctly?"

You should see a response generated by the model.

Step 3: Install the OpenAI Agents SDK

Clone the repository and install the package:

bash git clone https://github.com/openai/openai-agents-python.git cd openai-agents-python pip install -e .

This installs the package in development mode, allowing you to modify the code if needed.

Step 4: Set Up Required Dependencies

Install additional dependencies:

bash pip install requests python-dotenv pydantic

Integrating Ollama with OpenAI Agents SDK

The OpenAI Agents SDK uses the OpenAI Python client underneath. We need to create a custom client that directs requests to Ollama instead of OpenAI's servers.

Step 1: Create a Custom Client

Create a file named ollama_client.py:

```python import os from openai import OpenAI

class OllamaClient(OpenAI): """Custom OpenAI client that routes requests to Ollama."""

def __init__(self, model_name="mistral", **kwargs): # Configure to use Ollama's endpoint kwargs["base_url"] = "http://localhost:11434/v1"

Ollama doesn't require an API key but the client expects one kwargs["api_key"] = "ollama-placeholder-key"

super().__init__(**kwargs) self.model_name = model_name # Check if the model exists print(f"Using Ollama model: {model_name}")

def create_completion(self, *args, **kwargs): # Override model name if not explicitly provided if "model" not in kwargs: kwargs["model"] = self.model_name

return super().create_completion(*args, **kwargs)

def create_chat_completion(self, *args, **kwargs): # Override model name if not explicitly provided if "model" not in kwargs: kwargs["model"] = self.model_name

return super().create_chat_completion(*args, **kwargs) # These methods are needed for compatibility with agents library def completion(self, prompt, **kwargs): if "model" not in kwargs: kwargs["model"] = self.model_name return self.completions.create(prompt=prompt, **kwargs) def chat_completion(self, messages, **kwargs): if "model" not in kwargs: kwargs["model"] = self.model_name return self.chat.completions.create(messages=messages, **kwargs) ```

Step 2: Create an Adapter for OpenAI Agents SDK

Now we'll create an adapter that makes the OpenAI Agents SDK compatible with our Ollama client. Create a file named agent_adapter.py:

```python from ollama_client import OllamaClient from openai.types.chat import ChatCompletion, ChatCompletionMessage import agents.agent as agent_module from agents.agent import Agent from agents.run import Runner, RunConfig from agents.models import _openai_shared import json import logging

Configure logging logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s') logger = logging.getLogger(__name__)

Set placeholder OpenAI API key to avoid initialization errors _openai_shared.set_default_openai_key("placeholder-key")

Store original init for Agent class original_init = Agent.__init__

def patched_init(self, *args, **kwargs): """Replace the model with OllamaClient if not provided.""" if "model" not in kwargs: kwargs["model"] = OllamaClient(model_name="mistral") original_init(self, *args, **kwargs)

Apply the patched init Agent.__init__ = patched_init

Class for a structured tool call class ToolCall: def __init__(self, name, inputs=None): self.name = name self.inputs = inputs or {}

Define a response class that matches what main.py expects class AgentResponse: def __init__(self, result): # Extract the message from the final output if hasattr(result, 'final_output'): if isinstance(result.final_output, str): self.message = result.final_output else: self.message = str(result.final_output) else: self.message = "I'm sorry, I couldn't process that request." # Get conversation ID if available self.conversation_id = getattr(result, 'conversation_id', None) # Initialize tool_calls self.tool_calls = [] # Extract tool calls from raw_responses if hasattr(result, 'raw_responses'): for response in result.raw_responses: try: if hasattr(response, 'output') and hasattr(response.output, 'tool_calls'): for tool_call in response.output.tool_calls: # Handle the case where tool_call is a dict if isinstance(tool_call, dict): name = tool_call.get('name', 'unknown_tool') inputs = tool_call.get('inputs', {}) self.tool_calls.append(ToolC