'Custom AI Agent Framework: Next.js & Ollama Integration Guide' [post] deterministic


Building a Custom AI Agent Framework with Next.js and Ollama

In today's rapidly evolving AI landscape, agent-based systems have emerged as powerful tools for task automation and complex problem-solving. This blog post will guide you through creating a sophisticated Next.js application with a custom AI agent framework powered by Ollama, an open-source local LLM runner.

What We're Building

We'll develop an application where users can submit goals like "Create a content calendar for social media" and watch as an AI agent systematically works through the problem, documenting its reasoning and delivering high-quality results. The beauty of this approach is that everything runs locally on your machine using Ollama, providing privacy benefits and eliminating API costs.

Key Concepts in Our Agent Framework

Before diving into the code, let's understand the core concepts that make our custom agent framework powerful:

1. Step-Based Task Decomposition

Complex tasks become manageable when broken down into smaller steps. Our agent takes a user's goal and automatically divides it into logical steps, similar to how a human would approach a complex problem:

typescript // Sample task decomposition const steps = [ "Analyze target audience and choose platforms", "Establish content themes and post types", "Create first half of weekly content calendar", "Create second half of weekly content calendar", "Add engagement strategies and hashtag recommendations" ];

2. Reasoning Before Action

For each step, our agent first explains its reasoning before taking action. This creates transparency and allows users to understand the agent's thought process:

typescript // Sample reasoning for a step const reasoning = "Before creating content, I need to understand who we're targeting and which platforms would be most effective for a coffee shop. Typically, Instagram and Facebook work well for food/beverage businesses.";

3. Streaming Progress Updates

Users receive real-time updates as the agent works through each step, maintaining engagement and giving visibility into the process:

typescript // Sending a real-time update to the client await sendUpdate({ type: 'log', message: 📝 Step ${step.number}: ${step.description} });

4. Contextual Memory

Each step builds upon previous steps, maintaining context throughout the execution:

typescript const stepPrompt = Task: "${this.goal}" Step ${stepNumber}/${Math.min(steps.length, this.maxSteps)}: ${stepDescription} Previous steps: ${this.steps.map(s => Step ${s.number}: ${s.description} -> ${s.output?.substring(0, 100)}...).join('\n')} Execute this step and provide the output. Be thorough but focused on just this step. ;

Setting Up the Project

Let's begin by creating a Next.js project and installing dependencies:

bash npx create-next-app@latest next-ollama-agent cd next-ollama-agent npm install dotenv react-markdown

Next, download and install [Ollama](https://ollama.com/), then pull the Mistral model:

bash ollama pull mistral

Building the Custom Agent Class

The heart of our application is the Agent class, which handles the execution of tasks:

```typescript // src/lib/agent.ts export interface Step { number: number; description: string; reasoning?: string; output?: string; }

export interface AgentResult { goal: string; steps: Step[]; output: string; }

export type StepCallback = (step: Step) => Promise<void> | void;

export class Agent { private goal: string; private maxSteps: number; private onStepComplete?: StepCallback; private steps: Step[] = [];

constructor(options: { goal: string; maxSteps?: number; onStepComplete?: StepCallback; }) { this.goal = options.goal; this.maxSteps = options.maxSteps || 5; this.onStepComplete = options.onStepComplete; }

async execute(): Promise<AgentResult> { // Step 1: Task analysis const taskAnalysis = await this.callOllama( Analyze this task: "${this.goal}". Break it down into ${this.maxSteps} clear steps that would lead to a high-quality result. Return a JSON array of step descriptions only, no additional text. );

// Parse steps from the model response let steps: string[] = []; try { const parsed = JSON.parse(this.extractJSON(taskAnalysis)); steps = Array.isArray(parsed) ? parsed : []; } catch (e) { // Fallback extraction with regex if JSON parsing fails const stepRegex = /\d+\.\s*(.*?)(?=\d+\.|$)/gs; const matches = [...taskAnalysis.matchAll(stepRegex)]; steps = matches.map(match => match[1].trim()); }

// Default steps if extraction fails if (steps.length === 0) { steps = ["Analyze the problem", "Generate solution", "Refine the output"]; }

// Execute each step for (let i = 0; i < Math.min(steps.length, this.maxSteps); i++) { const stepNumber = i + 1; const stepDescription = steps[i];

// Generate reasoning for this step const reasoning = await this.callOllama( For the task: "${this.goal}", I am on step ${stepNumber}: "${stepDescription}". Explain your reasoning for how you'll approach this step. Keep it clear and concise. );

// Execute the step with context from previous steps const stepPrompt = Task: "${this.goal}" Step ${stepNumber}/${Math.min(steps.length, this.maxSteps)}: ${stepDescription} Previous steps: ${this.steps.map(s => Step ${s.number}: ${s.description} -> ${s.output?.substring(0, 100)}...).join('\n')} Execute this step and provide the output. Be thorough but focused on just this step. ; const stepOutput = await this.callOllama(stepPrompt);

// Record the step const step: Step = { number: stepNumber, description: stepDescription, reasoning, output: stepOutput }; this.steps.push(step);

// Notify via callback if provided if (this.onStepComplete) { await this.onStepComplete(step); } }

// Generate final comprehensive output const finalPrompt = You've been working on: "${this.goal}" You've completed the following steps: ${this.steps.map(s => Step ${s.number}: ${s.description}).join('\n')} Now, compile all of your work into a comprehensive final output that achieves the original goal. Format your response using Markdown for readability. ; const finalOutput = await this.callOllama(finalPrompt);

return { goal: this.goal, steps: this.steps, output: finalOutput }; }

private async callOllama(prompt: string): Promise<string> { try { const response = await fetch('http://localhost:11434/api/generate', { method: 'POST', headers: { 'Content-Type': 'application/json', }, body: JSON.stringify({ model: 'mistral', prompt: prompt, stream: false, }), });

if (!response.ok) { throw new Error(Ollama API error: ${response.statusText}); }

const data = await response.json(); return data.response; } catch (error) { console.error('Error calling Ollama:', error); return Error: ${error instanceof Error ? error.message : 'Unknown error'}; } }

private extractJSON(text: string): string { // Try to extract JSON from the text const jsonRegex = /(\[.*\]|\{.*\})/s; const match = text.match(jsonRegex); return match ? match[0] : '[]'; } } ```

Building the Frontend

Our frontend uses React and Next.js to create a clean, responsive interface:

```tsx // src/app/page.tsx "use client";

import { useState, useRef, useEffect } from "react"; import ReactMarkdown from "react-markdown";

export default function Home() { const [goal, setGoal] = useState<string>(""); const [logs