Building and Evaluating a Local-First Research Assistant with GraphRAG and [post] deterministic

Building and Evaluating a Local-First Research Assistant with GraphRAG and vero-eval

*A comprehensive guide to creating a persona-driven AI assistant with rigorous evaluation using Neo4j, Ollama, and the vero-eval framework*

Introduction: Why Local GraphRAG Matters for Research Workflows

If you're building AI-powered applications in 2025, you've likely hit two major pain points: **context limitations** and **lack of systematic evaluation**. Large Language Models are powerful, but they struggle with long-term memory and consistent performance across edge cases. Enter GraphRAG—a methodology that combines knowledge graphs with retrieval-augmented generation to give your AI genuine memory and contextual awareness.

In this guide, we'll build a **Local Research Assistant** that: - Stores and retrieves research papers, notes, and conversations in a Neo4j knowledge graph - Uses Ollama for completely local inference (no API costs, full privacy) - Implements persona-driven responses that adapt based on RLHF feedback - **Most importantly**: Measures performance rigorously using the [vero-eval framework](https://github.com/vero-labs-ai/vero-eval)

This isn't another "hello world" tutorial. We're building production-ready infrastructure that you can deploy for real research workflows, with proper testing and evaluation baked in from day one.

Prerequisites and Starting Point

Before we dive in, you'll need:

**System Requirements:** - Python 3.9+ - Node.js 18+ - Docker (for Neo4j) - 16GB+ RAM recommended

**Core Technologies:** - [Ollama](https://ollama.ai) for local LLM inference - [Neo4j](https://neo4j.com) for graph database - [vero-eval](https://github.com/vero-labs-ai/vero-eval) for evaluation - Next.js + FastAPI (from the starter template)

**Clone the Starter Repository:**

bash git clone https://github.com/kliewerdaniel/chrisbot.git research-assistant cd research-assistant

This gives us a solid foundation with the frontend, basic chat interface, and project structure already in place. We'll extend it to build our research-focused GraphRAG system.

Part 1: Understanding the Architecture

Our Research Assistant follows the **PersonaGen architecture** pattern outlined by Daniel Kliewer, but applied to academic research workflows:

┌─────────────────────────────────────────────────────────┐ │ User Interface │ │ (Next.js Chat Interface) │ └────────────────────┬────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────┐ │ Reasoning Agent │ │ (Tool Calling + RLHF Threshold Logic) │ └────────────────────┬────────────────────────────────────┘ │ ┌──────────┴──────────┐ ▼ ▼ ┌──────────────────┐ ┌──────────────────┐ │ Neo4j Graph │ │ Ollama LLM │ │ RAG System │ │ (Mistral/Llama) │ │ │ │ │ │ • Papers │ │ • Generation │ │ • Authors │ │ • Embeddings │ │ • Concepts │ │ • Extraction │ │ • Citations │ │ │ └──────────────────┘ └──────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────┐ │ vero-eval Framework │ │ • Test Dataset Generation │ │ • Retrieval Metrics (Precision, Recall, MRR) │ │ • Generation Metrics (Faithfulness, BERTScore) │ │ • Persona Stress Testing │ └─────────────────────────────────────────────────────────┘

**Key Insight**: The persona system adapts its behavior based on evaluation feedback. If vero-eval shows poor retrieval for technical queries, the RLHF thresholds adjust to require more context before responding.

Part 2: Setting Up Neo4j GraphRAG

Neo4j is our memory layer. Following the [official Neo4j GenAI integration patterns](https://neo4j.com/docs/cypher-manual/current/genai-integrations/), we'll create a graph schema optimized for research.

Installing Neo4j GraphRAG for Python

```bash # Install the official Neo4j GraphRAG package pip install neo4j-graphrag

Install Ollama integration pip install "neo4j-graphrag[ollama]"

Start Neo4j (using Docker) docker run \ --name research-neo4j \ -p 7474:7474 -p 7687:7687 \ -e NEO4J_AUTH=neo4j/research2025 \ -v $PWD/neo4j-data:/data \ neo4j:latest ```


Defining the Research Knowledge Schema

Create scripts/graph_schema.py:

```python from neo4j_graphrag import GraphSchema from dataclasses import dataclass

@dataclass class ResearchSchema(GraphSchema): """ Knowledge graph schema for research assistant. Nodes: - Paper: Research papers with metadata - Author: Paper authors with affiliation - Concept: Extracted key concepts/topics - Note: User's research notes - Question: User queries with context Relationships: - AUTHORED: Author -> Paper - CITES: Paper -> Paper - DISCUSSES: Paper -> Concept - RELATES_TO: Concept -> Concept - ANSWERS: Paper -> Question """ node_types = { 'Paper': { 'properties': ['title', 'abstract', 'year', 'doi', 'pdf_path'], 'embedding_property': 'abstract_embedding' }, 'Author': { 'properties': ['name', 'affiliation', 'h_index'], 'embedding_property': None }, 'Concept': { 'properties': ['name', 'definition', 'domain'], 'embedding_property': 'definition_embedding' }, 'Note': { 'properties': ['content', 'timestamp', 'tags'], 'embedding_property': 'content_embedding' }, 'Question': { 'properties': ['query', 'timestamp', 'answered'], 'embedding_property': 'query_embedding' } } relationship_types = { 'AUTHORED': ('Author', 'Paper'), 'CITES': ('Paper', 'Paper'), 'DISCUSSES': ('Paper', 'Concept'), 'RELATES_TO': ('Concept', 'Concept'), 'ANSWERS': ('Paper', 'Question'), 'ANNOTATES': ('Note', 'Paper') } ```

**Why this schema?** Research workflows have natural graph structures: - Papers cite each other (transitive relationships) - Concepts relate to multiple papers - Authors collaborate across papers - User notes connect to specific papers

This lets us traverse the graph to find: "What papers discussing transformer architectures were cited by papers on RAG systems after 2023?"

Building the Graph Ingestion Pipeline

Create scripts/ingest_research_data.py:

```python import ollama from neo4j import GraphDatabase from neo4j_graphrag import GraphRAG from pathlib import Path import PyPDF2

class ResearchGraphBuilder: def __init__(self, neo4j_uri="bolt://localhost:7687", neo4j_user="neo4j", neo4j_password="research2025", ollama_model="mistral"): self.driver = GraphDatabase.driver(neo4j_uri, auth=(neo4j_user, neo4j_password)) self.ollama_model = ollama_model self.graph_rag = GraphRAG(self.driver) def extract_paper_metadata(self, pdf_path: Path) -> dict: """Extract title, abstract, and key sections from PDF""" with open(pdf_path, 'rb') as file: reader = PyPDF2.PdfReader(file) # Extract first 3 pages (usually contains abstract) text = "" for page in reader.pages[:3]: text += page.extract_text() # Use Ollama to extract structured metadata prompt = f"""Extract from this researc