Lizard Brain + Soul: A Hybrid AI Architecture for Game NPCs

I have been building a Minecraft mod that adds AI-powered villagers controlled by an LLM, with the goal of creating a more dynamic and engaging open world experience. When you log in, NPCs are actually trying to solve their own goals in game and exercise their own agency in how far they want to cooperate with you. The core challenge: how do you give an LLM control over a real-time game entity without it being laggy, expensive, or stupid?

The solution I landed on is a hybrid architecture I call "Lizard Brain + Soul."

The Problem

The naive approach would be to query Claude on every game tick (20 times per second) with the full world state, then execute whatever it says. This fails for three reasons:

1. Cost - API calls add up fast at 20/second
2. Latency - Claude takes ~500ms to respond, far too slow for real-time movement
3. Granularity mismatch - Claude thinks in terms of goals ("chop down that tree"), not individual actions ("move leg forward 0.3 blocks")

The Architecture

Instead, I split the AI into two layers:

The Soul (Claude)

The Soul is the strategic planner. It runs every ~60 seconds (or when triggered by events), observes the world state, and decides what the villager should do next.

It receives:

• Current position, health, inventory
• Nearby players and items
• Recent action failures
• Events that triggered thinking (player approached, task completed, etc.)

It responds with tool calls like gatherWood(), buildStructure("house"), or say("Hello!").

The Soul doesn't micromanage. It sets goals and trusts the lower layer to figure out execution.

The Lizard Brain (Traditional Game AI)

The Lizard Brain is the executor. It takes high-level actions from the Soul and handles the moment-to-moment details using Minecraft's built-in pathfinding and entity systems.

When the Soul says gatherWood(), the Lizard Brain:

1. Scans for nearby trees
2. Pathfinds to the nearest one
3. Builds scaffolding if needed to reach high logs
4. Breaks blocks in the right order (bottom-to-top to avoid floating logs)
5. Collects dropped items
6. Cleans up scaffolding

This runs on the game thread at full tick rate. No API calls, no latency.

Why This Works

The key insight is that strategic reasoning and tactical execution have different time scales and require different tools.

Strategic reasoning (What should I do? Where should I go?) benefits from:

• Broad context about goals and environment
• Natural language understanding
• Common sense reasoning
• Memory of past interactions

Tactical execution (How do I get there? Which block do I break first?) benefits from:

• Low latency
• Precise spatial calculations
• Access to game engine internals
• Deterministic behavior

Trying to do both with the same system means either the LLM is micromanaging (expensive, slow) or the game AI is strategizing (rigid, dumb).

The hybrid approach lets each layer do what it's good at.

Implementation Details

Event-Driven Thinking

The Soul doesn't think on a fixed timer. Instead, the Lizard Brain triggers it when something interesting happens:

• A player approaches
• An action fails
• The action queue empties
• A certain time has passed since last thought

This reduces API costs while keeping the villager responsive.

Failure Feedback

When an action fails (can't reach target, missing resources, etc.), the Lizard Brain records it. The next time the Soul thinks, it sees these failures and can adjust strategy.

This creates a feedback loop where the villager learns from mistakes - at least within a session.

Tool Design

I distinguish between "query tools" and "action tools":

Query tools execute immediately and return information:

• observe() - scan surroundings
• findBlocks("diamond_ore") - search for specific blocks

Action tools get queued for the Lizard Brain:

• moveToPosition(x, y, z)
• placeBlock("cobblestone", x, y, z)
• say("Hello!")

This prevents Claude from trying to queue a query (which makes no sense) or expecting immediate results from an action (which would block).

Cost Management

Each villager costs roughly $0.01-0.05 per hour depending on activity level. The main cost-saving techniques:

1. Event-driven thinking - Don't query unless something changed
2. Memory compression - Summarize old conversation history instead of sending full transcripts
3. Compact observations - Send "5 oak trees nearby" not coordinates of every log block
4. Action decomposition - Let the Lizard Brain handle choreography instead of having Claude plan every step

What I Learned

LLMs are good at deciding, bad at doing. Claude excels at "should I gather wood or explore?" but struggles with "move 0.3 blocks north then swing axe." Design your interface accordingly.

Failure feedback is crucial. Without it, the AI gets stuck in loops. With it, it adapts.

Token economy matters. Every character in your observation costs money. Be ruthless about what context the AI actually needs.

Async is non-negotiable. Blocking the game thread for 500ms while waiting for Claude would be unplayable. CompletableFuture saved me here.

The Lizard Brain + Soul pattern might be useful beyond Minecraft. Any system where an LLM needs to control real-time behavior could benefit from separating strategic planning from tactical execution.

The mod is still in development, but the architecture has held up well. The villagers feel surprisingly alive - they plan, they fail, they adapt. Sometimes they even surprise me.