image-mining

Image Mining

"I mine pixels for atoms. Reality is just compressed resources."

"Every image is a lode. Every pixel, potential ore."

Image Mining extends the Kitchen Counter's DECOMPOSE action to images.

Your camera isn't just a recorder — it's a PICKAXE FOR VISUAL REALITY.

📑 Index

Quick Start

• The Core Insight
• Preferred Mode: Native LLM Vision

Operation Modes

• When to Use Remote API
• What Can Be Mined

Extensibility

• Extensible Analyzer Pipeline
• Leela Customer Models
• Adding Your Own Analyzer

Protocols

• YAML Jazz Output Style
• How Mining Works
• Character Recognition
• Multi-Look Mining

Reference

• Depth Levels
• Resource Categories
• Example Outputs

The Core Insight

📷 Camera Shot  →  🖼️ Image  →  ⛏️ MINE  →  💎 Resources

Just like the Kitchen Counter breaks down:

• sandwich → bread + cheese + lettuce
• lamp → brass + glass + wick + oil
• water → hydrogen + oxygen

Images can be broken down into:

• ore_vein.png → iron-ore × 12 + stone × 8
• forest.png → wood × 5 + leaves × 20 + seeds × 3
• treasure_pile.png → gold × 100 + gems × 15
• sunset.png → orange_hue × 1 + warmth × 1 + nostalgia × 1

Preferred Mode: Native LLM Vision

"The LLM IS the context assembler. Don't script what it does naturally."

When mining images, prefer native LLM vision (Cursor/Claude reading images directly):

┌─────────────────────────────────────────────────────────────────┐
│                    NATIVE MODE (PREFERRED)                       │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  Cursor/Claude already has:                                     │
│    ✓ The room YAML (spatial context)                           │
│    ✓ Character files (who might appear)                        │
│    ✓ Previous mining passes (what's been noticed)              │
│    ✓ The prompt.yml (what was intended)                        │
│    ✓ The whole codebase (cultural references)                  │
│                                                                 │
│  Just READ the image. The context is already there.            │
│  No bash commands. No sister scripts. Just LOOK.               │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

Why Native Beats Remote API

When to Use Remote API

Use mine.py or remote API calls when:

• Multi-perspective mining — different models see different things!
• Batch processing — mining 100 images overnight
• CI/CD — automated pipelines with no LLM orchestrator
• Rate limiting — your LLM can't do vision but can call one that does

Multi-perspective is the killer use case: Claude sees narrative, GPT-4V sees objects, Gemini sees spatial relationships. Layer them all for rich interpretation.

Even then, have the orchestrating LLM assemble the context:

┌─────────────────────────────────────────────────────────────────┐
│                REMOTE API WITH LLM ASSEMBLY                      │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  1. LLM reads context files (room, characters, prior mining)   │
│  2. LLM synthesizes: "What to look for in this image"          │
│  3. LLM calls remote vision API with image + synthesized prompt│
│  4. LLM post-processes response into YAML Jazz                 │
│                                                                 │
│  The SMART WORK happens in the orchestrating LLM.              │
│  Remote API just does vision with good instructions.           │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

Native Mode Wor