COMPILER_CLI.md

KEY Agent Compiler CLI Documentation

═══════════════════════════════════════════════════════════════════════════════

Overview

The KEY Agent Compiler (agent_compiler.py) compiles evolved brain agents into self-contained Python capsules. Each capsule is a "Glass Box AI" - fully transparent with embedded documentation and cryptographic provenance.

Brain Types Supported

Type	Class	Quine Class	Description
mlp	MLPBrain	QuineBrainMLP	Simple MLP baseline
dreamer	DreamerBrain	QuineBrain	DreamerV3 world model + LoRA
scarecrow	ScarecrowBrain	QuineScarecrowBrain	Universal model wrapper
council	CouncilBrain	QuineCouncilBrain	Multi-agent consensus
embedding	EmbeddingBrain	QuineEmbeddingBrain	Sentence transformer + head
lora	LoraBrain	QuineBrain	LoRA-adapted transformer

Quick Start

1. Compile a Single Agent

from node import Node
from brain import ScarecrowBrain
from agent_compiler import NodeCompiler

# Create and evolve a node
brain = ScarecrowBrain()
node = Node(brain=brain, fitness=0.85, generation=10)

# Compile to capsule
compiler = NodeCompiler(output_dir='children')
path = compiler.compile(node, 'my_agent')
print(f"Generated: {path}")

2. Compile an Ensemble

from agent_compiler import NodeCompiler

compiler = NodeCompiler()
nodes = [node1, node2, node3]  # Your evolved population
path = compiler.compile_ensemble(nodes, 'my_ensemble')

3. Use Compiled Capsule

import importlib.util

# Load capsule
spec = importlib.util.spec_from_file_location("agent", "children/my_agent.py")
capsule = importlib.util.module_from_spec(spec)
spec.loader.exec_module(capsule)

# Get quine brain
qb = capsule.get_quine_brain()

# Run inference
result = qb.forward({'input': np.random.randn(384)})
print(result['output'])

# For Scarecrow: plug in external model
qb.plug_model(lambda x: x * 2)

# For Council: add specialist
qb.add_specialist('expert', my_model)

Capsule Features

Quine Brain Interface

All quine brains implement a common interface:

class QuineBrain:
    def forward(self, inputs: dict) -> dict:
        """Run inference."""
        ...
    
    def get_merkle_hash(self) -> str:
        """Return cryptographic hash for verification."""
        ...

Additional methods vary by brain type:

Method	MLP	Dreamer	Scarecrow	Council	Embedding
forward()	✓	✓	✓	✓	✓
get_merkle_hash	✓	✓	✓	✓	✓
forward_minimal	✓	✓	-	-	-
plug_model()	-	-	✓	-	-
add_specialist()	-	-	-	✓	-
embed()	-	-	-	-	✓
get_params()	✓	-	✓	-	✓

Capsule Functions

Every capsule includes these functions:

# Core
get_quine_brain()        # Get the self-contained brain
get_quine_hash()         # Get merkle hash
verify_quine_integrity() # Verify brain hasn't been modified

# Documentation
show_readme()            # Print embedded README
get_node_docs()          # Get per-node documentation
export_all_docs()        # Export all documentation artifacts

# Export
export_pt(path)          # Export to PyTorch format
export_onnx(path)        # Export to ONNX format
export_pdf(mode, path)   # Export PDF factory report
replicate_quine(path)    # Create standalone quine file

# FelixBag (Vector Memory)
store(key, data)         # Store with semantic key
retrieve(key)            # Exact retrieval
recall(query, k=5)       # Semantic search
materialize(key, path)   # Export to filesystem
bag_stats()              # Memory statistics

FelixBag (Capsule Memory)

Every capsule includes FelixBag, a semantic vector storage system that uses the capsule's internal embedding model for similarity search.

agent = CapsuleAgent()

# Store anything
agent.store("project/readme", readme_text)
agent.store("config/main", {"key": "value"})
agent.store("model/checkpoint", binary_data)

# Semantic search (uses shared embedder)
results = agent.recall("how to configure", k=5)
for key, score in results:
    print(f"{key}: {score:.3f}")

# Check embedder status
print(agent.embedder_info())  # Shows which model + source
print(agent.bag_stats())      # Count, embedder, shared status

The embedder is shared with CapsuleBrain - same model for inference and memory.

PDF Factory Reports

Generate professional PDF documentation (requires pip install reportlab pygments):

agent = CapsuleAgent()

# Different modes
agent.export_pdf(mode='full')       # Entire source, syntax-highlighted
agent.export_pdf(mode='report')     # Diagnostic factory report
agent.export_pdf(mode='provenance') # CASCADE lineage chain
agent.export_pdf(mode='artifacts')  # README, config, node docs
agent.export_pdf(mode='summary')    # One-page executive summary

# Custom output path
agent.export_pdf(mode='report', path='my_report.pdf')

PDF Modes

Mode	Content
full	Complete .py source with line numbers, syntax colors
report	Brain info, traits, CASCADE status, FelixBag stats
provenance	Genesis root, merkle hashes, lineage verification
artifacts	README, config JSON, node docs, FelixBag keys
summary	One-page: generation, fitness, brain type, metrics

TUI (Interactive Mode)

Compiled capsules include an interactive TUI for live experimentation:

python children/my_agent.py

TUI Commands

BRAIN CONTROL:
  plug <path>         Load model into councilor slot
  council             Show councilor roster  
  debate <n>          Set debate rounds (for council)
  consensus <type>    Change consensus method

MEMORY (FelixBag):
  store <key>         Store item in bag
  recall <query>      Semantic search
  bag                 List all stored keys
  embedder            Show embedder status

EXPORT:
  export pt           PyTorch TorchScript (.pt)
  export onnx         ONNX format (.onnx)
  export quine        Self-replicating capsule (.py)
  export state        JSON config (.json)
  export pdf <mode>   PDF factory report
                      modes: full, report, provenance, artifacts, summary

PROVENANCE:
  cascade             Show CASCADE lattice status
  provenance          Full lineage chain
  genesis             Verify genesis link

INFERENCE:
  run <input>         Run forward pass
  imagine <horizon>   Imagination rollouts (Dreamer)
  hold                Forward with HOLD yield points

Configuration

CapsuleConfig

from agent_compiler import CapsuleConfig

config = CapsuleConfig(
    name="my_agent",
    compress=True,              # Gzip brain data
    include_requirements=True,  # Generate requirements.txt
    max_organisms=10,           # For ensemble: max agents
    lora_handling="embed_path", # For LoRA: "embed_path", "merge_weights", "download"
    base_model_path="...",      # For LoRA: base model location
)

Artifact Registry

The compiler uses an artifact registry to determine which artifacts each brain type requires. See artifact_registry.py for the complete mapping.

from artifact_registry import get_artifact_spec, get_requirements

spec = get_artifact_spec('council')
print(f"Required: {[a.value for a in spec.required]}")
print(f"Optional: {[a.value for a in spec.optional]}")
print(f"Packages: {get_requirements('council')}")

Testing

Run the artifact audit to verify all brain types compile correctly:

python artifact_audit.py

Expected output: ``` SUMMARY

MLP: ✓ PASS Scarecrow: ✓ PASS Council: ✓ PASS Embedding: ✓ PASS Dreamer: ✓ PASS

## Integration with HOLD

Capsules support HOLD (Human-Oriented Learning and Development) when
CASCADE-LATTICE is installed:

```python
# Enable HOLD for human oversight
result = capsule.forward_hold({'input': data}, blocking=True)
# System pauses for human approval before continuing

Integration with Rerun

For visual debugging, install rerun-sdk:

pip install rerun-sdk

Capsules will automatically stream inference data to Rerun for visualization.

Model Interface Server

Generate an HTTP server interface for game integration:

from agent_compiler import generate_model_interface

generate_model_interface(
    capsule_path="children/my_agent.py",
    output_dir="my_interface"
)

Then run:

cd my_interface
python server.py --port 8765

Connect your game client to http://127.0.0.1:8765/forward

Troubleshooting

"No module named 'test_xxx'"

Use importlib.util.spec_from_file_location() instead of import_module().

"matmul dimension mismatch"

Ensure input dimensions match the brain's expected input_size.

"get_quine_brain() returns None"

The capsule doesn't have a quine class defined for that brain type. Check that the brain type is supported in the compiler.

"verify_quine_integrity() returns False"

The brain parameters have been modified since compilation. This is expected if you've mutated or trained the brain.