COMPILER_CLI.md

# KEY Agent Compiler CLI Documentation
═══════════════════════════════════════════════════════════════════════════════

## Overview

The KEY Agent Compiler (`agent_compiler.py`) compiles evolved brain agents into
self-contained Python capsules. Each capsule is a "Glass Box AI" - fully
transparent with embedded documentation and cryptographic provenance.

## Brain Types Supported

| Type       | Class            | Quine Class          | Description                    |
|------------|------------------|----------------------|--------------------------------|
| mlp        | MLPBrain         | QuineBrainMLP        | Simple MLP baseline            |
| dreamer    | DreamerBrain     | QuineBrain           | DreamerV3 world model + LoRA   |
| scarecrow  | ScarecrowBrain   | QuineScarecrowBrain  | Universal model wrapper        |
| council    | CouncilBrain     | QuineCouncilBrain    | Multi-agent consensus          |
| embedding  | EmbeddingBrain   | QuineEmbeddingBrain  | Sentence transformer + head    |
| lora       | LoraBrain        | QuineBrain           | LoRA-adapted transformer       |


## Quick Start

### 1. Compile a Single Agent
```python
from node import Node
from brain import ScarecrowBrain
from agent_compiler import NodeCompiler

# Create and evolve a node
brain = ScarecrowBrain()
node = Node(brain=brain, fitness=0.85, generation=10)

# Compile to capsule
compiler = NodeCompiler(output_dir='children')
path = compiler.compile(node, 'my_agent')
print(f"Generated: {path}")
```

### 2. Compile an Ensemble
```python
from agent_compiler import NodeCompiler

compiler = NodeCompiler()
nodes = [node1, node2, node3]  # Your evolved population
path = compiler.compile_ensemble(nodes, 'my_ensemble')
```

### 3. Use Compiled Capsule
```python
import importlib.util

# Load capsule
spec = importlib.util.spec_from_file_location("agent", "children/my_agent.py")
capsule = importlib.util.module_from_spec(spec)
spec.loader.exec_module(capsule)

# Get quine brain
qb = capsule.get_quine_brain()

# Run inference
result = qb.forward({'input': np.random.randn(384)})
print(result['output'])

# For Scarecrow: plug in external model
qb.plug_model(lambda x: x * 2)

# For Council: add specialist
qb.add_specialist('expert', my_model)
```


## Capsule Features

### Quine Brain Interface

All quine brains implement a common interface:

```python
class QuineBrain:
    def forward(self, inputs: dict) -> dict:
        """Run inference."""
        ...
    
    def get_merkle_hash(self) -> str:
        """Return cryptographic hash for verification."""
        ...
```

Additional methods vary by brain type:

| Method          | MLP | Dreamer | Scarecrow | Council | Embedding |
|-----------------|-----|---------|-----------|---------|-----------|
| forward()       | ✓   | ✓       | ✓         | ✓       | ✓         |
| get_merkle_hash | ✓   | ✓       | ✓         | ✓       | ✓         |
| forward_minimal | ✓   | ✓       | -         | -       | -         |
| plug_model()    | -   | -       | ✓         | -       | -         |
| add_specialist()| -   | -       | -         | ✓       | -         |
| embed()         | -   | -       | -         | -       | ✓         |
| get_params()    | ✓   | -       | ✓         | -       | ✓         |


### Capsule Functions

Every capsule includes these functions:

```python
# Core
get_quine_brain()        # Get the self-contained brain
get_quine_hash()         # Get merkle hash
verify_quine_integrity() # Verify brain hasn't been modified

# Documentation
show_readme()            # Print embedded README
get_node_docs()          # Get per-node documentation
export_all_docs()        # Export all documentation artifacts

# Export
export_pt(path)          # Export to PyTorch format
export_onnx(path)        # Export to ONNX format
export_pdf(mode, path)   # Export PDF factory report
replicate_quine(path)    # Create standalone quine file

# FelixBag (Vector Memory)
store(key, data)         # Store with semantic key
retrieve(key)            # Exact retrieval
recall(query, k=5)       # Semantic search
materialize(key, path)   # Export to filesystem
bag_stats()              # Memory statistics
```


## FelixBag (Capsule Memory)

Every capsule includes FelixBag, a semantic vector storage system that uses the
capsule's internal embedding model for similarity search.

```python
agent = CapsuleAgent()

# Store anything
agent.store("project/readme", readme_text)
agent.store("config/main", {"key": "value"})
agent.store("model/checkpoint", binary_data)

# Semantic search (uses shared embedder)
results = agent.recall("how to configure", k=5)
for key, score in results:
    print(f"{key}: {score:.3f}")

# Check embedder status
print(agent.embedder_info())  # Shows which model + source
print(agent.bag_stats())      # Count, embedder, shared status
```

The embedder is shared with CapsuleBrain - same model for inference and memory.


## PDF Factory Reports

Generate professional PDF documentation (requires `pip install reportlab pygments`):

```python
agent = CapsuleAgent()

# Different modes
agent.export_pdf(mode='full')       # Entire source, syntax-highlighted
agent.export_pdf(mode='report')     # Diagnostic factory report
agent.export_pdf(mode='provenance') # CASCADE lineage chain
agent.export_pdf(mode='artifacts')  # README, config, node docs
agent.export_pdf(mode='summary')    # One-page executive summary

# Custom output path
agent.export_pdf(mode='report', path='my_report.pdf')
```

### PDF Modes

| Mode       | Content                                              |
|------------|------------------------------------------------------|
| full       | Complete .py source with line numbers, syntax colors |
| report     | Brain info, traits, CASCADE status, FelixBag stats  |
| provenance | Genesis root, merkle hashes, lineage verification   |
| artifacts  | README, config JSON, node docs, FelixBag keys       |
| summary    | One-page: generation, fitness, brain type, metrics  |


## TUI (Interactive Mode)

Compiled capsules include an interactive TUI for live experimentation:

```bash
python children/my_agent.py
```

### TUI Commands

```
BRAIN CONTROL:
  plug <path>         Load model into councilor slot
  council             Show councilor roster  
  debate <n>          Set debate rounds (for council)
  consensus <type>    Change consensus method

MEMORY (FelixBag):
  store <key>         Store item in bag
  recall <query>      Semantic search
  bag                 List all stored keys
  embedder            Show embedder status

EXPORT:
  export pt           PyTorch TorchScript (.pt)
  export onnx         ONNX format (.onnx)
  export quine        Self-replicating capsule (.py)
  export state        JSON config (.json)
  export pdf <mode>   PDF factory report
                      modes: full, report, provenance, artifacts, summary

PROVENANCE:
  cascade             Show CASCADE lattice status
  provenance          Full lineage chain
  genesis             Verify genesis link

INFERENCE:
  run <input>         Run forward pass
  imagine <horizon>   Imagination rollouts (Dreamer)
  hold                Forward with HOLD yield points
```


## Configuration

### CapsuleConfig

```python
from agent_compiler import CapsuleConfig

config = CapsuleConfig(
    name="my_agent",
    compress=True,              # Gzip brain data
    include_requirements=True,  # Generate requirements.txt
    max_organisms=10,           # For ensemble: max agents
    lora_handling="embed_path", # For LoRA: "embed_path", "merge_weights", "download"
    base_model_path="...",      # For LoRA: base model location
)
```


## Artifact Registry

The compiler uses an artifact registry to determine which artifacts each brain
type requires. See `artifact_registry.py` for the complete mapping.

```python
from artifact_registry import get_artifact_spec, get_requirements

spec = get_artifact_spec('council')
print(f"Required: {[a.value for a in spec.required]}")
print(f"Optional: {[a.value for a in spec.optional]}")
print(f"Packages: {get_requirements('council')}")
```


## Testing

Run the artifact audit to verify all brain types compile correctly:

```bash
python artifact_audit.py
```

Expected output:
```
SUMMARY
============================================================
  MLP: ✓ PASS
  Scarecrow: ✓ PASS
  Council: ✓ PASS
  Embedding: ✓ PASS
  Dreamer: ✓ PASS
```


## Integration with HOLD

Capsules support HOLD (Human-Oriented Learning and Development) when
CASCADE-LATTICE is installed:

```python
# Enable HOLD for human oversight
result = capsule.forward_hold({'input': data}, blocking=True)
# System pauses for human approval before continuing
```


## Integration with Rerun

For visual debugging, install rerun-sdk:

```bash
pip install rerun-sdk
```

Capsules will automatically stream inference data to Rerun for visualization.


## Model Interface Server

Generate an HTTP server interface for game integration:

```python
from agent_compiler import generate_model_interface

generate_model_interface(
    capsule_path="children/my_agent.py",
    output_dir="my_interface"
)
```

Then run:
```bash
cd my_interface
python server.py --port 8765
```

Connect your game client to `http://127.0.0.1:8765/forward`


## Troubleshooting

### "No module named 'test_xxx'"
Use `importlib.util.spec_from_file_location()` instead of `import_module()`.

### "matmul dimension mismatch"
Ensure input dimensions match the brain's expected input_size.

### "get_quine_brain() returns None"
The capsule doesn't have a quine class defined for that brain type.
Check that the brain type is supported in the compiler.

### "verify_quine_integrity() returns False"
The brain parameters have been modified since compilation.
This is expected if you've mutated or trained the brain.