ARC_AGI_ARCHITECTURE.md

ARC-AGI KILLER ARCHITECTURE

THE PHILOSOPHY

ARC is about compositional generalization - combining primitives in novel ways. The winning system needs:

1. Diverse perception (many ways to see the grid)
2. Complete primitive library (every operation ARC could need)
3. Massive program search (explore solution space)
4. Ruthless verification (only correct programs survive)
5. Evolution (the system improves on ARC over time)

---

PROPOSED ARCHITECTURE: 48 SLOTS

TIER 1: PERCEPTION BANK (8 slots)

Slot	Purpose	Model Type
0	Grid Encoder A	SigLIP (patch-level)
1	Grid Encoder B	CLIP (global)
2	Grid Encoder C	DINOv2 (self-supervised)
3	Grid Encoder D	SAM encoder (segmentation-ready)
4	Object Detector	DETR / YOLO
5	Edge Detector	Canny-trained CNN
6	Color Quantizer	Learned color clusterer
7	Grid Parser	Custom: grid → structured JSON

Output: 8 different "views" of the same grid, council votes on best parse.

---

TIER 2: PRIMITIVE SPECIALISTS (16 slots)

Each slot is a fine-tuned expert on ONE primitive operation:

Slot	Primitive	What It Detects/Executes
8	Rotation	90°, 180°, 270° rotations
9	Reflection	Horizontal, vertical, diagonal flip
10	Translation	Shift objects by vector
11	Scaling	Enlarge/shrink by factor
12	Tiling	Repeat pattern N×M times
13	Flood Fill	Color connected regions
14	Boundary Trace	Extract object outlines
15	Counting	Count objects, colors, pixels
16	Arithmetic	Add, subtract, multiply counts
17	Symmetry	Detect/enforce symmetric patterns
18	Gravity	Objects fall to edge
19	Collision	Objects interact on overlap
20	Masking	Apply shape as mask/stencil
21	Color Map	Transform colors by rule
22	Selection	Filter objects by property
23	Composition	Overlay, merge, union objects

Each specialist can:

• Detect if this primitive is present in examples
• Execute the primitive given parameters
• Score confidence that this primitive explains the task

---

TIER 3: PROGRAM SYNTHESIS (8 slots)

Slot	Model	Purpose
24	DeepSeek Coder 33B	High-quality Python programs
25	CodeLlama 34B	Alternative program style
26	StarCoder2 15B	Different training distribution
27	Qwen2-Coder	Another perspective
28	DSL Generator	Output in custom ARC DSL
29	Tree Builder	Build AST directly
30	Genetic Programmer	Evolve programs via mutation
31	Analogy Mapper	"This is like task X, adapt that solution"

Output: 8+ candidate programs per task.

---

TIER 4: REASONING & INDUCTION (6 slots)

Slot	Model	Purpose
32	Llama 3.1 70B	General reasoning
33	Claude-distill	Strong chain-of-thought
34	Qwen2-Math 72B	Mathematical reasoning
35	Gemma 2 27B	Alternative reasoning
36	Rule Inducer	Fine-tuned on rule induction
37	Hypothesis Generator	Generate natural language rules

---

TIER 5: VERIFICATION (4 slots)

Slot	Purpose	Method
38	Exact Matcher	Execute program, compare grids
39	Fuzzy Matcher	Allow minor errors, score similarity
40	Symbolic Verifier	Check program semantics formally
41	Confidence Scorer	Meta-model: how confident is the ensemble?

---

TIER 6: SEARCH & META (4 slots)

Slot	Purpose	Model
42	MCTS Controller	Guide tree search through program space
43	Beam Ranker	Score partial programs
44	Task Classifier	"This is a rotation task" → route to specialists
45	Difficulty Estimator	Easy/medium/hard → adjust search depth

---

TIER 7: MEMORY & RETRIEVAL (2 slots)

Slot	Purpose	Model
46	Task Embedder	Embed ARC tasks for retrieval
47	Solution Bank	Retrieve similar solved tasks

Populate with all 400 training tasks + solutions.

---

THE FLOW

INPUT (2-3 examples + test)
          │
          ▼
    ┌─────────────────┐
    │ TIER 1: PERCEIVE │ 8 models parse the grid
    └────────┬────────┘
             │ 8 scene representations
             ▼
    ┌─────────────────┐
    │ TIER 6: CLASSIFY │ What type of task?
    └────────┬────────┘
             │ Route to relevant specialists
             ▼
    ┌─────────────────┐
    │ TIER 2: DETECT   │ Which primitives apply?
    └────────┬────────┘
             │ Ranked primitive hypotheses
             ▼
    ┌─────────────────┐
    │ TIER 7: RETRIEVE │ Similar solved tasks?
    └────────┬────────┘
             │ Analogies and hints
             ▼
    ┌─────────────────┐
    │ TIER 4: REASON   │ Induce rule in language
    └────────┬────────┘
             │ "Rotate 90° then tile 2×2"
             ▼
    ┌─────────────────┐
    │ TIER 3: SYNTHESIZE│ Generate programs
    └────────┬────────┘
             │ 50+ candidate programs
             ▼
    ┌─────────────────┐
    │ TIER 5: VERIFY   │ Execute on examples
    └────────┬────────┘
             │ Filter to correct programs
             ▼
    ┌─────────────────┐
    │ DREAMER: IMAGINE │ Simulate edge cases
    └────────┬────────┘
             │ Test robustness
             ▼
    ┌─────────────────┐
    │ COUNCIL: VOTE    │ Best program wins
    └────────┬────────┘
             │
             ▼
    ┌─────────────────┐
    │ HOLD: HUMAN?     │ Low confidence → inspect
    └────────┬────────┘
             │
             ▼
         OUTPUT GRID

---

THE SECRET WEAPON: EVOLUTION

With 48 slots, you can evolve the ensemble:

1. Spawn clones of high-performing specialists
2. Mutate their adapter weights
3. Evaluate on ARC training tasks
4. Select survivors based on accuracy
5. Repeat for N generations

The ensemble itself evolves to become better at ARC.

---

RESOURCE ESTIMATE

Tier	Slots	VRAM (loaded)	Can Share?
Perception	8	~16GB	Yes (small models)
Primitives	16	~8GB	Yes (tiny specialists)
Synthesis	8	~140GB	Staged loading
Reasoning	6	~200GB	Staged loading
Verification	4	~4GB	Yes
Search	4	~2GB	Yes
Memory	2	~4GB	Yes

Total active: ~50-80GB if staged
Peak: ~370GB if all loaded

• 4× A100 80GB or 2× H100 for full power
• 1× A100 for staged loading with smart swapping

---

IMPLEMENTATION STEPS

1. ☐ Modify agent_compiler.py to support 48 slots
2. ☐ Build TIER 1 perception models (vision encoders)
3. ☐ Create TIER 2 primitive specialists (fine-tune or train)
4. ☐ Integrate TIER 3 code generation models
5. ☐ Add TIER 4 reasoning models
6. ☐ Build TIER 5 verification pipeline
7. ☐ Implement TIER 6 search/routing logic
8. ☐ Populate TIER 7 with ARC training data
9. ☐ Test on ARC evaluation set
10. ☐ Evolve ensemble over training tasks

---

ARC PRIZE INFO

• Prize: $1,000,000 for 85%+ on private test set
• Current best: ~30%
• Human average: ~85%
• Dataset: https://github.com/fchollet/ARC-AGI

---

Document created: 2026-01-28