# MDAP Multi-Technique Chaining & Validation Plan

## Executive Summary
Build a multi-agent system that chains techniques from the library, validates solutions through consensus, and tackles novel olympiad problems through decomposition and synthesis.

---

## Phase 1: Technique Dependency Graph (Foundation)

### 1.1 Build Technique Relationships
**Goal**: Discover which techniques commonly appear together

**Implementation**:
```typescript
// Analyze co-occurrence patterns from original extractions
interface TechniqueGraph {
  technique_id: number;
  prerequisite_techniques: number[];  // Must know before this
  complementary_techniques: number[]; // Often used together
  follow_up_techniques: number[];     // Next logical steps
  confidence: number;
}
```

**Data Mining**:
- Parse `extraction-results-batch.json` (13,469 problems)
- For each problem, find which techniques appeared together
- Build co-occurrence matrix
- Identify sequential patterns (technique A → technique B)

**Example Chains**:
```
Prime Factorization → Fundamental Theorem of Arithmetic → Modular Arithmetic
Pythagorean Theorem → Similar Triangles → Area Calculation
Vieta's Formulas → Quadratic Formula → Substitution Method
```

**Output**: `data/technique-graph.json`

**Estimated Cost**: $0 (just data processing)
**Time**: 1-2 hours

---

## Phase 2: Technique Chaining Engine

### 2.1 Chain Discovery Agent
**Purpose**: Given a problem, find the optimal sequence of techniques

**Algorithm**:
```
1. Find top 5 relevant techniques (semantic search)
2. For each technique:
   - Check prerequisites (do we need other techniques first?)
   - Find complementary techniques (what else helps?)
   - Identify follow-up steps (what comes next?)
3. Build technique DAG (directed acyclic graph)
4. Find optimal path through DAG
```

**Agent**: `technique-chain-planner`
```typescript
interface TechniqueChain {
  techniques: Technique[];
  reasoning: string;
  confidence: number;
  estimated_difficulty: 1-10;
}
```

### 2.2 Sequential Solver
**Purpose**: Apply techniques in sequence, building on previous results

**Flow**:
```
Problem → [Technique 1] → Partial Solution 1
                        → [Technique 2] → Partial Solution 2
                                       → [Technique 3] → Final Solution
```

**Agent**: `sequential-technique-solver`
- Takes problem + technique chain
- Applies each technique to previous results
- Maintains context across steps
- Detects when stuck (returns confidence < 0.3)

**Key Features**:
- Context passing between steps
- Backtracking if technique fails
- Alternative path exploration

---

## Phase 3: Validation Layer (Consensus)

### 3.1 Multi-Model Validation
**Goal**: Use different models to solve same problem, compare answers

**Models**:
1. **GPT-4o** - Strong reasoning, expensive
2. **Claude Sonnet 3.5** - Best for math
3. **Llama 3.3 70B** (Groq) - Fast, cheap
4. **DeepSeek-V3** - Strong on STEM
5. **Gemini 2.0 Flash** - Good balance

**Consensus Logic**:
```typescript
interface ValidationResult {
  model: string;
  answer: string;
  confidence: number;
  reasoning: string;
  technique_used: string;
}

function buildConsensus(results: ValidationResult[]): ConsensusAnswer {
  // If 3+ models agree on exact answer → high confidence
  // If 2 models agree + reasoning similar → medium confidence
  // If all different → flag for human review

  return {
    answer: mostCommonAnswer,
    confidence: agreementScore,
    dissenting_opinions: minorityAnswers,
    reasoning_synthesis: mergeReasonings(results)
  };
}
```

### 3.2 Proof Verification Agent
**Purpose**: Check mathematical validity of solutions

**Checks**:
- Algebraic manipulation correctness
- Logical step validity
- Completeness (did we consider all cases?)
- Counterexample testing

**Agent**: `proof-validator`
```typescript
interface ProofValidation {
  is_valid: boolean;
  gaps: string[];         // Missing steps
  errors: string[];       // Logical errors
  suggestions: string[];  // How to fix
  confidence: number;
}
```

### 3.3 Answer Verification
**Purpose**: Verify numerical answers by substitution

**Methods**:
- Plug answer back into original equation
- Check constraints (e.g., n must be positive integer)
- Verify uniqueness (are there other solutions?)

---

## Phase 4: MDAP Architecture

### 4.1 Agent Roles

**Coordinator Agent** (`mdap-coordinator`):
- Receives problem
- Orchestrates all other agents
- Makes final decision on answer

**Decomposition Agent** (`problem-decomposer`):
- Breaks complex problems into sub-problems
- Identifies problem type (algebra, geometry, etc.)
- Suggests technique categories

**Technique Router** (`technique-router`):
- Semantic search for relevant techniques
- Builds technique chains
- Routes to specialist solvers

**Specialist Solvers** (category-specific):
- `algebra-specialist`: Equations, polynomials, inequalities
- `geometry-specialist`: Triangles, circles, coordinate geometry
- `number-theory-specialist`: Primes, divisibility, modular arithmetic
- `combinatorics-specialist`: Counting, permutations, graph theory

**Synthesizer Agent** (`solution-synthesizer`):
- Combines partial solutions
- Fills gaps between steps
- Creates coherent narrative

**Validator Agent** (`solution-validator`):
- Runs consensus validation
- Checks proof correctness
- Verifies answer

### 4.2 Communication Flow

```
┌─────────────────┐
│   User Problem  │
└────────┬────────┘
         ↓
┌────────────────────────┐
│  MDAP Coordinator      │
└────────┬───────────────┘
         ↓
┌────────────────────────┐
│  Problem Decomposer    │ → Sub-problems: [P1, P2, P3]
└────────┬───────────────┘
         ↓
┌────────────────────────┐
│  Technique Router      │ → Finds techniques for each sub-problem
└────────┬───────────────┘
         ↓
┌────────────────────────────────────────┐
│  Parallel Specialist Solvers           │
│  - Algebra: solves P1 with T1→T2      │
│  - Geometry: solves P2 with T3        │
│  - Number Theory: solves P3 with T4→T5│
└────────┬───────────────────────────────┘
         ↓
┌────────────────────────┐
│  Solution Synthesizer  │ → Combines P1+P2+P3 solutions
└────────┬───────────────┘
         ↓
┌────────────────────────┐
│  Consensus Validator   │ → Validates with 5 models
└────────┬───────────────┘
         ↓
┌────────────────────────┐
│  Proof Validator       │ → Checks mathematical validity
└────────┬───────────────┘
         ↓
┌────────────────────────┐
│  Final Answer          │
└────────────────────────┘
```

### 4.3 Redis Coordination
Use Redis pub/sub for agent communication:

```
Channels:
- mdap:coordinator:commands
- mdap:specialist:algebra:tasks
- mdap:specialist:geometry:tasks
- mdap:validator:tasks
- mdap:results

Message Format:
{
  task_id: uuid,
  problem: string,
  techniques: Technique[],
  context: object,
  deadline_ms: number
}
```

---

## Phase 5: Novel Problem Strategies

### 5.1 Unknown Technique Detection
**When**: Semantic search returns < 40% similarity for all techniques

**Strategy**:
1. Fall back to general problem-solving heuristics
2. Try multiple approaches in parallel
3. Use strongest models (GPT-4, Claude)
4. Request human guidance for technique identification

### 5.2 Technique Synthesis
**Purpose**: Combine existing techniques in novel ways

**Example**:
```
Problem requires: Geometry + Number Theory (rare combination)
→ Find best geometry technique
→ Find best number theory technique
→ Synthesizer agent creates bridge between them
```

### 5.3 Learning from Failures
**When**: Problem not solved after all attempts

**Action**:
1. Save problem + attempted techniques
2. Flag for human expert review
3. Expert adds new technique or technique chain
4. Re-run batch extraction on similar problems
5. Update technique library

---

## Phase 6: Performance Optimizations

### 6.1 Caching
- Cache semantic embeddings for common problem patterns
- Cache technique chains for problem types
- Cache model responses for similar sub-problems

### 6.2 Cost Management
```
Tier 1 (Cheap): Groq Llama 3.3 70B - Quick first pass
Tier 2 (Medium): GPT-4o-mini, Gemini Flash - Validation
Tier 3 (Expensive): GPT-4, Claude - Hard problems only
```

**Decision Logic**:
- If Tier 1 gets 3+ agents agreeing → Done ($0.01)
- If Tier 1 uncertain → Escalate to Tier 2 ($0.10)
- If Tier 2 split → Escalate to Tier 3 ($1.00)

### 6.3 Parallel Execution
- Run specialist solvers in parallel
- Run validation models in parallel
- Use async/await for all API calls

**Expected Speedup**: 5-10x vs sequential

---

## Implementation Roadmap

### Sprint 1: Technique Graph (Week 1)
- [ ] Mine co-occurrence patterns from batch results
- [ ] Build technique dependency graph
- [ ] Visualize technique relationships
- [ ] Identify common chains (top 50)

**Deliverables**:
- `data/technique-graph.json`
- `scripts/build-technique-graph.ts`
- `scripts/visualize-graph.ts`

### Sprint 2: Chain Planning (Week 2)
- [ ] Implement chain discovery algorithm
- [ ] Build sequential solver
- [ ] Add backtracking logic
- [ ] Test on 20 AIME problems

**Deliverables**:
- `agents/technique-chain-planner.ts`
- `agents/sequential-technique-solver.ts`
- Test suite with 20 problems

### Sprint 3: Validation Layer (Week 3)
- [ ] Multi-model consensus system
- [ ] Proof validator
- [ ] Answer verification
- [ ] Confidence scoring

**Deliverables**:
- `agents/consensus-validator.ts`
- `agents/proof-validator.ts`
- Validation metrics dashboard

### Sprint 4: MDAP Integration (Week 4)
- [ ] Problem decomposer
- [ ] Specialist solvers (4 categories)
- [ ] Solution synthesizer
- [ ] Redis coordination

**Deliverables**:
- Full MDAP pipeline
- 6+ specialized agents
- Coordinator orchestration

### Sprint 5: Testing & Optimization (Week 5)
- [ ] Test on 50 USAMO problems
- [ ] Benchmark performance
- [ ] Optimize costs
- [ ] Add caching

**Deliverables**:
- Performance benchmarks
- Cost analysis
- Optimization report

### Sprint 6: Novel Problem Handling (Week 6)
- [ ] Unknown technique detection
- [ ] Technique synthesis
- [ ] Learning from failures
- [ ] Human-in-the-loop workflow

**Deliverables**:
- Novel problem solver
- Feedback loop system
- Expert review interface

---

## Success Metrics

### Correctness
- **Target**: 70% correct on AIME problems
- **Stretch**: 50% correct on USAMO problems
- **Measurement**: Compare to official answers

### Speed
- **Target**: < 2 minutes per AIME problem
- **Stretch**: < 5 minutes per USAMO problem
- **Measurement**: End-to-end latency

### Cost
- **Target**: < $0.50 per AIME problem (avg)
- **Stretch**: < $2.00 per USAMO problem (avg)
- **Measurement**: Total API costs / problems solved

### Coverage
- **Target**: 80% of problems use technique library
- **Stretch**: 90% with technique chaining
- **Measurement**: % problems where library techniques helped

---

## Risk Mitigation

### Risk 1: Technique chains too complex
**Mitigation**: Limit chain depth to 3-4 techniques, use simplest chain first

### Risk 2: Models hallucinate solutions
**Mitigation**: Require 3+ model consensus, proof validation mandatory

### Risk 3: Costs spiral out of control
**Mitigation**: Hard cap at $5/problem, use cheap models first, cache aggressively

### Risk 4: Novel problems can't be solved
**Mitigation**: Human-in-the-loop fallback, technique learning pipeline

---

## Technical Stack

**Languages**: TypeScript, Python (for graph analysis)
**Databases**: SQLite (techniques), Redis (coordination)
**APIs**: OpenAI, Anthropic, Groq, DeepSeek, Google
**Frameworks**: Node.js, better-sqlite3, ioredis
**Visualization**: D3.js for technique graphs

---

## Estimated Costs

### Development (One-Time)
- Technique graph mining: $0 (CPU only)
- Chain testing (20 problems): ~$2
- Validation testing (50 problems): ~$25
- USAMO benchmarking (50 problems): ~$100
- **Total Development**: ~$130

### Production (Per Problem)
- Tier 1 attempt: $0.01
- Tier 2 validation: $0.10 (30% of problems)
- Tier 3 hard problems: $1.00 (10% of problems)
- **Average**: $0.14 per problem

### Monthly (1000 problems)
- **Estimated**: $140/month

---

## Expected Outcomes

After 6 sprints:
1. ✅ Working MDAP system for olympiad math
2. ✅ 70%+ accuracy on AIME problems
3. ✅ < $0.50 average cost per problem
4. ✅ Technique chaining for complex problems
5. ✅ Multi-model consensus validation
6. ✅ Learning pipeline for novel techniques

**This positions you to**:
- Tackle novel research-level math problems
- Build a math tutoring system
- Create a competition math training platform
- Extend to other STEM domains (physics, chemistry)

---

## Next Steps

1. **Approve Plan**: Review and approve this roadmap
2. **Sprint 1**: Start with technique graph mining (minimal cost)
3. **Validate Early**: Test chain planning on 5 problems before building full system
4. **Iterate**: Adjust based on initial results

---

**Status**: 📋 Planning Complete - Ready for Implementation
**Investment**: $4.30 (technique library) + $130 (development) = **$134.30 total**
**ROI**: Platform for solving novel olympiad problems + technique for research-level math