# NX SurrealDB Migrations Plugin - Architecture Documentation

## Overview

This plugin implements a comprehensive database migration system for SurrealDB within NX monorepos, following the **Repository Pattern** with clean separation of concerns and domain-driven design principles.

## Architectural Principles

### 1. Repository Pattern
- **Data Access Layer**: `MigrationRepository` handles all database operations
- **Business Logic Layer**: `MigrationService` orchestrates workflows and business rules
- **Clean Separation**: No business logic in data access, no data access in business logic

### 2. Domain-Driven Design
- **Infrastructure**: Database connectivity, utilities, external dependencies
- **Configuration**: Settings, types, environment management
- **Filesystem**: File operations, NX Tree integration
- **Domain**: Core business logic, migration workflows

### 3. Single Responsibility Principle
- Each class has one clear purpose and responsibility
- Methods are focused and cohesive
- Minimal coupling between components

## Directory Structure

```
src/lib/
├── infrastructure/          # External dependencies and utilities
│   ├── client.ts           # SurrealDB client wrapper
│   ├── debug.ts            # Debugging and logging utilities
│   ├── env.ts              # Environment variable handling
│   └── project.ts          # NX project path resolution
├── configuration/          # Configuration and types
│   ├── config-loader.ts    # Module configuration loading
│   └── types.ts           # TypeScript type definitions
├── filesystem/            # File system operations
│   ├── migration-file-processor.ts  # Migration file parsing/processing
│   └── tree-utils.ts                # NX Tree API utilities
└── domain/               # Core business logic
    ├── dependency-resolver.ts       # Module dependency management
    ├── migration-repository.ts      # Data access layer (Repository)
    └── migration-service.ts         # Business logic layer (Service)
```

## Component Responsibilities

### Infrastructure Layer

#### `SurrealDBClient`
- **Purpose**: Database connectivity abstraction
- **Responsibilities**:
  - Connection management (connect, disconnect)
  - Query execution with error handling
  - Transaction management
  - Result formatting

#### `Debug`
- **Purpose**: Centralized logging and debugging
- **Responsibilities**:
  - Scoped logging (per component)
  - Conditional debug output
  - Error logging and formatting

#### `env.ts`
- **Purpose**: Environment variable management
- **Responsibilities**:
  - Variable interpolation (`${VAR}` replacement)
  - `.env` file loading
  - Environment validation

#### `project.ts`
- **Purpose**: NX project integration
- **Responsibilities**:
  - Project path resolution
  - Workspace context handling

### Configuration Layer

#### `ConfigLoader`
- **Purpose**: Module configuration management
- **Responsibilities**:
  - Load and parse `config.json`/`config.yaml`
  - Configuration validation
  - Default value handling
  - Error reporting for malformed configs

#### `types.ts`
- **Purpose**: TypeScript type definitions
- **Responsibilities**:
  - Interface definitions for all data structures
  - Type safety across the application
  - Contract definitions between layers

### Filesystem Layer

#### `MigrationFileProcessor`
- **Purpose**: Migration file operations
- **Responsibilities**:
  - Parse migration filenames (`0001_name_up.surql`)
  - Read and process file contents
  - Content transformation (variable substitution)
  - Checksum generation
  - Migration file validation

#### `TreeUtils`
- **Purpose**: NX Tree API utilities
- **Responsibilities**:
  - Directory operations (find, create, list)
  - File operations (read, write, copy)
  - Module directory discovery
  - Migration number sequence management

### Domain Layer

#### `DependencyResolver`
- **Purpose**: Module dependency management
- **Responsibilities**:
  - Dependency graph construction
  - Topological sorting for execution order
  - Circular dependency detection
  - Rollback order calculation
  - Dependency conflict validation

#### `MigrationRepository` (Data Access Layer)
- **Purpose**: Database operations for migration state
- **Responsibilities**:
  - CRUD operations on `system_migrations` table
  - Migration status queries
  - Data validation (required fields, formats)
  - Raw database interactions
  - Schema initialization

**Methods:**
```typescript
// Simple database operations
async addMigration(record: MigrationRecord): Promise<void>
async getLatestMigrationStatus(number: string, name: string): Promise<Migration | null>
async findLastMigrations(moduleIds: string[]): Promise<Migration[]>
async getMigrationsByDirectionAndPath(direction: string, path: string): Promise<Migration[]>
async updateMigrationStatus(recordId: string, status: 'success' | 'fail'): Promise<void>
```

**What it should NOT do:**
- Business logic about when migrations can be applied
- File operations (reading schema files)
- Complex validation rules
- Workflow decisions

#### `MigrationService` (Business Logic Layer)
- **Purpose**: Migration workflow orchestration
- **Responsibilities**:
  - Coordinate between repository, resolver, and file processor
  - Complex business rules (migration applicability)
  - Workflow management (execution order, error handling)
  - High-level operations (migrate, rollback, status)
  - Transaction coordination

**Methods:**
```typescript
// Complex workflow and rules
async initialize(options: MigrationServiceOptions): Promise<void>
async executeMigrations(modules?: string[]): Promise<MigrationResult>
async validateRollback(modules: string[]): Promise<RollbackValidation>
async findPendingMigrations(modules?: string[]): Promise<MigrationFile[]>
async getMigrationStatus(modules?: string[]): Promise<StatusResult>
```

**What it should NOT do:**
- Direct database operations (use repository methods)
- Raw SQL construction

## Data Flow

### Migration Execution Flow

```
1. MigrationService.executeMigrations()
   ↓
2. findPendingMigrations()
   ├─ DependencyResolver.getExecutionOrder()
   ├─ MigrationFileProcessor.findModuleMigrations()
   └─ MigrationRepository.canApplyMigration()
   ↓
3. For each migration:
   ├─ MigrationFileProcessor.processContent()
   ├─ SurrealDBClient.query()
   └─ MigrationRepository.addMigration()
```

### Rollback Validation Flow

```
1. MigrationService.validateRollback()
   ↓
2. For each module:
   ├─ DependencyResolver.validateRollback()
   ├─ MigrationRepository.getAppliedMigrations()
   └─ MigrationFileProcessor.findModuleMigrations()
   ↓
3. Validate rollback safety
   ├─ Check dependency conflicts
   ├─ Verify rollback files exist
   └─ Return validation result
```

## Communication Patterns

### Layer Communication Rules

1. **Infrastructure ← All Layers**: Infrastructure can be used by any layer
2. **Configuration ← Domain/Filesystem**: Configuration used by domain and filesystem
3. **Filesystem ← Domain**: Filesystem utilities used by domain layer
4. **Domain → Repository → Database**: Business logic delegates to repository

### Forbidden Communications

- ❌ Repository calling Service methods (creates circular dependency)
- ❌ Service bypassing Repository for database operations
- ❌ Infrastructure depending on Domain logic
- ❌ Configuration containing business logic

## Error Handling Strategy

### Repository Layer
- **Database Errors**: Wrap in descriptive error messages
- **Validation Errors**: Throw with specific field information
- **Connection Errors**: Bubble up with context

### Service Layer
- **Business Logic Errors**: Detailed validation messages
- **Workflow Errors**: Comprehensive error context
- **Dependency Errors**: Clear conflict descriptions

### Error Propagation
```
Domain (Business Logic Errors)
    ↓ wraps/enriches
Repository (Data Errors)
    ↓ wraps/enriches
Infrastructure (Connection/System Errors)
```

## Testing Strategy

### Unit Testing
- **Repository**: Mock SurrealDBClient, test data operations
- **Service**: Mock Repository, test business logic
- **FileProcessor**: Mock filesystem, test parsing logic
- **DependencyResolver**: Test graph algorithms with fixtures

### Integration Testing
- **Repository + Database**: Real database operations
- **Service + Repository**: End-to-end workflow testing
- **File Operations**: Real file system interactions

### Test Isolation
- Each layer can be tested independently
- Clear interfaces enable easy mocking
- Repository pattern enables database abstraction

## Future Architecture Considerations

### Planned Refactoring
1. **Move `canApplyMigration()` from Repository to Service**
   - Business logic should be in Service layer
   - Repository should only handle data operations

2. **Extract file operations from Repository.initialize()**
   - File reading should be in Service layer
   - Repository should receive processed schema content

3. **Add schema initialization method to Repository**
   - `initializeSchema(content: string): Promise<void>`
   - Removes file dependency from Repository

### Extensibility Points
- **Plugin Architecture**: Could add migration plugins for different operations
- **Multiple Databases**: Repository pattern enables easy database swapping
- **Custom Validators**: Service layer can accommodate custom validation rules
- **Event System**: Could add migration events for monitoring/logging

## Performance Considerations

### Query Optimization
- **Batched Queries**: `findLastMigrations()` uses single query for multiple modules
- **Selective Fields**: Only query needed fields, not `SELECT *`
- **Indexed Queries**: Use database indexes for migration status queries

### Memory Management
- **Streaming**: Large migration files handled in streams
- **Lazy Loading**: Configuration loaded only when needed
- **Connection Pooling**: Reuse database connections

### Scalability
- **Horizontal Scaling**: Repository pattern enables connection pooling
- **Caching**: Configuration and dependency graphs can be cached
- **Parallel Execution**: Independent modules could run in parallel (future)

## Security Considerations

### Credential Management
- **Environment Variables**: Never hardcode credentials
- **Connection Encryption**: Always use secure connections in production
- **Access Control**: Repository validates all inputs

### SQL Injection Prevention
- **Parameterized Queries**: All queries use parameter binding
- **Input Validation**: Repository validates all inputs
- **Content Sanitization**: Migration content is validated before execution

### Error Information
- **Credential Hiding**: Errors never expose credentials
- **Sanitized Logging**: Debug logs exclude sensitive information

This architecture ensures maintainability, testability, and scalability while following industry best practices for database migration systems.