# {{Agent Name}} Voice Agent Configuration

[[LLM: Initial Setup

1. Replace {{Agent Name}} with the actual agent name
2. Choose voice platform (Gemini Live, OpenAI Realtime, ElevenLabs, etc.)
3. Define voice interaction patterns and personas
4. Configure audio processing pipeline
5. Set up multimodal capabilities

Output file location: `config/voice-agents/{{agent-name}}-voice-config.md`]]

## Voice Agent Overview

[[LLM: Provide a comprehensive overview of the voice agent's purpose, capabilities, and user experience goals.]]

- **Agent Name**: {{agent-name}}
- **Voice Platform**: {{Gemini-Live/OpenAI-Realtime/Custom}}
- **Interaction Mode**: {{Voice-only/Voice+Text/Multimodal}}
- **Primary Use Case**: {{education/customer-service/companion}}
- **Target Audience**: {{user-demographics}}

## Voice Configuration

### Voice Profile

[[LLM: Define the agent's voice characteristics and personality.]]

#### Voice Selection

- **Voice Model**: {{voice-id-or-name}}
- **Gender**: {{male/female/neutral}}
- **Age Range**: {{young/middle-aged/mature}}
- **Accent/Language**: {{en-US/en-GB/other}}
- **Speaking Style**: {{formal/casual/friendly/professional}}

#### Prosody Settings

````yaml
voice_parameters:
  speaking_rate:
    '[object Object]': null
  pitch:
    '[object Object]': null
  volume:
    '[object Object]': null
  emphasis_level:
    '[object Object]': null
  emotion_settings:
    default_emotion:
      '[object Object]': null
    emotion_range:
      '[object Object]': null
    context_awareness:
      '[object Object]': null
```text

### Audio Processing

#### Input Configuration
```yaml
audio_input:
  sample_rate:
    '[object Object]': null
  encoding:
    '[object Object]': null
  channels:
    '[object Object]': null
  preprocessing:
    noise_reduction:
      '[object Object]': null
    echo_cancellation:
      '[object Object]': null
    automatic_gain_control:
      '[object Object]': null
    voice_activity_detection:
      enabled: true
      sensitivity:
        '[object Object]': null
      silence_threshold:
        '[object Object]': null
````

#### Output Configuration

````yaml
audio_output:
  format:
    '[object Object]': null
  sample_rate:
    '[object Object]': null
  bitrate:
    '[object Object]': null
  enhancements:
    dynamic_range_compression:
      '[object Object]': null
    equalization_preset:
      '[object Object]': null
    spatial_audio:
      '[object Object]': null
```text

## Conversation Design

### Turn-Taking Management

[[LLM: Define how the agent handles conversation flow in voice interactions.]]

#### Interruption Handling
```yaml
interruption_policy:
  allow_interruptions:
    '[object Object]': null
  interruption_threshold:
    '[object Object]': null
  on_interruption:
    - pause_immediately:
        '[object Object]': null
    - save_context: true
    - acknowledgment: '{{I heard you, go ahead}}'
    - resume_strategy:
        '[object Object]': null
````

#### Silence Management

````yaml
silence_handling:
  end_of_turn_silence:
    '[object Object]': null
  thinking_silence_filler:
    '[object Object]': null
  filler_phrases:
    - Let me think about that...
    - Hmm, that's interesting...
    - Give me a moment...
  max_silence_before_prompt:
    '[object Object]': null
  silence_prompt: '{{Are you still there?}}'
```text

### Voice-Specific Prompting

#### System Prompt for Voice
````

You are {{agent-name}}, a voice-based LLM assistant. Your responses should be:

1. **Conversational**: Use natural speech patterns, contractions, and casual language
2. **Concise**: Keep responses brief and to the point for voice interaction
3. **Clear**: Avoid complex sentences or jargon that's hard to understand when heard
4. **Interactive**: Ask clarifying questions and confirm understanding
5. **Contextual**: Remember this is a voice conversation, reference sounds or verbal cues

Special instructions:

- {{voice-specific-behavior-guidelines}}
- {{personality-traits-for-voice}}
- {{domain-specific-knowledge}}

Remember: Users can't see text, so spell out acronyms, provide verbal descriptions, and use voice-appropriate formatting.

````text

#### Response Formatting
```yaml
voice_formatting:
  numbers: spell_out_below_10
  urls: provide_verbal_description
  lists: use_verbal_markers
  emphasis: use_prosody_not_caps
  code_handling:
    strategy:
      '[object Object]': null
    explanation_style:
      '[object Object]': null
````

## Multimodal Capabilities

### Voice + Visual Integration

[[LLM: For agents that combine voice with visual elements.]]

````yaml
multimodal_config:
  screen_sharing:
    enabled:
      '[object Object]': null
    annotations:
      '[object Object]': null
    pointer_control:
      '[object Object]': null
  visual_references:
    description_style:
      '[object Object]': null
    spatial_awareness:
      '[object Object]': null
    gesture_recognition:
      '[object Object]': null
  document_interaction:
    read_aloud:
      '[object Object]': null
    summarization:
      '[object Object]': null
    navigation_commands:
      - next
      - previous
      - go to
```text

### Context Switching
```yaml
modality_switching:
  voice_to_text:
    trigger:
      '[object Object]': null
    preservation: maintain_conversation_history
    notification: '{{Switching to text mode}}'
  text_to_voice:
    trigger:
      '[object Object]': null
    warm_start: true
    greeting: '{{Let''s continue our conversation}}'
````

## Educational Features (Example: Tutor Agent)

### Learning Interaction Patterns

[[LLM: Define specialized features for educational voice agents.]]

````yaml
educational_config:
  questioning_strategies:
    socratic_method: true
    wait_time_after_question:
      '[object Object]': null
    encouragement_phrases:
      - Take your time thinking about it
      - That's a great start, what else?
      - You're on the right track
  feedback_delivery:
    positive_reinforcement: immediate
    correction_style: gentle_guidance
    explanation_depth: adaptive
  pacing:
    adaptive_speed: true
    comprehension_checks: true
    break_reminders: every_20_minutes
```text

### Progress Tracking
```yaml
learning_metrics:
  track_understanding: true
  vocabulary_growth: true
  concept_mastery: true
  verbal_assessments:
    quiz_format: conversational
    self_assessment_prompts: true
    progress_summaries: daily
````

## Platform-Specific Configuration

### Gemini Live Integration

[[LLM: Specific configuration for Gemini Live API.]]

````python
# Gemini Live configuration example
gemini_config = {
    "model": "gemini-live",
    "voice_settings": {
        "voice_name": "{{voice-selection}}",
        "streaming": True,
        "audio_config": {
            "encoding": "LINEAR16",
            "sample_rate_hertz": 16000
        }
    },
    "interaction_config": {
        "enable_interruptions": True,
        "live_transcription": True,
        "emotion_detection": True
    }
}
```text

### OpenAI Realtime API
```python
# OpenAI Realtime configuration
openai_config = {
    "model": "gpt-4-realtime",
    "voice": "{{alloy/echo/fable/onyx/nova/shimmer}}",
    "response_format": {
        "type": "audio",
        "voice_settings": {
            "speed": {{0.25-4.0}},
            "temperature": {{0-1}}
        }
    }
}
````

## Safety and Moderation

### Voice-Specific Safety

[[LLM: Address unique safety concerns for voice interactions.]]

````yaml
voice_safety:
  content_filtering:
    profanity_filter:
      '[object Object]': null
    inappropriate_content: block_and_redirect
  impersonation_prevention:
    verify_ai_identity: true
    disclosure_frequency: every_session
    cannot_pretend_to_be:
      - human
      - specific_person
      - authority
  emotional_safety:
    detect_distress: true
    emergency_protocols: true
    supportive_responses: true
```text

### Privacy Protection
```yaml
privacy_config:
  audio_retention:
    store_audio:
      '[object Object]': null
    retention_period:
      '[object Object]': null
  transcription_handling:
    store_transcripts:
      '[object Object]': null
    anonymization: true
    pii_detection: true
  voice_biometrics:
    collect_voiceprint: false
    speaker_identification: disabled
````

## Performance Optimization

### Latency Optimization

[[LLM: Configure for optimal voice interaction latency.]]

````yaml
performance_tuning:
  streaming_config:
    chunk_size:
      '[object Object]': null
    buffer_size:
      '[object Object]': null
  prefetching:
    common_responses: true
    contextual_predictions: true
  caching:
    tts_cache: true
    response_cache: true
    cache_ttl:
      '[object Object]': null
```text

### Resource Management
```yaml
resource_limits:
  concurrent_sessions:
    '[object Object]': null
  session_timeout:
    '[object Object]': null
  compute_allocation:
    cpu_cores:
      '[object Object]': null
    memory:
      '[object Object]': null
    gpu:
      '[object Object]': null
  bandwidth_management:
    audio_codec:
      '[object Object]': null
    adaptive_bitrate: true
    minimum_quality:
      '[object Object]': null
````

## Integration Examples

### Web Application Integration

````javascript
// Browser-based voice agent integration
class VoiceAgent {
  constructor(config) {
    this.config = config;
    this.mediaStream = null;
    this.recognition = new webkitSpeechRecognition();
    this.synthesis = window.speechSynthesis;
  }

  async startConversation() {
    // Initialize audio context
    const audioContext = new AudioContext();

    // Configure speech recognition
    this.recognition.continuous = true;
    this.recognition.interimResults = true;

    // Start listening
    this.recognition.start();
  }
}
```text

### Mobile App Integration
```swift
// iOS voice agent example
class VoiceAgentManager {
    let speechRecognizer = SFSpeechRecognizer()
    let audioEngine = AVAudioEngine()

    func configureVoiceInteraction() {
        // Configure audio session
        let audioSession = AVAudioSession.sharedInstance()
        try audioSession.setCategory(.playAndRecord)

        // Set up voice processing
        // ... implementation details
    }
}
````

## Testing and Validation

### Voice Quality Testing

````yaml
test_scenarios:
  acoustic_conditions:
    - quiet_room
    - background_noise
    - echo_environment
    - multiple_speakers
  speech_variations:
    - different_accents
    - speech_impediments
    - children_voices
    - elderly_speakers
  interaction_patterns:
    - rapid_exchanges
    - long_pauses
    - interruptions
    - parallel_talking
```text

### Performance Benchmarks

- **First Token Latency**: < {{200ms}}
- **End-to-End Latency**: < {{500ms}}
- **Speech Recognition Accuracy**: > {{95%}}
- **Natural Conversation Flow**: {{subjective-rating}}

## Monitoring and Analytics

### Voice-Specific Metrics

```yaml
voice_analytics:
  conversation_metrics:
    - average_turn_duration
    - interruption_rate
    - silence_ratio
    - conversation_completion_rate
  quality_metrics:
    - audio_quality_score
    - recognition_confidence
    - tts_naturalness_rating
    - user_satisfaction_score
  technical_metrics:
    - audio_packet_loss
    - jitter_measurements
    - bandwidth_usage
    - processing_latency
````

### User Feedback Collection

```yaml
feedback_system:
  implicit_signals:
    - conversation_length
    - return_rate
    - task_completion
  explicit_feedback:
    - voice_quality_rating
    - understanding_rating
    - helpfulness_score
    - would_recommend
```

## Deployment Checklist

- [ ] Voice model selected and tested
- [ ] Audio pipeline configured
- [ ] Latency targets met
- [ ] Safety measures implemented
- [ ] Privacy compliance verified
- [ ] Integration tested across platforms
- [ ] Monitoring dashboard configured
- [ ] Feedback system active
- [ ] Documentation complete
- [ ] User training materials ready