LLMRegistry

This page documents the LLMRegistry, which manages provider configurations and enables multi-provider setups. The registry provides a central location for storing API credentials and default provider selection.

Registry Overview

┌─────────────────────────────────────────────────────────────────┐
│                    AgentConfig                                   │
│  Loads provider configurations from YAML/env                    │
└─────────────────────────────────┬───────────────────────────────┘
                                  │
                                  ▼
┌─────────────────────────────────────────────────────────────────┐
│                    LLMRegistry                                   │
│  - Stores LLMSessionConfig entries                              │
│  - Tracks default provider                                       │
│  - Provides config lookup                                        │
└─────────────────────────────────┬───────────────────────────────┘
                                  │
                                  ▼
┌─────────────────────────────────────────────────────────────────┐
│                    AgentAir                                     │
│  - Creates sessions from registry configs                       │
│  - Supports switching between providers                          │
└─────────────────────────────────────────────────────────────────┘

LLMRegistry Structure

The registry stores provider configurations:

pub struct LLMRegistry {
    configs: Vec<LLMSessionConfig>,
    default_index: usize,
}
FieldTypePurpose
configsVec<LLMSessionConfig>All registered provider configurations
default_indexusizeIndex of the default provider

Creating a Registry

Empty Registry

impl LLMRegistry {
    pub fn new() -> Self {
        Self {
            configs: Vec::new(),
            default_index: 0,
        }
    }
}

From Configuration

impl LLMRegistry {
    pub fn from_configs(configs: Vec<LLMSessionConfig>) -> Self {
        Self {
            configs,
            default_index: 0,
        }
    }
}

Registering Providers

Add configurations to the registry:

impl LLMRegistry {
    pub fn register(&mut self, config: LLMSessionConfig) {
        self.configs.push(config);
    }

    pub fn register_default(&mut self, config: LLMSessionConfig) {
        self.default_index = self.configs.len();
        self.configs.push(config);
    }
}

Provider Lookup

Get Default Provider

impl LLMRegistry {
    pub fn get_default(&self) -> Option<&LLMSessionConfig> {
        self.configs.get(self.default_index)
    }
}

Get Provider by Index

impl LLMRegistry {
    pub fn get(&self, index: usize) -> Option<&LLMSessionConfig> {
        self.configs.get(index)
    }
}

Get Provider by Name

impl LLMRegistry {
    pub fn get_by_model(&self, model: &str) -> Option<&LLMSessionConfig> {
        self.configs.iter().find(|c| c.model == model)
    }

    pub fn get_by_provider(&self, provider: LLMProvider) -> Option<&LLMSessionConfig> {
        self.configs.iter().find(|c| c.provider == provider)
    }
}

Setting Default Provider

Change the default provider:

impl LLMRegistry {
    pub fn set_default(&mut self, index: usize) -> bool {
        if index < self.configs.len() {
            self.default_index = index;
            true
        } else {
            false
        }
    }

    pub fn set_default_by_model(&mut self, model: &str) -> bool {
        if let Some(index) = self.configs.iter().position(|c| c.model == model) {
            self.default_index = index;
            true
        } else {
            false
        }
    }
}

Iterating Providers

Access all registered providers:

impl LLMRegistry {
    pub fn iter(&self) -> impl Iterator<Item = &LLMSessionConfig> {
        self.configs.iter()
    }

    pub fn len(&self) -> usize {
        self.configs.len()
    }

    pub fn is_empty(&self) -> bool {
        self.configs.is_empty()
    }
}

Usage in AgentAir

The registry is used during session creation:

impl AgentAir {
    pub fn create_initial_session(&mut self) -> Result<(i64, String, i32), AgentError> {
        // Get registry
        let registry = self.llm_registry.as_ref()
            .ok_or_else(|| AgentError::NoConfiguration(
                "No LLM configuration found".to_string()
            ))?;

        // Get default provider config
        let config = registry.get_default()
            .ok_or_else(|| AgentError::NoConfiguration(
                "No default provider configured".to_string()
            ))?;

        // Create session
        let session_id = self.create_session(config.clone())?;

        Ok((session_id, config.model.clone(), config.context_limit))
    }
}

Multi-Provider Configuration

Register multiple providers for switching:

let mut registry = LLMRegistry::new();

// Register Anthropic as default
registry.register_default(
    LLMSessionConfig::anthropic(&anthropic_key, "claude-sonnet-4-20250514")
        .with_system_prompt("You are a helpful assistant.")
);

// Register OpenAI as backup
registry.register(
    LLMSessionConfig::openai(&openai_key, "gpt-4o")
        .with_system_prompt("You are a helpful assistant.")
);

// Register Claude for complex tasks
registry.register(
    LLMSessionConfig::anthropic(&anthropic_key, "claude-opus-4-20250514")
        .with_system_prompt("You are an expert analyst.")
        .with_max_tokens(8192)
);

Provider Switching

Switch between providers at runtime:

// Create session with default provider
let session1 = core.create_initial_session()?;

// Switch to different provider
let registry = core.llm_registry.as_ref().unwrap();
if let Some(gpt4_config) = registry.get_by_model("gpt-4o") {
    let session2 = core.create_session(gpt4_config.clone())?;
}

// Switch to high-capability model
if let Some(opus_config) = registry.get_by_model("claude-opus-4-20250514") {
    let session3 = core.create_session(opus_config.clone())?;
}

Configuration Loading

The registry is typically populated from configuration:

impl AgentConfig for MyAgent {
    fn load_llm_registry(&self) -> Option<LLMRegistry> {
        let mut registry = LLMRegistry::new();

        // Load from environment
        if let Ok(api_key) = std::env::var("ANTHROPIC_API_KEY") {
            registry.register_default(
                LLMSessionConfig::anthropic(&api_key, "claude-sonnet-4-20250514")
            );
        }

        if let Ok(api_key) = std::env::var("OPENAI_API_KEY") {
            registry.register(
                LLMSessionConfig::openai(&api_key, "gpt-4o")
            );
        }

        if registry.is_empty() {
            None
        } else {
            Some(registry)
        }
    }
}

YAML Configuration

Load providers from YAML:

providers:
  - name: anthropic-sonnet
    provider: anthropic
    model: claude-sonnet-4-20250514
    api_key_env: ANTHROPIC_API_KEY
    max_tokens: 4096
    default: true

  - name: openai-gpt4
    provider: openai
    model: gpt-4o
    api_key_env: OPENAI_API_KEY
    max_tokens: 4096

  - name: anthropic-opus
    provider: anthropic
    model: claude-opus-4-20250514
    api_key_env: ANTHROPIC_API_KEY
    max_tokens: 8192

Error Handling

Registry operations handle missing providers:

// Safe access with Option
if let Some(config) = registry.get_default() {
    let session = core.create_session(config.clone())?;
} else {
    return Err(AgentError::NoConfiguration("No providers configured".into()));
}

// Check before switching
if !registry.set_default_by_model("gpt-4o") {
    tracing::warn!("Provider gpt-4o not found in registry");
}

Registry Inspection

List available providers:

fn list_providers(registry: &LLMRegistry) {
    println!("Available providers:");
    for (i, config) in registry.iter().enumerate() {
        let default = if i == registry.default_index { " (default)" } else { "" };
        println!("  {} - {:?} / {}{}", i, config.provider, config.model, default);
    }
}

Example output:

Available providers:
  0 - Anthropic / claude-sonnet-4-20250514 (default)
  1 - OpenAI / gpt-4o
  2 - Anthropic / claude-opus-4-20250514

Thread Safety

The registry is typically stored in AgentAir:

pub struct AgentAir {
    llm_registry: Option<LLMRegistry>,
    // ...
}

For concurrent access, wrap in appropriate synchronization:

// Read-only access after initialization
let config = self.llm_registry.as_ref()?.get_default()?;

// For runtime modifications, use RwLock
pub struct AgentAir {
    llm_registry: RwLock<Option<LLMRegistry>>,
}

Next Steps