Contributing New LLM Providers

Guide for Adding New LLM Providers to Paladin

This guide walks you through implementing a new LLM provider adapter for Paladin. All providers implement the LlmPort trait, ensuring consistent behavior across the framework.

Table of Contents

• Prerequisites
• Implementation Steps
• Adapter Template
• Testing Requirements
• Documentation Requirements
• Submission Guidelines

Prerequisites

Before implementing a new provider:

1. API Documentation: Have access to the provider's API documentation
2. API Key: Obtain an API key for testing
3. Rust Knowledge: Familiarity with async Rust and the tokio runtime
4. Project Setup: Clone and build the Paladin project

Implementation Steps

Step 1: Create Adapter File

Create a new file in src/infrastructure/adapters/llm/:

touch src/infrastructure/adapters/llm/myprovider_adapter.rs

Step 2: Define Configuration Struct

#![allow(unused)]
fn main() {
use serde::{Deserialize, Serialize};

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct MyProviderConfig {
    /// API key for authentication
    pub api_key: String,
    /// Base URL for API
    pub base_url: String,
    /// Default model to use
    pub model: String,
    /// Request timeout in seconds
    pub timeout_seconds: u64,
}

impl MyProviderConfig {
    /// Load configuration from environment variables
    pub fn from_env() -> Result<Self, String> {
        let api_key = std::env::var("MYPROVIDER_API_KEY")
            .map_err(|_| "MYPROVIDER_API_KEY not set")?;

        let base_url = std::env::var("MYPROVIDER_BASE_URL")
            .unwrap_or_else(|_| "https://api.myprovider.com/v1".to_string());

        let model = std::env::var("MYPROVIDER_MODEL")
            .unwrap_or_else(|_| "default-model".to_string());

        let timeout_seconds = 60;

        Ok(Self {
            api_key,
            base_url,
            model,
            timeout_seconds,
        })
    }

    /// Create custom configuration
    pub fn new(api_key: String, base_url: String, model: String) -> Self {
        Self {
            api_key,
            base_url,
            model,
            timeout_seconds: 60,
        }
    }

    fn validate(&self) -> Result<(), String> {
        if self.api_key.is_empty() {
            return Err("API key cannot be empty".to_string());
        }
        if !self.base_url.starts_with("http") {
            return Err("Base URL must start with http/https".to_string());
        }
        Ok(())
    }
}
}

Step 3: Implement Adapter Struct

#![allow(unused)]
fn main() {
use crate::paladin_ports::output::llm_port::{
    LlmError, LlmPort, LlmRequest, LlmResponse, ProviderCapabilities
};
use async_trait::async_trait;
use reqwest::{Client, header::{HeaderMap, HeaderValue, AUTHORIZATION, CONTENT_TYPE}};
use std::time::Duration;

pub struct MyProviderAdapter {
    client: Client,
    config: MyProviderConfig,
}

impl MyProviderAdapter {
    pub fn new(config: MyProviderConfig) -> Result<Self, LlmError> {
        config.validate()
            .map_err(|e| LlmError::AuthenticationError(e))?;

        let timeout = Duration::from_secs(config.timeout_seconds);

        let mut headers = HeaderMap::new();
        headers.insert(CONTENT_TYPE, HeaderValue::from_static("application/json"));
        headers.insert(
            AUTHORIZATION,
            HeaderValue::from_str(&format!("Bearer {}", config.api_key))
                .map_err(|e| LlmError::AuthenticationError(e.to_string()))?
        );

        let client = Client::builder()
            .timeout(timeout)
            .default_headers(headers)
            .build()
            .map_err(|e| LlmError::ProviderError(e.to_string()))?;

        Ok(Self { client, config })
    }
}
}

Step 4: Implement LlmPort Trait

#![allow(unused)]
fn main() {
#[async_trait]
impl LlmPort for MyProviderAdapter {
    async fn generate(&self, request: &LlmRequest) -> Result<LlmResponse, LlmError> {
        // 1. Build provider-specific request
        let provider_request = self.build_request(request)?;

        // 2. Make HTTP request with retry logic
        let response = self.make_request(provider_request).await?;

        // 3. Parse and convert to LlmResponse
        self.parse_response(response, request).await
    }

    async fn generate_stream(
        &self,
        request: &LlmRequest,
    ) -> Result<Pin<Box<dyn Stream<Item = Result<StreamChunk, LlmError>> + Send>>, LlmError> {
        // Implement SSE streaming if supported
        unimplemented!("Streaming not yet implemented")
    }

    fn get_capabilities(&self) -> ProviderCapabilities {
        ProviderCapabilities {
            supports_streaming: true,  // Set based on provider
            supports_tool_calling: true,
            supports_function_calling: true,
            supports_vision: false,  // Set based on provider
            supports_embeddings: false,
            max_context_tokens: Some(128_000),  // Provider's limit
            supports_system_messages: true,
        }
    }

    fn get_provider_name(&self) -> String {
        "myprovider".to_string()
    }

    async fn validate_model(&self, model: &str) -> Result<bool, LlmError> {
        let available = self.get_available_models().await?;
        Ok(available.contains(&model.to_string()))
    }

    async fn get_available_models(&self) -> Result<Vec<String>, LlmError> {
        Ok(vec![
            "model-1".to_string(),
            "model-2".to_string(),
            // Add provider's models
        ])
    }
}
}

Step 5: Add to Module

Update src/infrastructure/adapters/llm/mod.rs:

#![allow(unused)]
fn main() {
pub mod myprovider_adapter;
}

Step 6: Update Provider Factory

Add to src/infrastructure/adapters/llm/provider_factory.rs:

#![allow(unused)]
fn main() {
"myprovider" => {
    let config = MyProviderConfig::from_env()
        .map_err(|e| LlmError::ConfigurationError(e))?;
    Ok(Arc::new(MyProviderAdapter::new(config)?))
}
}

Adapter Template

See adapter_template.rs for a complete template with:

• Full error handling
• Retry logic with exponential backoff
• Request/response serialization
• SSE streaming implementation
• Comprehensive documentation

Testing Requirements

Unit Tests (Required)

Create tests/unit/llm/myprovider_adapter_test.rs:

#![allow(unused)]
fn main() {
use mockito::Server;
use paladin::infrastructure::adapters::llm::myprovider_adapter::*;

#[tokio::test]
async fn test_successful_completion() {
    let mut server = Server::new_async().await;

    let mock = server.mock("POST", "/v1/completions")
        .with_status(200)
        .with_body(r#"{"response": "test"}"#)
        .create_async()
        .await;

    let config = MyProviderConfig::new(
        "test-key".to_string(),
        server.url(),
        "test-model".to_string()
    );

    let adapter = MyProviderAdapter::new(config).unwrap();
    // Test adapter functionality

    mock.assert_async().await;
}

#[tokio::test]
async fn test_authentication_error() {
    // Test 401 handling
}

#[tokio::test]
async fn test_rate_limiting() {
    // Test 429 handling
}

// Add tests for all error cases and success paths
}

Required test coverage:

• ✅ Successful completion
• ✅ Streaming responses
• ✅ Authentication errors (401)
• ✅ Rate limiting (429)
• ✅ Timeouts
• ✅ Invalid model errors
• ✅ Malformed responses

Integration Tests (Optional)

Create tests/integration/llm/myprovider_integration_test.rs with tests marked #[ignore] for live API testing.

Documentation Requirements

1. Rustdoc Comments

Add comprehensive rustdoc to all public items:

#![allow(unused)]
fn main() {
/// MyProvider LLM adapter
///
/// Implements the LlmPort trait for MyProvider's API.
///
/// # Examples
///
/// ```no_run
/// use paladin::infrastructure::adapters::llm::myprovider_adapter::*;
///
/// let config = MyProviderConfig::from_env()?;
/// let adapter = MyProviderAdapter::new(config)?;
/// ```
pub struct MyProviderAdapter {
    // ...
}
}

2. Configuration Guide

Add section to docs/PROVIDER_EXPANSION.md:

• Configuration examples
• Use case recommendations
• Pricing information
• Performance characteristics

3. Example Code

Create examples/myprovider_example.rs demonstrating usage.

Submission Guidelines

Checklist

Before submitting a pull request:

• Adapter implements all LlmPort trait methods
• Configuration struct with from_env() and validation
• Unit tests with ≥80% coverage
• All tests passing (cargo test)
• Code formatted (cargo fmt)
• No clippy warnings (cargo clippy -- -D warnings)
• Rustdoc for all public items
• Added to provider factory
• Documentation updated
• Example code created

Pull Request Template

## New Provider: [Provider Name]

### Description
Brief description of the provider and its strengths.

### Changes
- [ ] Adapter implementation
- [ ] Unit tests (XX% coverage)
- [ ] Integration tests
- [ ] Documentation
- [ ] Examples

### Testing
- All unit tests passing
- Integration tests verified with API key
- Tested on: [OS/Platform]

### Documentation
- [ ] PROVIDER_EXPANSION.md updated
- [ ] Rustdoc complete
- [ ] Example added

### Checklist
- [ ] Follows project code style
- [ ] No breaking changes
- [ ] Backward compatible

Common Pitfalls

1. Incomplete Error Handling

❌ Bad:

#![allow(unused)]
fn main() {
let response = self.client.post(&url).send().await.unwrap();
}

✅ Good:

#![allow(unused)]
fn main() {
let response = self.client.post(&url)
    .send()
    .await
    .map_err(|e| LlmError::NetworkError(e.to_string()))?;
}

2. Missing Retry Logic

Implement exponential backoff for rate limits:

#![allow(unused)]
fn main() {
async fn make_request_with_retry(&self, request: Request) -> Result<Response, LlmError> {
    let mut attempt = 0;
    loop {
        match self.client.execute(request.try_clone()?).await {
            Ok(resp) if resp.status().is_success() => return Ok(resp),
            Ok(resp) if resp.status() == 429 => {
                attempt += 1;
                if attempt >= 3 {
                    return Err(LlmError::RateLimitExceeded { retry_after: 60 });
                }
                tokio::time::sleep(Duration::from_millis(1000 * 2u64.pow(attempt))).await;
            }
            Err(e) => return Err(LlmError::NetworkError(e.to_string())),
        }
    }
}
}

3. Hardcoded Values

Use configuration for all provider-specific values.

Getting Help

• GitHub Discussions: Ask questions
• Discord: Real-time community help
• GitHub Issues: Report bugs or request features

Happy Contributing! 🗡️

Thank you for helping expand Paladin's LLM provider ecosystem.