LLM Provider Router

Intelligent query classification and automatic provider selection system that routes requests to the most appropriate LLM provider based on query type and provider specialties.

Query Classification Smart Routing Auto-Execution Fallback System

What is LLM Provider Router?

LLM Provider Router is an intelligent orchestration system that automatically determines the best LLM provider for any given query. Instead of manually choosing which provider to use, it classifies your query type and routes it to the provider that specializes in that type of task.

Think of it as a smart dispatcher that understands your intent and sends your query to the right expert every time.

āœØ Why Use LLM Provider Router?

  • Automatic Provider Selection: No more guessing which provider is best for your task
  • Query Classification: Understands intent through pattern matching or AI analysis
  • Provider Specialization: Each provider configured with strengths and specialties
  • Intelligent Fallback: Multiple fallback layers ensure reliability
  • Performance Optimization: Optional caching for fast classification
  • Configuration Export: Save and reuse routing configurations

How It Works

LLM Provider Router follows a 4-step process to intelligently route your queries:

1

Classify

Determine query type (coding, creative, analysis, etc.) using patterns or AI

2

Match

Find providers whose specialties match the query type

3

Select

Choose best provider based on priority or semantic matching

4

Execute

Send query to selected provider and return response with metadata

šŸ“Š Example Flow

User Query: "Write a Python function to calculate factorial"

Step 1: CLASSIFY
  ā†’ Pattern match: r"write.*function|implement|code"
  ā†’ Query Type: "coding"
  ā†’ Confidence: 0.95 (matched by pattern)

Step 2: MATCH
  ā†’ Find providers with "coding" specialty
  ā†’ Found: OpenAI (gpt-4o-mini), Anthropic (claude-3-5-haiku)

Step 3: SELECT
  ā†’ Both match, use priority scores
  ā†’ OpenAI priority: 8
  ā†’ Anthropic priority: 7
  ā†’ Selected: OpenAI (gpt-4o-mini)

Step 4: EXECUTE
  ā†’ Send query to OpenAI
  ā†’ Receive response
  ā†’ Return RoutingResult with answer + metadata

Quick Start

Get started with LLM Provider Router in minutes: 

from SimplerLLM.language import (
    LLM, LLMProvider, LLMProviderRouter, ProviderConfig
)

# Step 1: Configure providers with specialties
provider_configs = [
    ProviderConfig(
        llm_provider="OPENAI",
        llm_model="gpt-4o-mini",
        specialties=["coding", "technical_writing", "data_analysis"],
        description="Best for programming and technical tasks",
        priority=8
    ),
    ProviderConfig(
        llm_provider="ANTHROPIC",
        llm_model="claude-3-5-haiku-20241022",
        specialties=["creative_writing", "analysis", "general"],
        description="Best for creative and analytical tasks",
        priority=7
    ),
]

# Step 2: Create LLM instances
llm_instances = [
    LLM.create(LLMProvider.OPENAI, model_name="gpt-4o-mini"),
    LLM.create(LLMProvider.ANTHROPIC, model_name="claude-3-5-haiku-20241022"),
]

# Step 3: Initialize router
router = LLMProviderRouter(
    provider_configs=provider_configs,
    llm_instances=llm_instances,
    classification_method="hybrid",  # Pattern + AI fallback
    enable_cache=True,                # Cache classifications
    verbose=True                      # Show routing decisions
)

# Step 4: Route and execute queries
result = router.route("Write a Python function to check if a number is prime")

# Access results
print(f"Answer: {result.answer}")
print(f"Provider Used: {result.provider_used}")
print(f"Query Type: {result.query_classification.query_type}")
print(f"Routing Confidence: {result.routing_confidence:.2%}")
print(f"Execution Time: {result.execution_time:.2f}s")

Query Classification Methods

LLM Provider Router supports three classification methods, each with different trade-offs:

āš” Pattern-Based

Uses regex patterns to match query types. Fast and cost-effective but less flexible.

Speed: Instant (~1ms)

Cost: Free (no LLM calls)

Accuracy: Good for clear queries

Best for: High-volume, cost-sensitive

šŸ¤– LLM-Based

Uses AI to classify queries. Most accurate but slower and requires LLM calls.

Speed: ~500ms-2s

Cost: ~$0.0001 per classification

Accuracy: Excellent

Best for: Complex, ambiguous queries

āœØ Hybrid

Recommended

Tries pattern matching first, falls back to LLM if confidence is low. Best of both worlds.

Speed: Instant-2s (adaptive)

Cost: Low (LLM only when needed)

Accuracy: High

Best for: Most use cases

Classification Examples

# Pattern-based classification (fast, free)
router = LLMProviderRouter(
    provider_configs=configs,
    llm_instances=llms,
    classification_method="pattern"
)

# LLM-based classification (accurate, slower)
router = LLMProviderRouter(
    provider_configs=configs,
    llm_instances=llms,
    classification_method="llm",
    classifier_llm=LLM.create(LLMProvider.OPENAI, model_name="gpt-4o-mini")
)

# Hybrid classification (recommended)
router = LLMProviderRouter(
    provider_configs=configs,
    llm_instances=llms,
    classification_method="hybrid",  # Default
    classifier_llm=LLM.create(LLMProvider.OPENAI, model_name="gpt-4o-mini"),
    confidence_threshold=0.8  # Use LLM if pattern confidence < 0.8
)

Default Query Types & Patterns

LLM Provider Router comes with built-in patterns for common query types:

Query Type Example Patterns Example Queries
coding write code, implement, debug, function, python, javascript "Write a function to sort arrays"
creative_writing write a story, poem, article, blog, creative, fiction "Write a short story about AI"
technical_explanation explain how, what is, how does, describe, overview "Explain how neural networks work"
analysis analyze, compare, evaluate, pros and cons, advantages "Compare React vs Vue.js"
data_analysis SQL, pandas, statistics, data, dataset, analyze data "Write SQL to find top customers"
summarization summarize, summary, key points, tldr, brief "Summarize this article"
translation translate, translation, convert to, in [language] "Translate this to Spanish"
general Catch-all for queries that don't match other patterns "What's the capital of France?"

Provider Configuration

Configure each provider with its specialties, priorities, and fallback options:

ProviderConfig Structure

ProviderConfig(
    llm_provider: str,                      # Provider name (e.g., "OPENAI", "ANTHROPIC")
    llm_model: str,                         # Model name (e.g., "gpt-4o-mini")
    specialties: List[str],                 # Query types this provider excels at
    description: str,                       # Provider strengths description
    priority: int = 5,                      # Priority score (1-10, higher = preferred)
    enabled: bool = True,                   # Enable/disable provider
    has_fallback: bool = False,             # Whether this provider has fallback
    fallback_provider: Optional[str] = None, # Fallback provider name
    fallback_model: Optional[str] = None    # Fallback model name
)

Complete Configuration Example

from SimplerLLM.language import LLMProviderRouter, ProviderConfig

# Configure multiple providers with specialties
provider_configs = [
    # OpenAI - Best for coding and technical tasks
    ProviderConfig(
        llm_provider="OPENAI",
        llm_model="gpt-4o-mini",
        specialties=["coding", "technical_writing", "data_analysis"],
        description="Excellent at code generation, debugging, and technical documentation",
        priority=8,
        has_fallback=True,
        fallback_provider="ANTHROPIC",
        fallback_model="claude-3-5-haiku-20241022"
    ),

    # Anthropic - Best for creative and analytical tasks
    ProviderConfig(
        llm_provider="ANTHROPIC",
        llm_model="claude-3-5-haiku-20241022",
        specialties=["creative_writing", "analysis", "summarization"],
        description="Best for creative content, detailed analysis, and thoughtful responses",
        priority=7,
        enabled=True
    ),

    # Gemini - Fast general-purpose provider
    ProviderConfig(
        llm_provider="GEMINI",
        llm_model="gemini-1.5-flash",
        specialties=["general", "translation", "technical_explanation"],
        description="Fast and efficient for general queries and explanations",
        priority=6
    ),

    # Cohere - Specialized for summarization
    ProviderConfig(
        llm_provider="COHERE",
        llm_model="command-r",
        specialties=["summarization", "general"],
        description="Efficient at summarization and general conversation",
        priority=5
    ),
]

# Create corresponding LLM instances
llm_instances = [
    LLM.create(LLMProvider.OPENAI, model_name="gpt-4o-mini"),
    LLM.create(LLMProvider.ANTHROPIC, model_name="claude-3-5-haiku-20241022"),
    LLM.create(LLMProvider.GEMINI, model_name="gemini-1.5-flash"),
    LLM.create(LLMProvider.COHERE, model_name="command-r"),
]

# Initialize router
router = LLMProviderRouter(
    provider_configs=provider_configs,
    llm_instances=llm_instances,
    classification_method="hybrid",
    enable_cache=True,
    cache_ttl=3600,  # Cache for 1 hour
    verbose=True
)

šŸ’” Priority Guidelines

  • Priority 9-10: Premium models (GPT-4, Claude Opus) - use for critical tasks
  • Priority 7-8: High-quality models (GPT-4o-mini, Claude Haiku) - balanced performance
  • Priority 5-6: Fast models (Gemini Flash, Cohere) - speed over quality
  • Priority 1-4: Specialized/fallback models - specific use cases

Working with Results

The RoutingResult object provides comprehensive information about the routing decision and execution: 

result = router.route("Write a Python function to reverse a string")

# Core Results
print(result.answer)              # The LLM response
print(result.provider_used)       # Provider that was used (e.g., "OPENAI")
print(result.model_used)          # Model that was used (e.g., "gpt-4o-mini")

# Query Classification Details
classification = result.query_classification
print(f"Query Type: {classification.query_type}")  # e.g., "coding"
print(f"Confidence: {classification.confidence:.2%}")  # e.g., 95%
print(f"Reasoning: {classification.reasoning}")
print(f"Matched By: {classification.matched_by}")  # "pattern", "llm", or "cache"

# Alternative query types (if any)
if classification.alternative_types:
    print(f"Could also be: {classification.alternative_types}")

# Routing Information
print(f"Routing Confidence: {result.routing_confidence:.2%}")
print(f"Routing Reasoning: {result.routing_reasoning}")

# Fallback Information
print(f"Used Fallback: {result.used_fallback}")
if result.used_fallback:
    print(f"Fallback Reason: {result.fallback_reason}")

# Performance Metrics
print(f"Execution Time: {result.execution_time:.2f}s")
print(f"Timestamp: {result.timestamp}")

šŸ“Š Example Output

Answer: Here's a Python function to reverse a string: [code]...
Provider Used: OPENAI
Model Used: gpt-4o-mini

Query Classification:
  Query Type: coding
  Confidence: 95.0%
  Reasoning: Query contains 'write' and 'function' keywords indicating code request
  Matched By: pattern

Routing:
  Confidence: 100.0%
  Reasoning: OpenAI specializes in coding tasks (priority 8)
  Used Fallback: False

Performance:
  Execution Time: 1.24s
  Timestamp: 2024-01-15 10:30:45

Advanced Features

Custom Query Patterns

Add custom patterns for domain-specific query types: 

# Define custom patterns for new query types
custom_patterns = {
    "blockchain": [
        r"\bblockchain\b",
        r"\bcryptocurrency\b",
        r"\bweb3\b",
        r"\bsmart contract",
        r"\bethereum\b"
    ],
    "medical": [
        r"\bdiagnosis\b",
        r"\bsymptoms?\b",
        r"\btreatment\b",
        r"\bmedical\b",
        r"\bhealth\b"
    ],
}

# Create provider specializing in blockchain
blockchain_config = ProviderConfig(
    llm_provider="ANTHROPIC",
    llm_model="claude-3-5-haiku-20241022",
    specialties=["blockchain", "technical_explanation"],
    description="Expert in blockchain and cryptocurrency topics",
    priority=9
)

# Initialize router with custom patterns
router = LLMProviderRouter(
    provider_configs=[blockchain_config, ...],
    llm_instances=[...],
    custom_patterns=custom_patterns,
    classification_method="hybrid"
)

# Now blockchain queries are correctly classified
result = router.route("Explain how Ethereum smart contracts work")
# Classification: blockchain (matched by custom pattern)

Force Provider (Bypass Routing)

Override automatic routing to use a specific provider: 

# Force use of specific provider (bypasses classification and routing)
result = router.route(
    query="Explain quantum computing",
    force_provider="ANTHROPIC"  # Always use Anthropic regardless of query type
)

print(f"Provider: {result.provider_used}")  # Always "ANTHROPIC"

# Useful for:
# - Testing specific providers
# - User preference overrides
# - Known provider strengths for specific queries

Top-K Fallback Strategy

Try multiple providers in order until one succeeds: 

# Try top 3 matching providers if first one fails
result = router.route(
    query="Generate a creative story about time travel",
    top_k=3  # Try top 3 providers until success
)

# Routing process:
# 1. Try best match (e.g., Anthropic - creative writing specialist)
# 2. If fails: Try 2nd best (e.g., OpenAI - also handles creative)
# 3. If fails: Try 3rd best (e.g., Gemini - general purpose)

print(f"Provider Used: {result.provider_used}")
print(f"Used Fallback: {result.used_fallback}")
if result.used_fallback:
    print(f"Reason: {result.fallback_reason}")

# Use cases:
# - High reliability requirements
# - Provider rate limiting
# - API outages

Export/Import Configuration

Save and reuse routing configurations across projects: 

# Export configuration to JSON
router.export_config("router_config.json")

# Configuration file includes:
# - Provider configs (without API keys)
# - Custom patterns
# - Classification settings
# - Default options

# Load configuration in another project
from SimplerLLM.language import LLMProviderRouter, LLM, LLMProvider

# Create LLM instances (API keys from environment)
llm_instances = [
    LLM.create(LLMProvider.OPENAI, model_name="gpt-4o-mini"),
    LLM.create(LLMProvider.ANTHROPIC, model_name="claude-3-5-haiku-20241022"),
]

# Load router from config
router = LLMProviderRouter.from_config(
    config_path="router_config.json",
    llm_instances=llm_instances
)

# Router ready to use with saved configuration
result = router.route("Your query here")

Note: API keys are never exported. You must provide LLM instances when importing.

Classification Caching

Cache classification results for repeated queries: 

# Enable caching with TTL
router = LLMProviderRouter(
    provider_configs=configs,
    llm_instances=llms,
    enable_cache=True,
    cache_ttl=3600  # Cache for 1 hour (in seconds)
)

# First call: Classifies query
result1 = router.route("Explain machine learning")
print(result1.query_classification.matched_by)  # "pattern" or "llm"

# Second call with same/similar query: Uses cache
result2 = router.route("Explain machine learning")
print(result2.query_classification.matched_by)  # "cache" (instant)

# Check cache statistics
stats = router.classifier.get_cache_stats()
print(f"Cache Hits: {stats['hits']}")
print(f"Cache Misses: {stats['misses']}")
print(f"Hit Rate: {stats['hit_rate']:.2%}")

# Clear cache if needed
router.classifier.clear_cache()

# Benefits:
# - Faster repeated queries (instant vs ~500ms LLM call)
# - Reduced API costs
# - Better performance for high-volume applications

Dynamic Provider Management

Add or remove providers at runtime: 

# Add new provider dynamically
new_llm = LLM.create(LLMProvider.COHERE, model_name="command-r-plus")
router.add_provider(
    llm=new_llm,
    specialties=["summarization", "analysis"],
    description="Excellent at document summarization",
    priority=7
)

# Remove provider by index
router.remove_provider(index=2)  # Remove third provider

# Add custom pattern rule
router.add_pattern_rule(
    query_type="legal",
    pattern=r"\blegal\b|\bcontract\b|\blawsuit\b"
)

# Useful for:
# - A/B testing providers
# - Gradual rollout of new models
# - Disabling underperforming providers
# - Cost optimization

Fallback Strategies

LLM Provider Router implements a three-tier fallback system for maximum reliability:

1. Provider-Level Fallback

Each provider can have its own fallback configured in ProviderConfig. 

ProviderConfig(
    llm_provider="OPENAI",
    has_fallback=True,
    fallback_provider="ANTHROPIC"
)

2. Router-Level Fallback

Router can have a default fallback provider for all queries. 

router = LLMProviderRouter(
    ...,
    default_provider="GEMINI"
)

3. Top-K Fallback

Try multiple matching providers in priority order. 

result = router.route(
    query,
    top_k=3
)

šŸ”„ Fallback Execution Flow

Query: "Write a function to sort an array"

Step 1: Primary Provider (OpenAI - coding specialist)
  ā†’ Try: OpenAI gpt-4o-mini
  ā†’ Result: API rate limit error

Step 2: Provider-Level Fallback
  ā†’ Try: Anthropic claude-3-5-haiku (configured fallback)
  ā†’ Result: Success! āœ“
  ā†’ Return: RoutingResult with used_fallback=True

If Step 2 fails:
Step 3: Router-Level Fallback
  ā†’ Try: Default provider (if configured)

If Step 3 fails:
Step 4: Top-K Fallback (if enabled)
  ā†’ Try: Next provider in priority list

LLM Provider Router vs LLM Router

SimplerLLM has two router systems. Choose the right one for your use case:

Aspect LLM Provider Router LLM Router (Basic)
Purpose Route queries to best LLM provider Route input to best choice from list
Classification Query type classification (coding, creative, etc.) Direct semantic matching
Execution Executes LLM call and returns answer Returns selected choice (no execution)
Configuration Provider specialties, priorities, fallbacks List of choices with optional metadata
Output LLM answer + routing metadata Selected choice index + confidence
Use Case Smart provider selection for LLM queries FAQ matching, content recommendation
Fallback Multi-tier fallback system No fallback (returns None if no match)
Caching Classification caching with TTL No caching

Use LLM Provider Router when:

  • āœ“ You need automatic provider selection for LLM queries
  • āœ“ You want to optimize costs by routing to appropriate models
  • āœ“ You need classification of query intent
  • āœ“ You want fallback mechanisms for reliability
  • āœ“ You're building production applications with multiple providers

Use LLM Router (Basic) when:

  • āœ“ You need to match user input to content choices
  • āœ“ You're building FAQ systems or chatbots
  • āœ“ You need content recommendation
  • āœ“ You want semantic matching without execution
  • āœ“ Selection logic is simpler (no classification needed)

šŸ’” They Work Together!

LLM Provider Router actually uses LLM Router internally when multiple providers match a query type. It creates an internal LLM Router instance to select the best provider based on their descriptions.

Best Practices

Classification Method Selection

Use Pattern-Based when:

  • ā€¢ Query patterns are well-defined and predictable
  • ā€¢ Volume is high and cost optimization is critical
  • ā€¢ Latency must be minimal (real-time applications)
  • ā€¢ Query types have clear keyword indicators

Use Hybrid when:

  • ā€¢ You want balance between cost and accuracy (recommended default)
  • ā€¢ Query complexity varies (some clear, some ambiguous)
  • ā€¢ You have budget for occasional LLM classification
  • ā€¢ Best for most production use cases

Use LLM-Based when:

  • ā€¢ Accuracy is paramount (classification must be perfect)
  • ā€¢ Queries are complex or ambiguous
  • ā€¢ Domain-specific classification beyond standard patterns
  • ā€¢ Volume is low enough that LLM costs are acceptable

Provider Configuration Tips

  • Overlap specialties: Multiple providers can share specialties for redundancy
  • Use descriptive descriptions: Clear descriptions help LLM Router select best provider when multiple match
  • Set realistic priorities: Higher cost = higher priority, but consider performance/cost trade-offs
  • Configure fallbacks: At least 1-2 providers should have fallback configured
  • Test with force_provider: Verify each provider works correctly before relying on automatic routing

Performance Optimization

# Optimal configuration for production
router = LLMProviderRouter(
    provider_configs=configs,
    llm_instances=llms,

    # Use hybrid for balance
    classification_method="hybrid",
    confidence_threshold=0.8,  # Higher = more pattern usage

    # Enable caching
    enable_cache=True,
    cache_ttl=3600,  # 1 hour

    # Use cheaper model for classification
    classifier_llm=LLM.create(LLMProvider.OPENAI, model_name="gpt-4o-mini"),

    # Verbose only in development
    verbose=False  # Disable in production
)

# Performance tips:
# 1. Higher confidence_threshold = more pattern matching (faster, cheaper)
# 2. Cache TTL based on query stability (1 hour default is good)
# 3. Use mini models for classification (gpt-4o-mini, claude-haiku)
# 4. Monitor cache hit rates and adjust TTL accordingly

Real-World Example: Multi-Provider Application

Complete setup for a production application with custom patterns and comprehensive fallback: 

from SimplerLLM.language import (
    LLM, LLMProvider, LLMProviderRouter, ProviderConfig
)

# Step 1: Define custom patterns for your domain
custom_patterns = {
    "product_description": [
        r"write.*product description",
        r"describe.*product",
        r"marketing.*copy"
    ],
    "code_review": [
        r"review.*code",
        r"code.*review",
        r"refactor",
        r"improve.*code"
    ],
    "legal": [
        r"\blegal\b",
        r"\bcontract\b",
        r"\bterms.*service",
        r"\bprivacy.*policy"
    ],
}

# Step 2: Configure providers with specializations
provider_configs = [
    # Premium coding provider
    ProviderConfig(
        llm_provider="OPENAI",
        llm_model="gpt-4o",
        specialties=["coding", "code_review", "technical_writing"],
        description="Top-tier coding expertise, best for complex programming tasks",
        priority=9,
        has_fallback=True,
        fallback_provider="ANTHROPIC",
        fallback_model="claude-3-5-haiku-20241022"
    ),

    # Creative and analytical provider
    ProviderConfig(
        llm_provider="ANTHROPIC",
        llm_model="claude-3-5-haiku-20241022",
        specialties=["creative_writing", "product_description", "analysis", "legal"],
        description="Excellent at creative content, product descriptions, and careful analysis",
        priority=8
    ),

    # Fast general-purpose provider
    ProviderConfig(
        llm_provider="GEMINI",
        llm_model="gemini-1.5-flash",
        specialties=["general", "technical_explanation", "summarization"],
        description="Fast and efficient for explanations and general queries",
        priority=7
    ),

    # Cost-effective fallback
    ProviderConfig(
        llm_provider="OPENAI",
        llm_model="gpt-4o-mini",
        specialties=["general", "coding", "analysis"],
        description="Cost-effective option for most tasks",
        priority=6
    ),
]

# Step 3: Create LLM instances
llm_instances = [
    LLM.create(LLMProvider.OPENAI, model_name="gpt-4o"),
    LLM.create(LLMProvider.ANTHROPIC, model_name="claude-3-5-haiku-20241022"),
    LLM.create(LLMProvider.GEMINI, model_name="gemini-1.5-flash"),
    LLM.create(LLMProvider.OPENAI, model_name="gpt-4o-mini"),
]

# Step 4: Initialize router with full configuration
classifier_llm = LLM.create(LLMProvider.OPENAI, model_name="gpt-4o-mini")

router = LLMProviderRouter(
    provider_configs=provider_configs,
    llm_instances=llm_instances,
    classification_method="hybrid",
    classifier_llm=classifier_llm,
    confidence_threshold=0.8,
    enable_cache=True,
    cache_ttl=3600,
    custom_patterns=custom_patterns,
    default_provider="OPENAI",  # Fallback to OpenAI gpt-4o-mini
    verbose=True
)

# Step 5: Export configuration for reuse
router.export_config("production_router_config.json")

# Step 6: Use router in your application
def handle_user_query(query: str, top_k: int = 1):
    """Handle user query with smart routing and error handling."""
    try:
        result = router.route(query, top_k=top_k)

        # Log routing decision
        print(f"Query Type: {result.query_classification.query_type}")
        print(f"Provider: {result.provider_used}")
        print(f"Confidence: {result.routing_confidence:.2%}")

        if result.used_fallback:
            print(f"āš ļø Fallback used: {result.fallback_reason}")

        return result.answer

    except Exception as e:
        print(f"āŒ Error: {e}")
        return None

# Example usage
queries = [
    "Write a Python function to parse JSON files",
    "Create a compelling product description for wireless headphones",
    "Review this code for best practices: [code snippet]",
    "Explain how neural networks work in simple terms",
    "Draft terms of service for a SaaS product",
]

for query in queries:
    print(f"\n{'='*80}")
    print(f"Query: {query[:60]}...")
    print('='*80)

    answer = handle_user_query(query, top_k=2)

    if answer:
        print(f"\nAnswer: {answer[:200]}...")

# Check cache performance
stats = router.classifier.get_cache_stats()
print(f"\nšŸ“Š Cache Performance:")
print(f"   Hit Rate: {stats['hit_rate']:.1%}")
print(f"   Total Queries: {stats['hits'] + stats['misses']}")

API Reference

LLMProviderRouter Class

class LLMProviderRouter:
    """Intelligent query classification and provider routing system."""

    def __init__(
        self,
        provider_configs: List[ProviderConfig],
        llm_instances: List[LLM],
        classification_method: str = "hybrid",  # "pattern", "llm", or "hybrid"
        classifier_llm: Optional[LLM] = None,
        confidence_threshold: float = 0.8,
        enable_cache: bool = False,
        cache_ttl: int = 3600,
        custom_patterns: Optional[Dict[str, List[str]]] = None,
        default_provider: Optional[str] = None,
        verbose: bool = False
    )

    def route(
        self,
        query: str,
        force_provider: Optional[str] = None,
        top_k: int = 1,
        **kwargs
    ) -> RoutingResult

    def get_provider_for_query(
        self,
        query: str
    ) -> Tuple[Optional[LLM], Optional[ProviderConfig]]

    def add_provider(
        self,
        llm: LLM,
        specialties: List[str],
        description: str,
        priority: int = 5
    ) -> None

    def remove_provider(self, index: int) -> None

    def add_pattern_rule(
        self,
        query_type: str,
        pattern: str
    ) -> None

    def export_config(self, filepath: str) -> None

    @classmethod
    def from_config(
        cls,
        config_path: str,
        llm_instances: List[LLM],
        **kwargs
    ) -> "LLMProviderRouter"

ProviderConfig

class ProviderConfig:
    llm_provider: str                     # Provider name (e.g., "OPENAI")
    llm_model: str                        # Model name (e.g., "gpt-4o-mini")
    specialties: List[str]                # Query types this provider handles
    description: str                      # Provider strengths description
    priority: int = 5                     # Priority score (1-10)
    enabled: bool = True                  # Enable/disable provider
    has_fallback: bool = False            # Has fallback configured
    fallback_provider: Optional[str] = None
    fallback_model: Optional[str] = None

RoutingResult

class RoutingResult:
    answer: str                            # LLM response
    provider_used: str                     # Provider name used
    model_used: str                        # Model name used
    query_classification: QueryClassification  # Classification details
    routing_confidence: float              # Routing confidence (0-1)
    routing_reasoning: str                 # Why this provider was selected
    used_fallback: bool                    # Whether fallback was used
    fallback_reason: Optional[str]         # Reason for fallback
    execution_time: float                  # Total execution time (seconds)
    timestamp: datetime                    # Execution timestamp

QueryClassification

class QueryClassification:
    query_type: str                        # Classified query type
    confidence: float                      # Classification confidence (0-1)
    reasoning: str                         # Classification reasoning
    matched_by: str                        # "pattern", "llm", or "cache"
    alternative_types: Optional[List[str]] # Alternative classifications

Next Steps

LLM Router (Basic) ā†’

Learn about content routing and choice selection (used internally by Provider Router)

LLM Judge ā†’

Use Judge to benchmark providers and generate routing recommendations
Edit this page on GitHub ā†’