mcp_llm library
Main library for integrating Large Language Models with MCP
Classes
-
AdvancedLlmTask<
T> - Task definition
- AdvancedTaskScheduler
- Advanced task scheduler
- AsrPortAdapter
- Adapter that implements mcp_bundle's AsrPort using mcp_llm's AsrProvider.
- AsrPortAdapterFactory
- Factory for creating AsrPortAdapter instances.
- AsrProvider
- Abstract interface for ASR (Automatic Speech Recognition) providers.
- AsrProviderConfig
- Configuration for ASR providers.
- AwsTextractProvider
- AWS Textract API provider.
- BaseAsrProvider
- Base class for ASR providers with common functionality.
- BaseBinaryStorageProvider
- Base class for storage providers with common functionality.
- BaseOcrProvider
- Base class for OCR providers with common functionality.
- BasePromptPlugin
- Base implementation of a prompt plugin
- BaseToolPlugin
- Base implementation of a tool plugin
- BaseVisionProvider
- Base class for vision providers with common functionality.
- BedrockProvider
- Implementation of LLM interface for AWS Bedrock. Provides access to multiple model families (Claude, Llama, Mistral, Titan). Note: Requires AWS credentials and proper IAM permissions.
- BedrockProviderFactory
- Factory for creating Bedrock providers.
- BinaryStoragePortAdapter
- Adapter that implements mcp_bundle's BinaryStoragePort using mcp_llm's BinaryStorageProvider.
- BinaryStoragePortAdapterFactory
- Factory for creating BinaryStoragePortAdapter instances.
- BinaryStorageProvider
- Abstract interface for binary storage providers.
- CacheHints
- Provider-agnostic prompt caching hints attached to an LlmRequest.
- CapabilityEvent
- Capability event
- CapabilityUpdateRequest
- Capability update request
- CapabilityUpdateResponse
- Capability update response
- ChatHistory
- Manages the history of messages for a chat session
- ChatSession
- Manages a chat session with an LLM, including history and context
- ClaudeProvider
- Implementation of LLM interface for Claude API
- ClaudeProviderFactory
- Factory for creating Claude providers
- CloudProviderRegistry
- Central registry for all cloud AI providers.
- CohereProvider
- Implementation of LLM interface for Cohere API. Supports completion, streaming, embeddings (strong), and tool calling.
- CohereProviderFactory
- Factory for creating Cohere providers.
- ConfidenceResultAggregator
- Aggregator that selects the response with highest confidence
- Conversation
- Manages a conversation with optional topics and multi-session support
- CoreLlmPluginFactory
- Factory to create all core LLM feature plugins
- CustomLlmProvider
- Base class for implementing custom LLM providers
- CustomLlmProviderFactory
- Base factory for custom LLM providers
- DefaultServiceBalancer
- Default implementation of ServiceBalancer
- DefaultServiceRouter
- Default implementation of ServiceRouter
- DefaultTokenizer
- DeferredToolManager
- Orchestration layer for deferred tool loading Uses internal registry for data management Automatically created when useDeferredLoading=true in LlmClient
- Document
- Represents a document with content and optional metadata
- DocumentCollection
- Represents a collection of documents
- DocumentStore
- Store for documents with vector search capabilities
- DynamicFileResourcePlugin
- Dynamic file resource plugin that can generate file content on demand
- DynamicPromptPlugin
- Dynamic prompt plugin generated by LLM
- DynamicResourcePlugin
- Dynamic resource plugin generated by LLM
- DynamicToolPlugin
- Dynamic tool plugin generated by LLM
- Embedding
- Represents a document's embedding vector
- EmbeddingPlugin
- Interface for embedding plugins that provide embedding functionality
- EmbeddingUtils
- Utilities for working with embeddings
- EnhancedErrorHandler
- Enhanced Error Handler for MCP specification
- ErrorCircuitBreaker
- Circuit breaker for error handling
- ErrorHandler
- Error handler for MCPLlm
- ErrorHandlingConfig
- Error handling configuration
- GcsStorageProvider
- Google Cloud Storage provider.
- GeminiProvider
- Implementation of LLM interface for Google Gemini API. Supports completion, streaming, embeddings, tool calling, and vision.
- GeminiProviderFactory
- Factory for creating Gemini providers.
-
GenericServicePool<
T> - Implementation of service pool for resource pooling of clients or servers
- GoogleSpeechProvider
- Google Cloud Speech-to-Text API provider.
- GoogleVisionOcrProvider
- Google Cloud Vision OCR API provider.
- GoogleVisionProvider
- Google Cloud Vision API provider.
- GroqProvider
- Implementation of LLM interface for Groq API (OpenAI-compatible). Groq provides fast inference for open-source models like Llama and Mixtral.
- GroqProviderFactory
- Factory for creating Groq providers.
- HealthCheckConfig
- Health check configuration (using models from core/models.dart)
- HealthCheckResult
- Health check result
- HealthReport
- Health report
- HttpDate
- HTTP date parsing helper.
- IPluginManager
- Manager interface for plugins
- Level
- Levels to control logging output. Logging can be enabled to include all levels above certain Level. Levels are ordered using an integer value Level.value. The predefined Level constants below are sorted as follows (in descending order): Level.SHOUT, Level.SEVERE, Level.WARNING, Level.INFO, Level.CONFIG, Level.FINE, Level.FINER, Level.FINEST, and Level.ALL.
- LifecycleEvent
- Lifecycle event
- LifecycleRequest
- Lifecycle operation request
- LifecycleResponse
- Lifecycle response
- LlmCacheMetadataKeys
- Standard keys that providers use to surface prompt-caching usage on LlmResponse.metadata. Keep these stable across providers so callers can compute savings without provider-specific branches.
- LlmCallToolResult
- Tool call result
- LlmClient
- Client for interacting with LLM providers
- LlmClientAdapter
- An MCP client as mcp_llm uses it. Listing and calls go through McpClientBridge (typed, every page, failures as McpLayerException); the map methods return the protocol JSON of those results.
- LlmClientFactory
- Implementation of service factory for LLM clients
- LlmClientServiceAdapter
- Adapter that makes LlmClient conform to ManagedService interface
- LlmCompletionPlugin
- Plugin for basic LLM text completion
- LlmConfiguration
- LLM configuration with enhanced retry capabilities
- LLmContent
- Base class for all MCP content types
- LlmContext
- Manages the execution context for LLM requests
- LlmEmbeddingPlugin
- Plugin for generating embeddings
- LlmGetPromptResult
- Get prompt result
- LlmImageContent
- Image content representation
- LlmMessage
- Represents a chat message in a conversation
- LlmPlugin
- Base interface for all MCPLlm plugins
- LlmPortAdapter
- Adapter that implements mcp_bundle's LlmPort using mcp_llm's LlmProvider.
- LlmPortAdapterFactory
- Factory for creating LlmPortAdapter instances.
- LlmPrompt
- Prompt definition
- LlmPromptArgument
- Prompt argument definition
- LlmProvider
- LLM provider interface for mcp_llm internal use.
- LlmProviderFactory
- Factory interface for creating LLM providers
- LlmRagPlugin
- Plugin for Retrieval Augmented Generation (RAG)
- LlmReadResourceResult
- Resource read result
- LlmRegistry
- Registry that dynamically registers and manages LLM providers
- LlmRequest
- Represents a request to an LLM
- LlmResource
- Resource definition
- LlmResourceContent
- Resource content representation
- LlmResponse
- LLM response for completions
- LlmResponseChunk
- LLM response chunk for streaming
- LlmRetrievalPlugin
- Plugin for document retrieval
- LlmServer
- Server for providing LLM capabilities Enhanced with plugin-based tool management
- LlmServerAdapter
- Adapter for interfacing with MCP server instances This adapter handles the conversion between the MCP Server and LLM-related functionality
- LlmServerFactory
- Implementation of service factory for LLM servers
- LlmServerServiceAdapter
- Adapter that makes LlmServer conform to ManagedService interface
- LlmStreamingPlugin
- Plugin for streaming LLM text completion
-
LlmTask<
T> - Represents a scheduled task with priority
- LlmTextContent
- Text content representation
- LlmTool
- Tool definition
- LlmToolCall
- Tool call request
- LlmToolCallContent
- Tool call result content
- LlmToolMetadata
- Lightweight tool metadata (name + description only) Used for token-efficient LLM context in deferred loading mode
- LlmToolResultContent
- Tool result content
- LlmUsageMetadataKeys
- Standard keys that providers use to report the tokens of one call on LlmResponse.metadata (streaming: on the final chunk).
- Logger
- Use a Logger to log debug messages.
- LogRecord
- A log entry representation used to propagate information from Logger to individual handlers.
- ManagedService
- Base interface for any service that can be managed (clients, servers, etc.)
- McpCapability
-
One capability a peer declared, as mcp_llm tracks it. configuration is
the declared capability object (e.g.
{listChanged: true}); version is the protocol revision it was read under; enabled is mcp_llm's own flag. - McpCapabilityManager
-
The capabilities of the MCP peers mcp_llm holds, as they declared them —
a client reports the server's
ServerCapabilities, a server its own. MCP has no method to change a peer's capabilities after initialization; updateCapabilities, enableCapability and disableCapability change only mcp_llm's own record. - McpClientBridge
- Typed operations on one MCP client.
- McpClientManager
-
Several MCP clients behind one LLM client. Each is reached through an
LlmClientAdapter (typed through its McpClientBridge); a client that
needs authorization carries it itself (mcp_client's
McpOAuth). - McpHealthMonitor
-
Health of the MCP peers mcp_llm holds. MCP defines no health method; a
client is checked with
pingand a server by its own health record (see_checkPeer). - McpLlm
- Main class for MCPLlm functionality
- McpSamplingHandler
- Answers sampling requests with provider.
- McpServerManager
- Manages multiple MCP servers
- McpToolResultRenderer
- Renders tool results and resource reads for the model.
- MemoryStorage
- In-memory implementation of StorageManager for testing and simple use cases
- MergeResultAggregator
- Aggregator that merges results
- MistralProvider
- Implementation of LLM interface for Mistral AI API. Supports completion, streaming, embeddings, and tool calling.
- MistralProviderFactory
- Factory for creating Mistral providers.
- MultiLlmClientManager
- Manager for multiple LLM clients that implements the ServiceManager interface
- MultiLlmServerManager
- Manager for multiple LLM servers that implements the ServiceManager interface
- OcrPortAdapter
- Adapter that implements mcp_bundle's OcrPort using mcp_llm's OcrProvider.
- OcrPortAdapterFactory
- Factory for creating OcrPortAdapter instances.
- OcrProvider
- Abstract interface for OCR providers.
- OcrProviderConfig
- Configuration for OCR providers.
- OpenAiProvider
- Implementation of LLM interface for OpenAI API
- OpenAiProviderFactory
- Factory for creating OpenAI providers
- OpenAIVisionProvider
- OpenAI GPT-4 Vision API provider.
- OpenAIWhisperProvider
- OpenAI Whisper API provider.
- OptimizedMemoryVectorStore
- High-performance in-memory vector store with persistence
- ParallelExecutor
- Class that manages parallel operations for multiple LLMs
- PerformanceMonitor
- Performance monitoring system for LLM and MCP usage
- PersistentStorage
- File-based persistent storage implementation
- PineconeVectorStore
- Pinecone vector store implementation
- PineconeVectorStoreFactory
- Factory for creating Pinecone vector stores
- PluginManager
- Manages plugins for the MCPLlm system
- PostprocessorPlugin
- Interface for postprocessor plugins that transform LLM outputs
- PreprocessorPlugin
- Interface for preprocessor plugins that transform inputs
- PromptPlugin
- Interface for prompt plugins that provide prompt templates
- ProviderOptions
- Configuration options for LLM providers
- ProviderPlugin
- Interface for provider plugins that implement new LLM providers
- RealPineconeVectorStore
- Pinecone vector store implementation
- RealQdrantVectorStore
- Qdrant vector store implementation
- RealWeaviateVectorStore
- Weaviate vector store implementation
- RenderedToolResult
- A tool result ready for the model.
- ResourcePlugin
- Interface for resource plugins that provide resources
- ResultAggregator
- Interface for aggregating multiple LLM responses
- RetrievalCache
- Cache to store retrieval results to improve performance
- RetrievalManager
- Enhanced Retrieval manager with improved RAG capabilities
- RetryableLlmProvider
- Interface for LLM providers with retry capabilities
- RetryStrategy
- Retry strategy
- S3StorageProvider
- AWS S3 storage provider.
- SampleEchoToolPlugin
- A simple example tool plugin implementation
- ScoredDoc
- Helper class for document scoring
- ScoredDocument
- Scored document result
- ScoredEmbedding
- Scored embedding result
- ServerInfo
- Server info
- ServerLifecycleManager
- The lifecycle state mcp_llm records for each MCP server it holds (starting, running, paused, stopped). MCP defines no server lifecycle methods, and the host owns the server and its transport: start, stop, pause and resume change this record (and run the health check on start when a monitor is given) — they do not open or close the server.
- ServiceBalancer
- Base interface for service balancers
-
ServiceFactory<
T> - Factory interface for creating services
-
ServiceManager<
T extends ManagedService> - Base interface for service managers that handle multiple services
-
ServicePool<
T> - Base interface for service pools
- ServiceRouter
- Base interface for service routers
- SimpleResultAggregator
- Simple result aggregator
- StorageManager
- Abstract interface for storage management
- StorageProviderConfig
- Configuration for storage providers.
- StoryStarterPromptPlugin
- Example prompt plugin for generating creative story starters
- TaskScheduler
- Task scheduler with priority queue and concurrency control
- TogetherProvider
- Implementation of LLM interface for Together AI API
- TogetherProviderFactory
- Factory for creating Together AI providers
- TokenCounter
- Tokenizer
- ToolPlugin
- Interface for tool plugins that provide tool functionality
- ValidationResult
- Simple validation result for tool call validation
- VectorDocument
- Vector document for internal use
- VectorSearchResult
- Vector search result for internal use
- VectorStore
- Base interface for vector stores
- VectorStoreConfig
- Configuration options for vector stores
- VectorStoreFactory
- Factory for creating vector stores
- VectorStoreRegistry
- Registry for vector store implementations
- VectorStoreStats
- Vector store statistics
- VertexAiProvider
- Implementation of LLM interface for Google Vertex AI. Enterprise-grade access to Gemini and other models with OAuth/Service Account auth.
- VertexAiProviderFactory
- Factory for creating Vertex AI providers.
- VisionPortAdapter
- Adapter that implements mcp_bundle's VisionPort using mcp_llm's VisionProvider.
- VisionPortAdapterFactory
- Factory for creating VisionPortAdapter instances.
- VisionProvider
- Abstract interface for vision providers.
- VisionProviderConfig
- Configuration for vision providers.
- WeaviateVectorStore
- Weaviate vector store implementation
Enums
- CapabilityEventType
- Capability event type
- CircuitState
- Circuit breaker state
- ErrorHandlingStrategy
- Error handling strategy
- ErrorSeverity
- Error severity levels
- ErrorType
- Error types for MCPLlm
- HealthStatus
- Health status enum
- LifecycleTransitionReason
- Lifecycle transition reason
- LlmCapability
- LLM capabilities enum
- LlmContentType
- Content type enum
- McpCapabilityType
-
The kind of a capability a peer declared — the keys of the protocol's
ServerCapabilities/ClientCapabilities. - McpErrorCategory
- MCP error category
- PriorityStrategy
- Task priority calculation strategy
- RoutingStrategy
- Routing strategy options
- SelectionStrategy
- ServerLifecycleState
- Server lifecycle state
- TaskStatus
- Task status
Extensions
- LlmServerAdvancedExtensions on LlmServer
- Extensions for advanced server operations
- LlmServerHelperExtensions on LlmServer
- Additional helper methods General extensions for LlmServer
- LlmServerPluginExtensions on LlmServer
- Extension methods for LlmServer Provides convenience methods for registering core LLM functionality as plugins
- LoggerExtensions on Logger
- RetryCapabilities on LlmProvider
- Extension to add retry capabilities to LLM providers.
Constants
- defaultLevel → const Level
- The default Level.
Properties
- hierarchicalLoggingEnabled ↔ bool
-
Whether to allow fine-grain logging and configuration of loggers in a
hierarchy.
getter/setter pair
- recordStackTraceAtLevel ↔ Level
-
Automatically record stack traces for any message of this level or above.
getter/setter pair
Functions
-
callToolResultOf(
Object? value) → CallToolResult -
A tool handler's return value as a
tools/callresult. -
contentBlockOf(
LLmContent content) → ContentBlock - One mcp_llm content value as an MCP content block.
-
getPromptResultOf(
Object? value) → GetPromptResult -
A prompt handler's return value as a
prompts/getresult. Prompt messages have onlyuserandassistantroles; a system message is given to the user role, its text unchanged. -
readResourceResultOf(
String uri, Object? value, {String? mimeType}) → ReadResourceResult -
A resource handler's return value as a
resources/readresult foruri. A plain string carriesmimeType— the type the resource was registered with.
Typedefs
- LlmInterface = LlmProvider
- Backward compatibility: LlmInterface is now an alias for LlmProvider.
- MCPLlm = McpLlm
-
PromptHandler
= Future Function(Map<
String, dynamic> arguments) - Type definition for prompt handler functions
-
ResourceHandler
= Future Function(String uri, Map<
String, dynamic> params) - Type definition for resource handler functions
-
ToolHandler
= Future Function(Map<
String, dynamic> arguments) -
A tool handler: returns a generated
CallToolResult, an LlmCallToolResult, a string, or a protocol-shaped map — converted to the protocol'stools/callresult before it is sent.
Exceptions / Errors
- AsrProviderException
- Exception thrown by ASR providers.
- AuthenticationError
- Authentication errors (API keys, tokens)
- McpEnhancedError
- MCP enhanced error
- McpEnhancedErrorExt
- Enhanced MCP error extension with additional fields
- McpLayerException
- A failed MCP operation: which method, and what the server or client said.
- McpLlmError
- Base class for MCPLlm errors
- NetworkError
- Network-related errors (API calls, connectivity)
- OcrProviderException
- Exception thrown by OCR providers.
- PermissionError
- Permission errors (access denied)
- ProviderError
- Provider-specific errors
- ResourceNotFoundError
- Resource not found errors
- StorageProviderException
- Exception thrown by storage providers.
- TaskCancelledException
- Exception thrown when a task is cancelled
- TimeoutError
- Timeout errors
- ValidationError
- Validation errors (invalid inputs)
- VisionProviderException
- Exception thrown by vision providers.