model_client

ModelClient is the protocol and base class for all models(either via APIs or local models) to communicate with components.

Classes

ModelClient(*args, **kwargs)

The protocol and abstract class for all models(either via APIs or local models) to communicate with components.

class ModelClient(*args, **kwargs)

Bases: Component

The protocol and abstract class for all models(either via APIs or local models) to communicate with components.

ModelClient is to separate the model API calls from the rest of the system, making it a plug-and-play component that can be used in functional components like Generator and Embedder.

For a particular API provider, such as OpenAI, we will have a class that inherits from ModelClient. It does four things:

1. Initialize the client, including both sync and async.

2. Convert the standard LightRAG components inputs to the API-specific format.

3. Call the API and parse the response.

4. Handle API specific exceptions and errors to retry the call.

Check the subclasses in components/model_client/ directory for the functional API clients we have.

This interface is designed to bridge the gap between LightRAG components inputs and model APIs.

You can see examples of the subclasses in components/model_client/ directory.

init_sync_client()

init_async_client()

call(api_kwargs: Dict = {}, model_type: ModelType = ModelType.UNDEFINED)

Subclass use this to call the API with the sync client. model_type: this decides which API, such as chat.completions or embeddings for OpenAI. api_kwargs: all the arguments that the API call needs, subclass should implement this method.

Additionally in subclass you can implement the error handling and retry logic here. See OpenAIClient for example.

async acall(api_kwargs: Dict = {}, model_type: ModelType = ModelType.UNDEFINED)

Subclass use this to call the API with the async client.

convert_inputs_to_api_kwargs(input: Any | None = None, model_kwargs: Dict = {}, model_type: ModelType = ModelType.UNDEFINED) → Dict

Bridge the Component’s standard input and model_kwargs into API-specific format, the api_kwargs that will be used in _call and _acall methods.

All types of models supported by this particular provider should be handled here. :param input: input to the model. Defaults to None. :type input: Optional[Any], optional :param model_kwargs: model kwargs :type model_kwargs: Dict :param model_type: model type :type model_type: ModelType

parse_chat_completion(completion: Any) → GeneratorOutput

Parse the chat completion to str.

track_completion_usage(*args, **kwargs) → CompletionUsage

Track the chat completion usage. Use OpenAI standard API for tracking.

parse_embedding_response(response: Any) → EmbedderOutput

Parse the embedding response to a structure LightRAG components can understand.