model_client#
ModelClient is the protocol and base class for all models(either via APIs or local models) to communicate with components.
Classes
|
The protocol and abstract class for all models(either via APIs or local models) to communicate with components. |
- class ModelClient(*args, **kwargs)[source]#
Bases:
Component
The protocol and abstract class for all models(either via APIs or local models) to communicate with components.
ModelClient is to separate the model API calls from the rest of the system, making it a plug-and-play component that can be used in functional components like Generator and Embedder.
For a particular API provider, such as OpenAI, we will have a class that inherits from ModelClient. It does four things:
Initialize the client, including both sync and async.
Convert the standard LightRAG components inputs to the API-specific format.
Call the API and parse the response.
Handle API specific exceptions and errors to retry the call.
Check the subclasses in components/model_client/ directory for the functional API clients we have.
This interface is designed to bridge the gap between LightRAG components inputs and model APIs.
You can see examples of the subclasses in components/model_client/ directory.
- call(api_kwargs: Dict = {}, model_type: ModelType = ModelType.UNDEFINED)[source]#
Subclass use this to call the API with the sync client. model_type: this decides which API, such as chat.completions or embeddings for OpenAI. api_kwargs: all the arguments that the API call needs, subclass should implement this method.
Additionally in subclass you can implement the error handling and retry logic here. See OpenAIClient for example.
- async acall(api_kwargs: Dict = {}, model_type: ModelType = ModelType.UNDEFINED)[source]#
Subclass use this to call the API with the async client.
- convert_inputs_to_api_kwargs(input: Any | None = None, model_kwargs: Dict = {}, model_type: ModelType = ModelType.UNDEFINED) Dict [source]#
Bridge the Component’s standard input and model_kwargs into API-specific format, the api_kwargs that will be used in _call and _acall methods.
All types of models supported by this particular provider should be handled here. :param input: input to the model. Defaults to None. :type input: Optional[Any], optional :param model_kwargs: model kwargs :type model_kwargs: Dict :param model_type: model type :type model_type: ModelType