groq_client

Groq ModelClient integration.

Classes

GroqAPIClient([api_key])

A component wrapper for the Groq API client.

class GroqAPIClient(api_key: str | None = None)

Bases: ModelClient

A component wrapper for the Groq API client.

Visit https://console.groq.com/docs/ for more api details. Check https://console.groq.com/docs/models for the available models.

Tested Groq models: 4/22/2024 - llama3-8b-8192 - llama3-70b-8192 - mixtral-8x7b-32768 - gemma-7b-it

init_sync_client()

init_async_client()

parse_chat_completion(completion: ChatCompletion) → GeneratorOutput

Parse the completion to a string output.

track_completion_usage(completion: Any) → CompletionUsage

Track the chat completion usage. Use OpenAI standard API for tracking.

convert_inputs_to_api_kwargs(input: Any | None = None, model_kwargs: Dict = {}, model_type: ModelType = ModelType.UNDEFINED) → Dict

Bridge the Component’s standard input and model_kwargs into API-specific format, the api_kwargs that will be used in _call and _acall methods.

All types of models supported by this particular provider should be handled here. :param input: input to the model. Defaults to None. :type input: Optional[Any], optional :param model_kwargs: model kwargs :type model_kwargs: Dict :param model_type: model type :type model_type: ModelType

call(api_kwargs: Dict = {}, model_type: ModelType = ModelType.UNDEFINED)

Subclass use this to call the API with the sync client. model_type: this decides which API, such as chat.completions or embeddings for OpenAI. api_kwargs: all the arguments that the API call needs, subclass should implement this method.

Additionally in subclass you can implement the error handling and retry logic here. See OpenAIClient for example.

async acall(api_kwargs: Dict = {}, model_type: ModelType = ModelType.UNDEFINED)

Subclass use this to call the API with the async client.

classmethod from_dict(data: Dict[str, Any]) → T

Create an instance from previously serialized data using to_dict() method.

to_dict() → Dict[str, Any]

Convert the component to a dictionary.