config#
Config helper functions to manage configuration and rebuilt your task pipeline.
Config format: json (1) include attribute and entity_name to reconstruct all attributes of a pipeline.
Example
- { # attribute and its config to recreate the component
- “document_splitter”: {
“component_name”: “DocumentSplitter”, “component_config”: {
“split_by”: “word”, “split_length”: 400, “split_overlap”: 200,
},
}, “to_embeddings”: {
“component_name”: “ToEmbeddings”, “component_config”: {
- “embedder”: {
“component_name”: “Embedder”, “component_config”: {
- “model_client”: {
“entity_name”: “OpenAIClient”, “entity_config”: {},
}, “model_kwargs”: {
“model”: “text-embedding-3-small”, “dimensions”: 256, “encoding_format”: “float”,
},
}, # the other config is to instantiate the entity (class and function) with the given config as arguments # “entity_state”: “storage/embedder.pkl”, # this will load back the state of the entity
}, “batch_size”: 100,
},
},
}
(2) only include the config as arguments and does not include any entity_name or attribute. You can use this manually to construct an entity yourself.
Example: {
- “Embedder”: # it can be any name
- {
“model_client”: “OpenAIClient”, “model_kwargs”: {
“api_key”: “your_api_key”, “model_name”: “text-embedder”
}
}, “FAISSRetriever”: {
“top_k”: 2, “dimensions”: 256, “vectorizer”: “embedder”
}
}
Functions
|
Create a single componenet from a configuration dictionary. |
|
Construct multiple components from a configuration dictionary. |