text_grad

Submodules

• backend_engine_prompt
• llm_text_loss
  • LLMAsTextLoss
    • LLMAsTextLoss.forward()
• ops
  • sum_ops()
  • Sum
    • Sum.name
    • Sum.forward()
    • Sum.backward()
    • Sum.backward_engine
    • Sum.training
    • Sum.teacher_mode
    • Sum.tracing
• text_loss_with_eval_fn
  • EvalFnToTextLoss
    • EvalFnToTextLoss.forward()
    • EvalFnToTextLoss.set_backward_engine()
    • EvalFnToTextLoss.backward()
• tgd_optimizer
  • HistoryPrompt
    • HistoryPrompt.id
    • HistoryPrompt.value
    • HistoryPrompt.eval_score
    • HistoryPrompt.method
    • HistoryPrompt.reasoning
  • Instruction
    • Instruction.text
    • Instruction.score
    • Instruction.responses
    • Instruction.gts
  • TGDData
    • TGDData.reasoning
    • TGDData.method
    • TGDData.proposed_variable
  • TGDOptimizerTrace
    • TGDOptimizerTrace.api_kwargs
    • TGDOptimizerTrace.output
  • extract_new_variable()
  • TGDOptimizer
    • TGDOptimizer.proposing
    • TGDOptimizer.params_history
    • TGDOptimizer.failed_proposals
    • TGDOptimizer.current_tgd_output
    • TGDOptimizer.params
    • TGDOptimizer.constraints
    • TGDOptimizer.one_parameter_at_a_time
    • TGDOptimizer.constraint_text
    • TGDOptimizer.increment_steps_from_last_improvement()
    • TGDOptimizer.reset_steps_from_last_improvement()
    • TGDOptimizer.add_score_to_params()
    • TGDOptimizer.add_score_to_current_param()
    • TGDOptimizer.add_history()
    • TGDOptimizer.render_history()
    • TGDOptimizer.add_failed_proposal()
    • TGDOptimizer.render_failed_proposals()
    • TGDOptimizer.update_gradient_memory()
    • TGDOptimizer.zero_grad()
    • TGDOptimizer.set_target_param()
    • TGDOptimizer.propose()
    • TGDOptimizer.revert()
    • TGDOptimizer.step()
    • TGDOptimizer.to_dict()

---

class LLMAsTextLoss(prompt_kwargs: Dict[str, str | Parameter], model_client: ModelClient, model_kwargs: Dict[str, object])

Bases: LossComponent

Evaluate the final RAG response using an LLM judge.

The LLM judge will have: - eval_system_prompt: The system prompt to evaluate the response. - y_hat: The response to evaluate. - Optional: y: The correct response to compare against.

The loss will be a Parameter with the evaluation result and can be used to compute gradients. This loss use LLM/Generator as the computation/transformation operator, so it’s gradient will be found from the Generator’s backward method.

forward(*args, **kwargs) → Parameter

Default just wraps the call method.

class EvalFnToTextLoss(eval_fn: Callable | BaseEvaluator, eval_fn_desc: str, backward_engine: BackwardEngine | None = None, model_client: ModelClient = None, model_kwargs: Dict[str, object] = None)

Bases: LossComponent

Convert an evaluation function to a text loss.

LossComponent will take an eval function and output a score (usually a float in range [0, 1], and the higher the better, unlike the loss function in model training).

In math:

score/loss = eval_fn(y_pred, y_gt)

The gradident/feedback = d(score)/d(y_pred) will be computed using a backward engine. Gradient_context = GradientContext(

context=conversation_str, response_desc=response.role_desc, variable_desc=role_desc,

)

Parameters:

• eval_fn – The evaluation function that takes a pair of y and y_gt and returns a score.

• eval_fn_desc – Description of the evaluation function.

• backward_engine – The backward engine to use for the text prompt optimization.

• model_client – The model client to use for the backward engine if backward_engine is not provided.

• model_kwargs – The model kwargs to use for the backward engine if backward_engine is not provided.

forward(kwargs: Dict[str, Parameter], response_desc: str = None, metadata: Dict[str, str] = None, id: str = None, gt: object = None, input: Dict[str, object] = None) → Parameter

Parameters:

• kwargs – The inputs to the eval_fn.

• response_desc – Description of the output.

• metadata – Additional notes on the input kwargs.

• id – The unique identifier for the data point.

• gt – The ground truth for the evaluation function.

set_backward_engine(backward_engine: BackwardEngine = None, model_client: ModelClient = None, model_kwargs: Dict[str, object] = None)

backward(response: Parameter, eval_fn_desc: str, kwargs: Dict[str, Parameter], ground_truth: object = None, backward_engine: BackwardEngine | None = None, metadata: Dict[str, str] = None, input: Dict[str, object] = None, disable_backward_engine: bool = False)

Ensure to set backward_engine for the text prompt optimization. It can be None if you are only doing demo optimization and it will not have gradients but simply backpropagate the score.

class TGDOptimizer(params: Iterable[Parameter] | Iterable[Dict[str, Any]], model_client: ModelClient, model_kwargs: Dict[str, object] = {}, constraints: List[str] = None, optimizer_system_prompt: str = 'You are an excellent prompt engineer tasked with instruction and demonstration tuning a compound LLM system.\nYour task is to refine a variable/prompt based on feedback from a batch of input data points.\n\nThe variable is either input or output of a functional component where the component schema will be provided.\nIf the same DataID has multiple gradients, it means this component/variable is called multiple times in the compound system(with a cycle) in the same order as it appears in the gradient list.\n\nYou Must edit the current variable with one of the following editing methods.\nYou can not rewrite everything all at once:\n\nYou have Four Editing Methods:\n1. ADD new elements(instruction) to address each specific feedback.\n2. ADD Examples (e.g., input-reasoning-answer) for tasks that require strong reasoning skills.\n3. Rephrase existing instruction(for more clarity), Replace existing sample with another, to address the feedback.\n4. DELETE unnecessary words to improve clarity.\n\nThese SIX prompting techniques can be a helpful direction.\n1. Set Context and Role: Establish a specific identity or domain expertise for the AI to guide style, knowledge, and constraints.\n2. Be Specific, Clear, and Grammarly correct: Clearly define instructions, desired format, and constraints to ensure accurate and relevant outputs with regards to the feedback.\n3. Illicit reasoning: "chain-of-thought" (e.g. "think step by step") helps the model reason better.\n4. Examples: Construct examples(e.g., input(optional)-reasoning(required)-answer) especially for tasks that require strong reasoning skills.\n5. Leverage Constraints and Formatting: Explicitly direct how the answer should be structured (e.g., bullet points, tables, or tone).\n6. Self-Consistency / Verification Prompts: Prompt the model to check its own logic for errors, inconsistencies, or missing details.\n\nYour final action/reasoning  = one of FOUR editing method + one of SIX prompting technique.\n\nYou must stick to these instructions:\n1. **MUST Resolve concerns raised in the feedback** while preserving the positive aspects of the original variable.\n2. **Observe past performance patterns** to retain good qualities in the variable and past failed ones to try things differently.\n3. **System Awareness**: When other system variables are given, ensure you understand how this variable works in the whole system.\n4. **Peer Awareness**: This variable works together with Peer variables, ensure you are aware of their roles and constraints.\n5. **Batch Awareness**: You are optimizing a batch of input data, ensure the change applys to the whole batch (except while using demonstration.)\n\n{{output_format_str}}\n\n{% if instruction_to_optimizer %}\n**Additional User Instructions**: {{instruction_to_optimizer}}\n{% endif %}\n', in_context_examples: List[str] = None, max_past_history: int = 3, max_failed_proposals: int = 5, steps_from_last_improvement: int = 0, one_parameter_at_a_time: bool = True)

Bases: TextOptimizer

Textual Gradient Descent(LLM) optimizer for text-based variables.

proposing: bool = False

params_history: Dict[str, List[HistoryPrompt]] = {}

failed_proposals: Dict[str, List[HistoryPrompt]] = {}

current_tgd_output: Dict[str, TGDData | None] = {}

params: Iterable[Parameter] | Iterable[Dict[str, Any]]

constraints: List[str]

one_parameter_at_a_time: bool

property constraint_text

Returns a formatted string representation of the constraints.

Returns:

A string containing the constraints in the format “Constraint {index}: {constraint}”.

Return type:

str

increment_steps_from_last_improvement()

reset_steps_from_last_improvement()

add_score_to_params(val_score: float)

add_score_to_current_param(param_id: str, param: Parameter, score: float)

add_history(param_id: str, history: HistoryPrompt)

render_history(param_id: str) → List[str]

add_failed_proposal()

Save a copy of the current value of the parameter in the failed proposals.

render_failed_proposals(param_id: str) → List[str]

update_gradient_memory(param: Parameter)

zero_grad()

Clear all the gradients of the parameters.

set_target_param()

propose()

Proposing a value while keeping previous value saved on parameter.

revert()

Revert to the previous value when the evaluation is worse.

step()

Discard the previous value and keep the proposed value.

to_dict()

sum_ops(params: List[Parameter]) → Parameter

Represents a sum operation on a list of variables. In TextGrad, sum is simply concatenation of the values of the variables.

Parameters:

variables (List[Variable]) – The list of variables to be summed (concatenated).

Returns:

A new variable representing the sum of the input variables.

Return type:

Variable

class Sum

Bases: GradComponent

The class to define a sum operation on a list of parameters, such as losses or gradients.

It enables gradients combination of a batch of data samples.

name: str = 'Sum'

forward(params: List[Parameter]) → Parameter

Performs the forward pass of the sum operation. This is a simple operation that concatenates the values of the parameters.

Parameters:

params (List[Parameter]) – The list of parameters to be summed.

Return type:

Parameter

backward(summation: Parameter, *args, **kwargs)

Computes gradients for the predecessors of the sum operation. There is no gradient computation for the sum operation itself. It is a simple way to combine multiple losses for convenience.

sum.backward() => [loss1.backward(), loss2.backward(), …]

Parameters:

summation (Parameter) – The parameter representing the sum.