OpenAI Integration¶

AdalFlow provides comprehensive support for OpenAI’s models through the new Response API (responses.create), which unifies all model interactions including text generation, vision, reasoning, and image generation.

Quick Start¶

Installation¶

pip install adalflow[openai]

Set up your API key from platform.openai.com.

Basic Usage¶

import adalflow as adal
from adalflow.components.model_client import OpenAIClient

# Initialize the client - automatically uses the Response API
generator = adal.Generator(
    model_client=OpenAIClient(),
    model_kwargs={"model": "gpt-4o-mini"}
)

response = generator({"input_str": "Hello, world!"})
print(response.data)

Response API Overview¶

OpenAI’s new Response API provides a unified interface for all model capabilities:

Key Features¶

Unified Interface: Single API endpoint for all model types
Typed Streaming: Structured events (ResponseTextDeltaEvent, etc.)
Native Tools: Built-in support for image generation
Simplified Multimodal: Clean handling of text + image inputs

API Differences¶

Feature	Chat Completions API	Response API
Endpoint	`chat.completions.create`	`responses.create`
Input Format	`messages`	`input` (string or messages)
Multimodal	Complex content arrays	Simplified with `images` param
Streaming	Untyped chunks	Typed events
Tools	Function calling	Native tool types

Model Capabilities¶

Text Generation¶

Basic and streaming text generation with GPT models:

# Basic generation
generator = adal.Generator(
    model_client=OpenAIClient(),
    model_kwargs={
        "model": "gpt-4o-mini",
        "temperature": 0.7,
        "max_tokens": 150
    }
)

# Streaming
from adalflow.components.model_client.utils import extract_text_from_response_stream

streaming_gen = adal.Generator(
    model_client=OpenAIClient(),
    model_kwargs={"model": "gpt-4o", "stream": True}
)

response = streaming_gen({"input_str": "Tell me a story"})
for event in response.raw_response:
    text = extract_text_from_response_stream(event)
    if text:
        print(text, end="", flush=True)

Vision Models¶

Analyze images from URLs, local files, or base64 data:

# Single image
vision_gen = adal.Generator(
    model_client=OpenAIClient(),
    model_kwargs={
        "model": "gpt-4o",
        "images": "https://example.com/image.jpg"
    }
)

# Multiple images
multi_gen = adal.Generator(
    model_client=OpenAIClient(),
    model_kwargs={
        "model": "gpt-4o",
        "images": ["image1.jpg", "image2.jpg"]
    }
)

# Local file (auto-encoded to base64)
local_gen = adal.Generator(
    model_client=OpenAIClient(),
    model_kwargs={
        "model": "gpt-4o",
        "images": "/path/to/local/image.jpg"
    }
)

Reasoning Models¶

O1 and O1-mini models for complex problem solving:

reasoning_gen = adal.Generator(
    model_client=OpenAIClient(),
    model_kwargs={
        "model": "o1-mini",  # or "o1"
        "reasoning": {
            "effort": "medium",  # low, medium, high
            "summary": "auto"    # detailed, auto, none
        }
    }
)

response = reasoning_gen({"input_str": "Solve this complex problem..."})
print(response.data)  # Solution
print(response.thinking)  # Reasoning process

Image Generation¶

Generate images using the new tools API:

image_gen = adal.Generator(
    model_client=OpenAIClient(),
    model_kwargs={
        "model": "gpt-4o",
        "tools": [{"type": "image_generation"}]
    }
)

response = image_gen({
    "input_str": "Generate a sunset over mountains, watercolor style"
})

# Save generated images
if response.images:
    saved_paths = response.save_images(
        directory="output",
        prefix="sunset",
        format="png"
    )
    print(f"Images saved to: {saved_paths}")

Mixed Generation¶

Generate both text and images in one call:

mixed_gen = adal.Generator(
    model_client=OpenAIClient(),
    model_kwargs={
        "model": "gpt-4o",
        "tools": [{"type": "image_generation"}]
    }
)

response = mixed_gen({
    "input_str": "Write a haiku and generate an image for it"
})

print(response.data)  # Haiku text
if response.images:
    response.save_images(prefix="haiku")

Text Embeddings¶

Generate embeddings for semantic search:

from adalflow.core import Embedder

embedder = Embedder(
    model_client=OpenAIClient(),
    model_kwargs={"model": "text-embedding-3-small"}
)

texts = ["text1", "text2", "text3"]
embeddings = embedder(input=texts)

Helper Functions¶

The integration provides several helper functions for working with the Response API:

from adalflow.components.model_client.utils import (
    extract_text_from_response_stream,
    extract_complete_text_from_response_stream,
    is_response_complete,
    process_images_for_response_api,
    format_content_for_response_api
)

Code Examples¶

Full working examples are available in:

Tutorial Script: tutorials/models/openai_models.py
Integration Notebook: notebooks/integration/openai_integration.ipynb

Source Code¶

OpenAI Client: openai_client.py
Response Utils: utils.py

Best Practices¶

Image Handling: URLs are most efficient; local files are auto-encoded to base64
Streaming: Use extract_text_from_response_stream() for text extraction
Error Handling: Always wrap API calls in try-except blocks
Performance: Set appropriate max_tokens and use streaming for better UX
Monitoring: Track usage with response.usage