diff --git a/content/docs/ingest-data/ai-agents/openai.mdx b/content/docs/ingest-data/ai-agents/openai.mdx index 01b58e9..f9e6e3d 100644 --- a/content/docs/ingest-data/ai-agents/openai.mdx +++ b/content/docs/ingest-data/ai-agents/openai.mdx @@ -1,327 +1,280 @@ --- title: OpenAI -description: Log OpenAI API calls and responses to Parseable +description: Send OpenAI Python SDK traces to Parseable using OpenTelemetry --- -Log OpenAI API calls, responses, and token usage to Parseable for LLM observability. +import { Step, Steps } from 'fumadocs-ui/components/steps'; +import { Tab, Tabs } from 'fumadocs-ui/components/tabs'; -## Overview +The OpenAI Python SDK does not emit OpenTelemetry data on its own — there is no built-in `openai.instrument()`. (The separate `openai-agents` package has its own built-in tracing, but that's for building agents, not for instrumenting plain `openai` client calls.) To get spans out of a plain `client.chat.completions.create()` call, you need a third-party instrumentor that monkey-patches the client. This guide covers the two most common ones: -Integrate OpenAI with Parseable to: +- **[OpenLIT](https://github.com/openlit/openlit)** — one-call `openlit.init()`, ships its own OTLP exporter setup, includes token cost calculation out of the box. Also what [CrewAI](/ingest-data/ai-agents/crewai) and [LiteLLM SDK](/ingest-data/ai-agents/litellm-sdk) integrations in this hub use. +- **[OpenInference](https://github.com/Arize-ai/openinference)** — a standard OpenTelemetry instrumentor (`OpenAIInstrumentor().instrument(tracer_provider=...)`), so you set up the OTel `TracerProvider`/exporter yourself and it composes cleanly with other OpenInference instrumentors (e.g. `CrewAIInstrumentor`) on the same provider. -- **API Logging** - Track all API calls and responses -- **Token Usage** - Monitor token consumption and costs -- **Latency Tracking** - Measure response times -- **Error Analysis** - Debug failed requests -- **Prompt Engineering** - Analyze prompt effectiveness +Both emit GenAI semantic-convention spans and land in the same shape of Parseable dataset. Pick one — don't run both against the same client, they'll double-instrument. + +## How it works + +```text +Python application using OpenAI SDK + | + | OpenLIT or OpenInference patches the OpenAI client + | + | OTLP traces + v +Parseable + | + +--> openai-sdk-traces traces dataset in Parseable +``` + +Each chat completion produces a `chat ` span carrying GenAI attributes (prompt, response, tokens, cost) plus a child `POST` span for the underlying HTTP call to `api.openai.com`. If the model responds with tool calls, the follow-up request that sends tool results back is captured as its own linked span in the same trace. ## Prerequisites -- OpenAI API key -- Parseable instance accessible -- Python or Node.js application +Before you start, keep these ready: + +- A running Parseable instance +- A Parseable API key with ingest access +- Python 3.10 or newer +- An `OPENAI_API_KEY` + +## Set up OpenAI SDK with Parseable + + + + +### Install dependencies + + + + +```bash +pip install openai openlit +``` + + + + +```bash +pip install openai \ + openinference-instrumentation-openai \ + opentelemetry-sdk \ + opentelemetry-exporter-otlp-proto-http +``` + + + -## Python Integration + + -### Basic Wrapper +### Instrument the client before making requests + +Whichever instrumentor you pick, it must run before you construct the `OpenAI` client, so the patch is in place when the client makes its first call. + + + + +`openlit.init()` sets up its own `TracerProvider` and OTLP exporter — no separate OTel setup needed. ```python -import openai -import requests -import time -from datetime import datetime -from functools import wraps - -PARSEABLE_URL = "http://parseable:8000" -PARSEABLE_AUTH = ("admin", "admin") -STREAM = "openai-logs" - -def log_to_parseable(log_entry): - try: - requests.post( - f"{PARSEABLE_URL}/api/v1/ingest", - json=[log_entry], - auth=PARSEABLE_AUTH, - headers={"X-P-Stream": STREAM} - ) - except Exception as e: - print(f"Failed to log: {e}") - -def log_openai_call(func): - @wraps(func) - def wrapper(*args, **kwargs): - start_time = time.time() - error = None - response = None - - try: - response = func(*args, **kwargs) - return response - except Exception as e: - error = str(e) - raise - finally: - duration = time.time() - start_time - - log_entry = { - "timestamp": datetime.utcnow().isoformat() + "Z", - "model": kwargs.get("model", "unknown"), - "endpoint": func.__name__, - "duration_ms": round(duration * 1000, 2), - "success": error is None, - "error": error - } - - if response: - usage = getattr(response, "usage", None) - if usage: - log_entry["prompt_tokens"] = usage.prompt_tokens - log_entry["completion_tokens"] = usage.completion_tokens - log_entry["total_tokens"] = usage.total_tokens - - log_to_parseable(log_entry) - - return wrapper - -# Wrap OpenAI client -client = openai.OpenAI() - -@log_openai_call -def chat_completion(**kwargs): - return client.chat.completions.create(**kwargs) - -# Usage -response = chat_completion( - model="gpt-4", - messages=[{"role": "user", "content": "Hello!"}] +import os + +import openlit +from openai import OpenAI + +openlit.init( + otlp_endpoint=os.environ["PARSEABLE_URL"], # e.g. http://:8010 + otlp_headers={ + "X-API-Key": os.environ["PARSEABLE_API_KEY"], + "X-P-Stream": "openai-sdk-traces", + "X-P-Log-Source": "otel-traces", + }, + service_name="openai-sdk-demo", + environment="production", + disable_batch=True, + disable_metrics=True, + disable_events=True, ) + +client = OpenAI(api_key=os.environ["OPENAI_API_KEY"]) + +response = client.chat.completions.create( + model="gpt-4o-mini", + messages=[{"role": "user", "content": "Say hello in five words."}], +) +print(response.choices[0].message.content) ``` -### Comprehensive Logger +`disable_batch=True` exports each span as soon as it finishes, which is useful for short-lived scripts. Remove it for long-running services so spans batch and export on a timer instead. + + + + +OpenInference is a plain OTel instrumentor — you build the `TracerProvider` and exporter yourself, then hand them to `OpenAIInstrumentor().instrument(...)`. This is the same pattern the [CrewAI](/ingest-data/ai-agents/crewai) integration uses to combine `CrewAIInstrumentor` and `OpenAIInstrumentor` on one provider. ```python -import openai -import requests -import json -import hashlib -from datetime import datetime -from typing import Optional, Dict, Any - -class OpenAILogger: - def __init__(self, parseable_url: str, dataset: str, username: str, password: str): - self.parseable_url = parseable_url - self.dataset = dataset - self.auth = (username, password) - self.client = openai.OpenAI() - - def _log(self, entry: Dict[str, Any]): - try: - requests.post( - f"{self.parseable_url}/api/v1/ingest", - json=[entry], - auth=self.auth, - headers={"X-P-Stream": self.dataset}, - timeout=5 - ) - except Exception as e: - print(f"Logging failed: {e}") - - def _hash_content(self, content: str) -> str: - return hashlib.sha256(content.encode()).hexdigest()[:16] - - def chat(self, messages: list, model: str = "gpt-4", **kwargs) -> Any: - start_time = datetime.utcnow() - request_id = self._hash_content(json.dumps(messages) + str(start_time)) - - log_entry = { - "timestamp": start_time.isoformat() + "Z", - "request_id": request_id, - "type": "chat_completion", - "model": model, - "message_count": len(messages), - "system_prompt": next((m["content"][:200] for m in messages if m["role"] == "system"), None), - "user_prompt": next((m["content"][:500] for m in messages if m["role"] == "user"), None), - **{k: v for k, v in kwargs.items() if k in ["temperature", "max_tokens", "top_p"]} - } - - try: - response = self.client.chat.completions.create( - model=model, - messages=messages, - **kwargs - ) - - end_time = datetime.utcnow() - log_entry.update({ - "success": True, - "duration_ms": (end_time - start_time).total_seconds() * 1000, - "prompt_tokens": response.usage.prompt_tokens, - "completion_tokens": response.usage.completion_tokens, - "total_tokens": response.usage.total_tokens, - "finish_reason": response.choices[0].finish_reason, - "response_preview": response.choices[0].message.content[:200] if response.choices else None - }) - - self._log(log_entry) - return response - - except Exception as e: - log_entry.update({ - "success": False, - "error": str(e), - "error_type": type(e).__name__ - }) - self._log(log_entry) - raise - -# Usage -logger = OpenAILogger( - parseable_url="http://parseable:8000", - dataset="openai-logs", - username="admin", - password="admin" +import os + +from openai import OpenAI +from openinference.instrumentation.openai import OpenAIInstrumentor +from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter +from opentelemetry.sdk.resources import Resource +from opentelemetry.sdk.trace import TracerProvider +from opentelemetry.sdk.trace.export import BatchSpanProcessor + +provider = TracerProvider( + resource=Resource.create({"service.name": "openai-sdk-demo"}) +) +exporter = OTLPSpanExporter( + endpoint=f"{os.environ['PARSEABLE_URL']}/v1/traces", + headers={ + "X-API-Key": os.environ["PARSEABLE_API_KEY"], + "X-P-Stream": "openai-sdk-traces", + "X-P-Log-Source": "otel-traces", + }, ) +provider.add_span_processor(BatchSpanProcessor(exporter)) +OpenAIInstrumentor().instrument(tracer_provider=provider) -response = logger.chat( - messages=[ - {"role": "system", "content": "You are a helpful assistant."}, - {"role": "user", "content": "What is the capital of France?"} - ], - model="gpt-4", - temperature=0.7 +client = OpenAI(api_key=os.environ["OPENAI_API_KEY"]) + +response = client.chat.completions.create( + model="gpt-4o-mini", + messages=[{"role": "user", "content": "Say hello in five words."}], ) +print(response.choices[0].message.content) ``` -## Node.js Integration - -```javascript -const OpenAI = require('openai'); -const axios = require('axios'); - -const PARSEABLE_URL = process.env.PARSEABLE_URL || 'http://parseable:8000'; -const PARSEABLE_AUTH = Buffer.from('admin:admin').toString('base64'); - -class OpenAILogger { - constructor() { - this.client = new OpenAI(); - } - - async log(entry) { - try { - await axios.post(`${PARSEABLE_URL}/api/v1/ingest`, [entry], { - headers: { - 'Authorization': `Basic ${PARSEABLE_AUTH}`, - 'X-P-Stream': 'openai-logs', - 'Content-Type': 'application/json' - } - }); - } catch (error) { - console.error('Logging failed:', error.message); - } - } - - async chat(messages, options = {}) { - const startTime = Date.now(); - const model = options.model || 'gpt-4'; - - const logEntry = { - timestamp: new Date().toISOString(), - type: 'chat_completion', - model, - message_count: messages.length - }; - - try { - const response = await this.client.chat.completions.create({ - model, - messages, - ...options - }); - - logEntry.success = true; - logEntry.duration_ms = Date.now() - startTime; - logEntry.prompt_tokens = response.usage?.prompt_tokens; - logEntry.completion_tokens = response.usage?.completion_tokens; - logEntry.total_tokens = response.usage?.total_tokens; - logEntry.finish_reason = response.choices[0]?.finish_reason; - - await this.log(logEntry); - return response; - - } catch (error) { - logEntry.success = false; - logEntry.error = error.message; - logEntry.error_type = error.constructor.name; - await this.log(logEntry); - throw error; - } - } -} - -// Usage -const logger = new OpenAILogger(); -const response = await logger.chat([ - { role: 'user', content: 'Hello!' } -], { model: 'gpt-4' }); +`OTLPSpanExporter` here builds the URL as `{PARSEABLE_URL}/v1/traces` explicitly — unlike OpenLIT, it does not append the path for you. `BatchSpanProcessor` batches on a timer by default; for short scripts call `provider.force_flush()` (or `provider.shutdown()`) before exit so spans aren't lost. + + + + +The dataset named in `X-P-Stream` is created automatically on first ingest if it does not already exist. + + + + +### Tool calls + +Tool-calling requests instrument the same way under both instrumentors — no extra setup. OpenAI SDK-level tool call and result messages get captured as part of the same trace. + +```python +tools = [{ + "type": "function", + "function": { + "name": "get_weather", + "description": "Get current weather for a city.", + "parameters": { + "type": "object", + "properties": {"city": {"type": "string"}}, + "required": ["city"], + }, + }, +}] + +response = client.chat.completions.create( + model="gpt-4o-mini", + messages=[{"role": "user", "content": "What's the weather in Bengaluru?"}], + tools=tools, + tool_choice="auto", +) + +message = response.choices[0].message +if message.tool_calls: + messages = [{"role": "user", "content": "What's the weather in Bengaluru?"}, message] + for call in message.tool_calls: + # ... execute the tool, then append its result ... + messages.append({ + "role": "tool", + "tool_call_id": call.id, + "content": '{"city": "Bengaluru", "temp_c": 28}', + }) + client.chat.completions.create(model="gpt-4o-mini", messages=messages) +``` + + + + +### Send a few requests + +Run the application a few times with different models and prompts, including at least one tool-calling request, to see the full range of spans in Parseable. + + + + +## What you get in Parseable + +Open `openai-sdk-traces` from the Traces page. Each chat completion appears as a `chat ` span carrying the full set of GenAI attributes, with a child `POST` span for the HTTP call. Multiple calls in one process share `service.instance.id`, and tool-call follow-up requests link back to the originating trace via `span_trace_id`. + +## Useful fields + +| Field | Meaning | +| --- | --- | +| `gen_ai.provider.name` | Always `openai` for this integration | +| `gen_ai.request.model` | The model requested by the application | +| `gen_ai.response.model` | The model version that actually served the request | +| `gen_ai.operation.name` | The GenAI operation, such as `chat` | +| `gen_ai.input.messages` | The request messages, including system/user/tool roles | +| `gen_ai.output.messages` | The response messages and finish reason | +| `gen_ai.usage.input_tokens` | Input token count | +| `gen_ai.usage.output_tokens` | Output token count | +| `gen_ai.usage.cost` | Computed request cost (OpenLIT computes this; OpenInference may not, depending on version) | +| `gen_ai.server.time_to_first_token` | Time to first token | +| `span_status_code` | Whether the span completed successfully (`1` = OK) | +| `span_trace_id` / `span_parent_span_id` | Use these to reconstruct the chat span and its child HTTP span | + +## Query examples + +Total requests and tokens by model: + +```sql +SELECT + "gen_ai.request.model" AS model, + COUNT(*) AS requests, + SUM(CAST("gen_ai.usage.input_tokens" AS BIGINT)) AS input_tokens, + SUM(CAST("gen_ai.usage.output_tokens" AS BIGINT)) AS output_tokens +FROM "openai-sdk-traces" +WHERE "gen_ai.operation.name" = 'chat' +GROUP BY model; ``` -## Querying OpenAI Logs +Error rate by model: ```sql --- Token usage over time -SELECT - DATE_TRUNC('hour', timestamp) as hour, - SUM(total_tokens) as total_tokens, - SUM(prompt_tokens) as prompt_tokens, - SUM(completion_tokens) as completion_tokens, - COUNT(*) as request_count -FROM "openai-logs" -WHERE timestamp > NOW() - INTERVAL '24 hours' -GROUP BY hour -ORDER BY hour DESC - --- Average latency by model -SELECT - model, - AVG(duration_ms) as avg_latency, - PERCENTILE_CONT(0.95) WITHIN GROUP (ORDER BY duration_ms) as p95_latency, - COUNT(*) as requests -FROM "openai-logs" -WHERE success = true -GROUP BY model - --- Error rate -SELECT - DATE_TRUNC('hour', timestamp) as hour, - COUNT(*) as total, - SUM(CASE WHEN success = false THEN 1 ELSE 0 END) as errors, - ROUND(SUM(CASE WHEN success = false THEN 1 ELSE 0 END)::float / COUNT(*) * 100, 2) as error_rate -FROM "openai-logs" -GROUP BY hour -ORDER BY hour DESC - --- Cost estimation (approximate) -SELECT - model, - SUM(prompt_tokens) / 1000.0 * 0.03 as prompt_cost, - SUM(completion_tokens) / 1000.0 * 0.06 as completion_cost, - SUM(prompt_tokens) / 1000.0 * 0.03 + SUM(completion_tokens) / 1000.0 * 0.06 as total_cost -FROM "openai-logs" -WHERE timestamp > NOW() - INTERVAL '30 days' -GROUP BY model +SELECT + "gen_ai.request.model" AS model, + COUNT(*) AS total, + SUM(CASE WHEN span_status_code != 1 THEN 1 ELSE 0 END) AS errors +FROM "openai-sdk-traces" +WHERE "gen_ai.operation.name" = 'chat' +GROUP BY model; ``` -## Best Practices +## OpenLIT or OpenInference + +Use **OpenLIT** when you want a single `init()` call, built-in cost calculation, and don't need to compose with other non-OpenInference instrumentors. + +Use **OpenInference** when you're already building an OTel `TracerProvider` for other instrumentors (e.g. combining `CrewAIInstrumentor` and `OpenAIInstrumentor` on one provider, as the [CrewAI integration](/ingest-data/ai-agents/crewai) does), or you want direct control over the exporter and processors. + +## Troubleshooting + +- **No traces appear** + + Confirm the instrumentor runs before the `OpenAI` client is constructed. If the client is imported and instantiated at module load time before instrumentation runs, it cannot be patched. + +- **Traces appear late or not at all in short scripts** + + OpenLIT: set `disable_batch=True` in `openlit.init()`. OpenInference: call `provider.force_flush()` or `provider.shutdown()` before the process exits — `BatchSpanProcessor` batches on a timer by default. + +- **Prompt or response text appears in telemetry and that's a concern** -1. **Hash Sensitive Data** - Don't log full prompts if sensitive -2. **Track Request IDs** - Correlate requests across systems -3. **Monitor Costs** - Set up alerts for token usage -4. **Log Errors** - Capture error details for debugging -5. **Sample High Volume** - Consider sampling for high-traffic apps + OpenLIT: pass `capture_message_content=False` to `openlit.init()`. OpenInference: check the instrumentor's config for a content-masking option, or set `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=false` (or the instrumentor-specific env var) before instrumenting. -## Next Steps +## See also -- Configure [Anthropic](/ingest-data/ai-agents/anthropic) logging -- Set up [LangChain](/ingest-data/ai-agents/langchain) tracing -- Create [dashboards](/user-guide/dashboards) for LLM metrics -- Set up [alerts](/user-guide/alerting) for cost thresholds +- [LiteLLM SDK](/ingest-data/ai-agents/litellm-sdk) +- [CrewAI](/ingest-data/ai-agents/crewai) +- [Pydantic AI](/ingest-data/ai-agents/pydantic-ai) +- [Traces](/user-guide/traces)