See what my agents do
I want to watch what my AI agents actually do once they're running. Here's a stack that does this job, and a worked example of it in use.
A platform engineer watches what agents do in production
A platform engineer runs agents in production for their team. “The agent did something weird” is not an answer they can give on call, so they need to see exactly what each run did.
Their stack has three pieces. The agent app is what the team ships. Instrumentation is a library they add that records each step an agent takes — every tool call, every model request — as a trace. An observability platform stores those traces and lets the engineer replay a run, find the failing step, and see the exact input and output.
The discipline is to treat a trace as the source of truth: once every run is recorded, debugging an agent looks like debugging any other distributed system. Instrumentation docs for tracing platforms walk through this end to end.
Sources
- Langfuse instrumentation docs (
raw/0004-langfuse-docs.md)
The stack
Click a box to filter the list to just the tools that fill it.
flowchart LR
app(("Your agent app"))
instrument["Instrumentation"]
observe["Observability"]
app -->|records each step with| instrument
instrument -->|sends traces to| observe Instrumentation
The library that records each step your agent takes.
- OpenLLMetry — OpenTelemetry instrumentation for LLMs.
Observability
Where traces are stored and shown so you can inspect them.
- AgentOps — Observability built specifically for AI agents.
- Arize Phoenix — Open-source LLM tracing and eval.
- Comet Opik — Open-source eval and tracing from Comet.
- Fiddler — AI observability and governance for enterprise.
- Helicone — Proxy-based LLM observability with instant cost tracking.
- Laminar — Open-source observability and eval for AI agents.
- Langfuse — The most-used open-source LLM observability tool.
- LangSmith — LangChain's observability and eval platform.
- Lunary — Open-source LLM observability and prompt management.
- Pydantic Logfire — Observability from the Pydantic team.
- SigNoz — Open-source OpenTelemetry observability.
- Weights and Biases Weave — LLM tracing and eval inside W&B.