Tracing and evaluation serve different jobs

Tracing shows what happened in production; evaluation judges whether the output (see: structure in structure out for llm calls) was good. People conflate them because platforms bundle both, but they prevent different failure modes. Tracing catches issues reactively after users see them. Evaluation catches regressions proactively, before they ship. Pair an observability tool with a dedicated eval tool for real regression testing.

Langfuse exemplifies the conflation: it does both, but its evaluation features trail dedicated eval tools. For teams that only trace, quality drift goes unnoticed until users complain. For teams that only evaluate, production surprises are invisible until they become incidents. The right architecture uses both: tracing for real-time visibility into what agents do in production, and evaluation for regression testing before changes ship. Without both, you are blind in one direction.