Remove all friction from looking at data

The single highest-leverage practice in LLM development is removing every barrier between you and your data. Build a custom viewer that renders traces in domain-specific ways, surfaces every source of information on one screen, and lets you label examples as good or bad with one click. Generic tools rarely fit; you need a purpose-built interface for your problem.

The heuristic is simple: keep reading traces until you feel like you are not learning anything new, then sample. Use LLMs to generate synthetic test cases, to label traces alongside humans, and to critique outputs. Track the correlation between model-based and human evaluation to know when you can trust the automated layer. You are doing it wrong if you are not looking at lots of data.