Field notes on reliable AI.
Practical writing on evaluation, drift, cost, and the engineering that keeps AI systems working after launch.
How to Evaluate an AI Feature Before You Ship It
A pass rate without a sample size is not a number. Here is the arithmetic that tells you how many eval cases you actually need, and why "we tested it manually" is a demo rather than a baseline.
What to Log So You Can Debug an AI System Later
When a bad answer surfaces three weeks late, the answer alone tells you nothing. Write one trace record per model call, capturing the assembled context, and the failure becomes reproducible.
Your Token Bill Is a Design Problem, Not a Pricing Problem
Renegotiating your rate card moves an AI bill by percentages. Changing what you send on every call moves it by multiples. Here are the five design levers that actually control spend, with the arithmetic.