We spent a week running AI against real data work, one run at a time. Three runs, three different lessons. None of them was “AI is magic.” All of them were checkable.
The 34-second reel: what happens when AI meets data engineering.
Smarter pipelines: 3 became 4
We asked two AI models to add one column to a win-back list query: each customer’s total October spend. Both models did the ask. Both also fixed a bug nobody mentioned. The date filter belonged in the ON clause, not the WHERE clause. The list went from 3 customers to 4. Dee, the one customer with zero October orders, was the one the old filter had silently dropped. Neither model said a word about the fix. The diff found it.
Faster insights: 10 wrong, then 9 right
We asked for our best marketing channel, with a wide sales table and one sentence of business context: podcast subscriptions count as attributed revenue. Without that sentence, the pick was wrong 10 times out of 10. Paid search on top every time. With the sentence, the pick was right 9 times out of 9. Podcast on top. One sentence closed the gap on every answer. The context was not garnish. It was the difference between a wrong answer and a right one.
Bigger possibilities: $920, twelve times
We asked two AI models for October revenue from the same wide sales table, 12 times. All 12 said $920. The real number was $400. The order total repeats on every line item, so summing it double-counts. The models read the data fine. The grain was wrong. Roll up to one row per order before you join.
The rule from the week
Check the grain, diff the output, then ship. The AI did real work in all three runs. It also made real mistakes in all three runs, and the mistakes were only visible because the output got checked against something known. The week did not prove AI is unreliable. It proved the check is the product.
This post started as an Instagram post →
- https://www.instagram.com/reel/DflKmpOE47w/
- https://www.threads.com/@dataeng.ai/post/DflKryRzRbP
- https://www.youtube.com/shorts/PMO37o_OZbM
- https://dataengai.com/posts/ai-fixed-our-bug/
- https://dataengai.com/posts/ai-oneline-context/
- https://dataengai.com/posts/ai-grain-kit/
Numbers above trace to these sources. If one moved, tell us and we fix it.


Talk it through
Argue with us on Instagram.