We asked AI which marketing channel drove our October signups. Ten separate times, all 10 answers put paid search on top. Podcast, our number one channel, came dead last: 29 signups instead of 101.
The 45-second reel: AI picked paid search as our best. One line fixed it.
The rule
Every answer credited the last channel a person touched before signing up. That is last touch attribution, and it is a reasonable default. We credit the first place a person found us. That is first touch, and it is our company rule. AI cannot know a company rule that nobody wrote down.
In this data, 35 people heard the podcast first and clicked a search ad last. Across 300 October signups, 215 had a different first channel than last channel, so the choice of rule moves most of the credit. Without the rule, the AI ranking was paid search 95, organic social 72, email 53, referral 51, podcast 29. With it, the ranking was podcast 101, organic social 70, referral 47, paid search 43, email 39.
What the models did
We gave the same request to two AI models from one provider, 10 tries without the definition and 9 with it. One run returned nothing twice and was left out. Without the line, all 10 answers put paid search on top, and 6 of the 10 even said in a comment that they were crediting last touch. With the line, all 9 answers put podcast back on top.
The one line, verbatim from our prompt: “we credit each signup to its first-touch channel, the earliest touch by that user before signed_up_at.”
The fix
Before you blame the AI, give it the rule. The model was not wrong about the data. It was answering a different question than the one our business asks, because the question as written did not say which touch gets the credit. One sentence in the prompt closed the gap on every answer.
What we did not verify
This was our own test on synthetic data and a company rule we chose, run Nov 2, 2026. Two AI models from one provider, 10 answers without the line and 9 with it. It is a small test, not a benchmark, and it says nothing about how often AI gets attribution wrong in general. The voiceover in the reel is AI-generated. Watch the reel on Instagram, Threads, and YouTube.
This post started as an Instagram post →
- https://www.instagram.com/reel/DfasESDCdBY/
- https://www.threads.com/@dataeng.ai/post/DfasESepofQ
- https://www.youtube.com/shorts/QOYHr5s-d1w
Numbers above trace to these sources. If one moved, tell us and we fix it.


Talk it through
Argue with us on Instagram.