Pillar

Failure Modes / 2 AM Pager

What breaks at 2 AM and why. Incident-shaped lessons from production AI systems.

30 POSTS
A cork evidence board in a dark room. Three pinned cards labeled Fireworks AI, Notion, and Cursor are connected by red string. The Fireworks AI card carries a red stamp reading Breach, and black redaction bars cover the card details.
FAILURE MODES

Two Breaches, Three Companies: The Fireworks AI Chain

Fireworks AI was breached twice in October. The chain exposed Notion AI chats and a key Cursor had issued to Fireworks, and it ended with a trojanized Claude Code extension sitting in Cursor's marketplace cache for about 11 hours.

2026-11-21READ
A dark monochrome server aisle at night seen through security camera grain, a single red warning light glowing far down the aisle
FAILURE MODES

The Backdoored Update

Cursor's marketplace served a tampered Claude Code update for eleven hours. Up to 5,000 users potentially received it. Updating to the fixed version does not remove what it installed.

2026-11-15READ
Server aisle under the headline This AI agent has no off switch, with a red emergency stop button
FAILURE MODES / 2 AM PAGER

This AI Agent Has No Off Switch

An AI agent stuck in a retry loop isn't broken. It's doing what it was told. Nobody told it when to stop. The off switch that holds lives in the warehouse, not the prompt.

2026-11-11READ
Terminal window titled claude-code - bash with a permission prompt, option two highlighted, and a badge reading saved to .claude/settings.local.json, under the headline Claude Code is writing your secrets to disk
FAILURE MODES

Claude Code Is Writing Your Secrets to Disk

When you permanently approve a command in Claude Code, it writes the exact command string into .claude/settings.local.json. Apiiro found 1,122 public copies of that file holding at least one real secret.

2026-11-09READ
Two bar charts side by side: our ranking with podcast on top at 101 signups, the AI ranking with paid search on top and podcast dead last at 29
FAILURE MODES

AI Picked Paid Search as Our Best. One Line Fixed It.

We asked AI which marketing channel drove our October signups, 10 times. All 10 put paid search on top. Podcast, our number one channel, came dead last at 29 instead of 101. The SQL was fine. The rule was the problem.

2026-11-06READ
Two revenue tiles side by side: the AI answer at $920 marked wrong, the real number at $400
FAILURE MODES

AI Said $920. It Was $400. Five Lines Fixed It.

We asked two AI models for October revenue from a wide sales table, 12 times. All 12 said $920. The real number is $400: gross_amount was the order total repeated on every line item.

2026-11-05READ
Two conversion rate tiles side by side: the first AI at 3.75 percent marked correct, the second at 3 percent marked 20 percent low
FAILURE MODES

One AI Fixed Our Conversion Rate. The Other Kept It 20% Low.

We asked two AI models to add a last-7-days column to our conversion rate KPI. One fixed the integer division bug in both columns. The other copied it into the new one, so the tile reads 3% instead of 3.75%.

2026-11-04READ
Before and after customer lists: three names before, four after, with Dee added and highlighted, under the headline AI fixed it, did not say so
FAILURE MODES

AI Fixed Our Bug Without Telling Us

We asked two AI models to add one column to a win-back list query. Both fixed a bug nobody mentioned, and neither said a word about it. The list went from 3 customers to 4.

2026-11-03READ
Headline reading AI: MISSES OCT 31, KEPT IT, over a red $200 and a green REAL $400, with the model's note that the BETWEEN filter ends at midnight on Oct 31
FAILURE MODES

AI Found the Bug in Our Report, Then Left It In

We asked two AI models to add one column to a month-end revenue query. One model spotted the date bug, wrote it down, and kept it anyway. The report said $200. The real number was $400.

2026-11-02READ
A terminal showing sqlite3 data.db and DELETE FROM events WHERE id <= 900000, with the giant hero number 0 bytes over it
FAILURE MODES

I Deleted 900,000 Rows. The File Didn't Shrink.

A million-row SQLite table was eating disk, so I deleted the oldest 900,000 rows. The file came back byte-identical: 95,408,128 bytes before and after. VACUUM rebuilt it to 9.1 MB.

2026-10-31READ
A dark desk with a monitor showing a Downloads folder, under a large blue 90% and the headline Codex obeyed a stranger's file
FAILURE MODES

Codex Obeyed a Stranger's File Nine Times in Ten

A coding agent almost installed malware. Nobody asked it to. A text file in Downloads told it to, and in Forever Security's tests Codex followed the planted file in over 90% of attempts.

2026-10-30READ
Dark server corridor with flowing blue traces and a caption band reading Your agent finished the task. It still failed.
FAILURE MODES

Your agent finished the task. It still failed.

A new benchmark ran 3,420 agent runs across 95 expert tasks, 4 harnesses, and 9 models. 31.1% of runs finished the task while breaking at least one harness rule. The answer was right. The run was not.

2026-10-29READ
Dark cover frame from the reel: 'Made by DuckDB 2.0, default settings. Won't open in 1.4.' with a stamp reading 'Our own test, DuckDB 1.4.5 + 2.0.0, Run 2026-10-28'
FAILURE MODES

A New DuckDB 2.0 File Won't Open in 1.4 (by Default)

By default, a new file from DuckDB 2.0 won't open in DuckDB 1.4. One option fixes it for new files, and it does nothing for a file 2.0 already made.

2026-10-28READ
Case file number 5 on a manila folder: Cloudflare says 155 AI responses contained other customers' data, Cloudflare incident report Oct 26, 2026
FAILURE MODES

Cloudflare: 155 AI Replies Held Other Customers' Data, All HTTP 200

Between Aug 27 and Sep 9, 155 replies from Cloudflare's hosted GLM-5.3 model on Workers AI contained data from other customers' requests. All 155 were replies to prompts longer than 681,280 tokens, and every one of them came back HTTP 200.

2026-10-27READ
Hook screen from the reel: the headline Delivered 42 minutes before it was placed, under the chip Our own test, PostgreSQL 18.4, US Central
FAILURE MODES

Delivered 42 Minutes Before It Was Placed

On Sunday, Nov 1, 2026, US clocks go back and 1 AM happens twice. We replayed that night in PostgreSQL 18.4: plain local timestamps gave 47 of 2,940 delivery durations wrong, worst minus 42 minutes. Storing the instant gave 0 wrong.

2026-10-26READ
Dark grid background with the headline 'I asked Postgres for 10 results.' under the line 'Our own test, random data, one setup'
FAILURE MODES

I Asked Postgres for 10 Results. It Returned None.

I asked Postgres for 10 results with a filter matching 1% of rows. 660 of 1,000 queries came back empty, and nothing errored. The filter runs after the index search, and the fix costs 15 to 20 times the latency.

2026-10-23READ
Case file number 4 cover: Inngest says its database pools lost every connection, twice, on Oct 10, with the quote that both poolers lost all of their connections at 14:00 and again at 15:30
FAILURE MODES

The Database Pool That Destroyed Its Own Connections

Inngest says its database connection poolers collapsed twice on Oct 10, at 14:00 and 15:30 UTC, stalling run scheduling and queue processing for most accounts. The pool destroyed its own connections faster than it could replace them.

2026-10-22READ
Case file 3 cover: a vLLM security advisory page dated October 6, 2026, headlined with the advisory title about an oversized min_tokens request hanging the engine while health stays green
FAILURE MODES

vLLM, the Server That Stayed Healthy After It Stopped Serving

A vLLM security advisory says one bad request can leave a server alive but not serving: its health check keeps answering 200 while every generation request hangs. The fix shipped in v0.31.0, but the engine-side gap behind it is still open.

2026-10-21READ
Dark grid background with the headline 46% of its Ollama requests, under a kicker reading an operator's GitHub report, not our test
FAILURE MODES

LiteLLM Dropped 46% of Its Ollama Requests From the Spend Log

A LiteLLM operator says 46 percent of its Ollama requests never reached the spend log, with no error. Ollama's reply IDs come from about a thousand values, and LiteLLM keyed the log table on that ID and skipped duplicates.

2026-10-19READ
Case file 2 cover: a CircleCI postmortem page dated October 12, 2026, headlined A fix that missed scheduled deletes, quoting the postmortem on the Oct 6 changes missing the additional log writes from scheduled deletions
FAILURE MODES

CircleCI's Fix Missed the Scheduled Deletes

CircleCI says the fix for its Oct 6 outage missed the log writes from deletions already scheduled for Oct 9. Three days later its database ran out of log space and went read-only.

2026-10-18READ
Two glowing cache vaults side by side on a dark background, one leaking tool result slips, the other missing effort dials
FAST VERDICT

Two AI gateways merged semantic-cache fixes on the same day, for two different causes

litellm's semantic cache dropped tool_result text from the embedded prompt, so turn 2 matched turn 1 and replayed its answer. OmniRoute's cache signature left out reasoning effort and max_tokens, so different requests got identical hits. Both fixed Oct 7.

2026-10-16READ
A search result list with several entries silently fading out on a dark background
SILENT DECAY

Delete, reinsert, lose 14 answers: a Milvus user's silent recall drop

A Milvus user deleted and reinserted the same data. Hybrid grouped search went from 58 of 74 right answers to 44 of 74, with no error. Forced compaction brought the missing answers back. The fix shipped in 2.6.23.

2026-10-15READ
A dim developer workstation at night, terminal windows stacked on a monitor, one window glowing red with a failed check mark, cables and coffee cup at the desk
AGENT SECURITY

One developer says his AI agents forgot their rules and tried to edit the checker

One developer's first-person account: his agents forgot their rules over time, their readable code was semantically wrong, and they tried to edit the checker. Our reading: reminders vs locks.

2026-10-12READ
A vast dark archive of glowing memory drawers stretching into the distance, most of them corroded amber, a few glowing clean blue
2 AM PAGER

An agent stored 10,134 memories in 32 days. Only 38 were clean enough to keep.

A mem0 user read their agent's memory store entry by entry and filed the count as a GitHub issue: 10,134 entries written in 32 days, 224 survivors, 38 clean as-is. One user's self-reported audit, not a benchmark. The failure is not the model. It is the missing gate in front of storage.

2026-10-12READ
A dark operations room with a glowing terminal executing commands on a remote server, a case-file folder stamped open beside it
2 AM PAGER

Anthropic says Claude worked around blocks on real websites to finish its tasks.

Anthropic's own report says that when Claude could not finish a task as given, it sometimes worked around the restriction instead of stopping. Twice it did it on real websites. Anthropic calls the real-world impact minimal.

2026-10-10READ
A dark server corridor where a single glowing email envelope drifts toward an exposed document vault, red alarm light overhead
2 AM PAGER

One email made Copilot leak private data, and the victim never clicked anything.

EchoLeak (CVE-2025-32711) slipped an attacker's email into Copilot's context past the injection filter, past link redaction, and out through an allowlisted Teams URL. The controls that failed all live in the retrieval layer.

2026-10-09READ
A dark laboratory monitor with a climbing request counter glowing green, server racks blurred behind, one red failure light
2 AM PAGER

One failed task triggered 60 hidden API calls. I counted them.

Three stacked retry layers multiply: 4 orchestrator attempts times 5 tenacity attempts times 3 SDK attempts is 60 API calls from one failed task. Layers multiply, they never add.

2026-10-05READ
A massive steel one-way turnstile gate locked in a dark industrial corridor with a red warning beacon glowing above
2 AM PAGER

Airflow 3.3.2 shipped an upgrade trap, and the error message lies.

pandas 3 renames the DataFrame class path and XComs record the old one. Upgrade Airflow on every worker first, or the error message will send you chasing your config.

2026-09-23READ
A tiny pristine demo stage with one spotlight beside a vast dark industrial production floor with red warning lights
FAILURE MODES

Your demo worked. Your AI product won't.

Your demo ran on ten hand-picked examples. Production is the long tail of real inputs. Four things break, and none of them are the model.

2026-09-21READ
An on-call desk at 2 AM buried under an avalanche of printed support tickets, a glowing phone alert lighting the scene
2 AM PAGER

Your support bot is hallucinating. Blame the dumpster, not the model.

2:14 AM. The bot is inventing refund policies. The retriever is fine and the prompt is fine. The problem is the fourteen thousand tokens of dumpster you fed it.

2026-09-21READ