Last week a two-stage analysis report crossed my desk. Nine sections. Forty-one cells. Every one of them carried the same three words: N/A — insufficient information. No crash. No stack trace. No red text. Just a clean, formatted refusal, repeated across a template that had obviously been built to be filled.
Here is what made me stop scrolling. That report was the most honest artifact the pipeline had produced in months. Stage one — the extraction layer — had returned an empty information-point list. Stage two — the reasoning layer — refused to invent the rest. Nobody celebrated it, because honest refusal and broken software look identical on a dashboard. The temptation in that moment is to "fix" it, to let the model fill the gaps. That temptation is the vulnerability.
Context
The architecture is not exotic. It is the default shape of nearly every AI-driven research agent, and increasingly of on-chain monitoring agents too. Stage one reads a source document and decomposes it into "information points" — the minimum analyzable fact units. A protocol name. A supply figure. A governance vote. A deployment date. Stage two takes that list and runs it through fixed analytical dimensions: technical, tokenomics, market, ecosystem position, regulatory exposure, governance, risk, narrative, supply-chain transmission.
The contract between the two stages is total. Stage two has exactly one input, and it is stage one's output. No second opinion. No cross-check. No external oracle. This is a linear data pipeline wearing the costume of a reasoning system, and linear pipelines fail the way all linear pipelines fail: quietly, at the seam.
When stage one returns an empty list, the pipeline reaches a fork. It can fail open — produce a plausible, complete-looking report from nothing. It can fail closed — halt and surface the gap. Or it can fail silent — return zeros, defaults, and placeholders that look like data. The first is dangerous. The second is expensive. The third is the one that kills you, because it is invisible.
I have seen all three in production. The distinction matters more than any model benchmark, because the fork is decided not by the model's intelligence but by whether anyone wrote a refusal state into the schema.
Core
Consider why LLM stages default to failing open. A nine-section template with empty placeholders is an irresistible completion target. Next-token prediction wants to fill the skeleton; low temperature only makes the fabrication more confident. A model with no formal refusal state does not return "N/A." It returns a fluent, internally consistent, entirely invented report — and it will read better than the honest one. The honest report is boring. The hallucinated report has numbers in it.
The EVM solved this problem a decade ago. require() reverts. State rolls back. Gas is consumed. The transaction fails loudly and leaves a trace. Fail-closed is not a design preference on-chain; it is the consensus default. Off-chain analysis has no revert instruction. There is no opcode that says "I do not know." You have to build the refusal state yourself — a schema guard, a non-nullable dependency, a hard gate between stages. Security is not a feature, it is the foundation, and here the foundation is the empty check.
Picture the handoff in JSON. Stage one emits {"information_points": []}. A correct stage two sees an empty array and halts. A fail-open stage two sees a schema with nine string fields and fills them — not from data, but from the distribution of plausible crypto sentences it was trained on. The output is syntactically perfect and semantically empty. Nothing downstream can tell the difference, because the shape is right.
The cost asymmetry is what makes this exploitable. A fabricated analysis is cheap to produce and expensive to falsify — it takes a human auditor hours to discover that a cited "information point" was never in the source. A refusal, by contrast, costs one rerun. Adversaries optimize for exactly this ratio. Complexity is the bug; clarity is the patch.
Based on my audit experience, the failure rarely announces itself. In 2020 I forked Aave V1 and ran fifty custom liquidation scenarios under simulated oracle manipulation. Three edge cases fell out of the price-feed aggregation logic — none documented in the official audit reports. They surfaced for one reason: the test harness printed raw state. Balances, reserves, block numbers. Not a narrative summary. Had the harness summarized its results, I would have shipped the summary and missed the bug.
That is the same property the nine-section report preserved. An output that cannot be reproduced from its inputs is not analysis; it is narrative. A hallucinated figure has no test vector, because fabrication is non-deterministic — you cannot write a unit test that reliably produces the wrong number. The moment an output stops being a function of its input, it has left engineering and entered marketing.
Now push it one layer further, into the agent economy. If stage one is LLM-based and reads untrusted documents — a token whitepaper, a governance forum post, a "research" PDF — an adversarial input can induce fabricated information points. That is prompt injection reframed as a data-integrity compromise. The fabricated point then flows downstream as ground truth. No signature check catches it, because the bytes are valid. No oracle disagrees, because there is no oracle. I fuzzed exactly this class of attack in 2026, against an agent protocol whose oracle verification layer trusted off-chain LLM output. The bytecode never lies, only the intent does — and here the bytecode was a JSON array whose intent was a lie.
The defensive pattern is unglamorous. Treat the extractor as an untrusted component. Unit-test it on the null, the empty array, the malformed document, and the adversarially poisoned one. Assert that empty input produces a hard stop, not a soft warning. Log provenance for every information point — which bytes of which source produced it — so a downstream claim can be traced back to a byte range. If it cannot be traced, it should not be trusted.
There is a subtlety worth naming. Not every gap is an error. Some analyses legitimately lack data, and the correct output is genuinely "insufficient information." The discipline is distinguishing a missing input from a missing conclusion. A missing input is a fact about the source. A missing conclusion is a fact about the analysis. Conflating them is how a system starts answering questions nobody asked.
The lesson generalizes: pipeline integrity is bounded by its weakest handoff, not its strongest endpoint. Teams instrument the model and ignore the seam. They log inference latency and token spend while the array between stage one and stage two sits unvalidated. Code compiles, but does it behave?
Contrarian
The industry rewards agents that always answer. Fluency is the product; a filled template demos better than a halt. But a system with a formal refusal state is strictly more secure than one that always completes, and the market has not priced that in. The market prices hope; the auditor prices risk. The "N/A" is not a bug in the report. It is the security feature.
The real defect was never that stage one returned empty. It was that stage two had no schema-level guard distinguishing "empty" from "absent" from "not yet fetched." Most teams test their pipeline on good data and never once on the null, the empty array, or the poisoned document. Every edge case is a door left unlatched — and the door everyone forgets is the one that opens onto nothing.
There is a counter-argument worth taking seriously: that refusing to analyze is a way to avoid accountability, and that a good analyst should reason under uncertainty. True — but reasoning under uncertainty means assigning probabilities to stated assumptions, not manufacturing facts to fill a form. Uncertainty is a variable. Fabrication is a constant, and a constant is what gets an agent's transactions signed.
Takeaway
As AI agents move from advice to on-chain execution, fail-closed stops being a best practice and becomes a consensus requirement. My forecast is specific: the first nine-figure agent exploit will not break a signature scheme or an elliptic curve. It will be a hallucinated data point that passed every test, because nobody tested the null. The next question an auditor should ask is not "does it work?" but "what does it do when it has nothing?"