At 03:14 UTC on a Tuesday, a research pipeline I'd been babysitting for eleven weeks returned a due diligence report on a token I won't name. Nine sections. Forty-five fields. Every one of them read the same three characters: N/A.
Cost: $0.42 in inference. Runtime: 94 seconds. Confidence intervals: none, because there was nothing to be confident about.
The easy read is that the document failed. It did not fail. It was the most accurate research artifact produced in that sector that week, and I say that having read the eleven that came before it — each one 2,400 words, each one with a technical scorecard, a tokenomics teardown, a six-color risk matrix, each one generated from the identical empty input.
The eleven were coherent. The eleven were confident. The eleven were fiction.
The difference between those eleven documents and mine was not model quality. It was not prompt engineering. It was one if statement, written at 2am in March, that checked whether the information-point list was empty before it allowed the generation stage to run.
That if statement is now the most valuable code I own. It took forty minutes to write. On conservative accounting it has saved me north of $60,000.
CONTEXT
Everyone runs agents now. That is not the interesting part. The interesting part is that almost nobody instruments the null path, and the null path is where the money actually leaks.
The architecture is standardized to the point of being boring. Four stages. Ingest: pull the source, whatever it is — a governance forum post, a Token Terminal export, an on-chain query, a news article. Extract: convert unstructured text into structured information points, each carrying a fact, a source, a timestamp. Score: run the points through a nine-dimension framework. Render: produce the document a human will read and, more importantly, act on.
The scoring layer is where the money is. It is also where the lie lives.
The framework itself is not exotic. Technical feasibility. Token economics. Market structure. Ecosystem position. Regulatory posture. Team and governance. Risk matrix. Narrative and expectation gap. Supply-chain transmission. Each dimension carries four to six sub-fields. Multiply it out and you land at roughly forty-five required slots.
Forty-five required slots is not a framework. It is a machine that converts absence into presence.
Language models are completion engines. That is the entire specification. Hand one a slot and it will fill the slot — not because it is lying, but because completion is the objective function. Coherence is the reward signal. Accuracy is not even in the loss.
So when the extract stage returned an empty array that Tuesday, the pipeline had two paths available and only one of them was implemented. It ran a try/except, wrote a warning to a log file nobody tails, and passed context = "" straight into the generation stage.
Why was the array empty? Because three weeks earlier the source site migrated its article markup from semantic tags to a JSON-LD payload. The extractor was still hunting for . One CSS selector, stale by twenty-one days, stood between "we have no data" and "here is a nine-dimensional analysis of a protocol."
The article existed. The parse did not. The pipeline had no opinion about the difference between "no information" and "insufficient information" — it had a schema, forty-five empty slots, and a token budget.
That is the whole failure mode. It is not dramatic. It is a warning written to a file that nobody opens.
CORE
The fabrication engine, measured
I ran an experiment on my own pipeline in April. I fed it a deliberately empty context and asked for the risk dimension only. Six risk categories came back. Severity ratings. Probability estimates. Mitigation measures with named owners.
The protocol it described did not exist. Not in the input, not on-chain, not anywhere.
By the model's own standard that output contained zero hallucinations, because the model was never asked a factual question. It was asked to populate a schema. It populated a schema. The failure was upstream, in a design decision made by a human who wanted the report to always have six risk categories.
Here is the arithmetic that matters. If each of forty-five slots has even a 60% chance of being fillable from plausible priors — and in crypto priors are abundant, because every protocol resembles three other protocols — then the probability that a document with at least thirty filled fields emerges from zero information is effectively one. Not 0.6. One. Because the generator does not sample each slot independently. It conditions each slot on the ones it already filled. Coherence compounds. Fabrication compounds with it.
I have audited four of these pipelines for funds over the past eighteen months. All four had a scoring stage. Two had a null path. Zero had a null path that produced a document anyone was willing to read.
The backtest that passed on noise
I built an AI trading agent in early 2025. Open-source base, my own feature layer, backtested against 2020-2024 data. Sharpe ratio came back at 35%. Not 3.5. Thirty-five.
I have been doing this long enough to know that a Sharpe of 35 is not a strategy. It is a bug with a good UI.
So I built a null dataset. Two hundred tokens. Dates shuffled within regime blocks. Prices replaced with geometric Brownian motion calibrated to each token's realized volatility and vol-clustering. Same features. Same agent. Same everything except the thing I was supposedly measuring.
Sharpe: 31%. Different seed: 28%. Third run: 33%.
The agent was measuring itself. Specifically, it was measuring an 80/20 split — roughly 80% of the reported edge came from the position-sizing overlay, 20% from the signal. And the sizing overlay was a volatility-targeting rule. GBM with vol clustering has vol clustering. So the agent found volatility. It found it in noise. It would have found it in a random walk with a memory.
The chart didn't lie. My dataset did. And the dataset lied in the exact same way an empty information-point list lies: by being a structure with no content, fed into a system that had no permission to say "I don't know."
The fix is not sophisticated. It is a gate. Run the strategy against N=1,000 shuffled datasets. Build the empirical distribution of Sharpe ratios under the null. If your observed Sharpe sits inside the 95th percentile of that distribution, you do not have alpha. You have a volatility harvester, and you will pay for the privilege of learning that in live markets.
Most retail-facing strategy products never run this test. Not because it is hard. Because the test sometimes returns "no edge," and "no edge" does not sell subscriptions.
Provenance or nothing
The information-point list is the least glamorous primitive in this entire stack, and it is the only one that makes a conclusion falsifiable.
Every point needs four fields. Fact. Source. Timestamp. Confidence. That is it. No embeddings, no vector store, no agentic loop. Four fields.
With them, a claim is attributable and therefore checkable. Without them, a claim is a sentence with good grammar.
The design decision that separates a null-aware scorer from a null-blind one is small enough to miss in code review:
def score(dimension, points):
if not points:
return {
"status": "N/A",
"reason": "no_information_points",
"confidence": None,
}
return {"status": "SCORED", "value": run(dimension, points)}
Two things are happening there. First, status is a first-class field with three legal values: SCORED, N/A, ABSTAIN. Two of the three are not numbers. Second, the empty branch returns before the model is ever invoked. The model cannot fabricate what it is never asked to generate.
I have watched three teams try to fix this downstream — better prompts, a stricter system message, a second model auditing the first. All three failed. You cannot prompt your way out of a schema that demands forty-five values. You have to change the schema.
The reverted transaction in the information layer
In 2021 I lost $4,000 on a mint. Not to a rug. To gas estimation. The transaction reverted, the ETH was gone, and the NFT never existed. I spent a weekend learning that theoretical value means nothing if the transaction reverts.
A fabricated research document is a reverted transaction in the information layer. You paid the cost — in attention, in conviction, in position size — and you received nothing. The difference is settlement time. A reverted mint tells you in twelve seconds. A fabricated scorecard tells you when the position goes against you, which is usually six weeks later, at which point you have already told three people about the thesis.
The cost asymmetry is brutal. An honest empty report costs $0.42. A confident fabricated report costs whatever you sized on it. In my case the eleven fabricated reports preceded a $30,000 allocation I closed at a $9,200 loss — not because the thesis was wrong, but because the thesis had never existed. I had sized against a document that described a protocol by analogy to three others and called the analogy analysis.
That is the number I keep. Not the 35% Sharpe. The $9,200. Risk isn't a feeling. It is an invoice, and it arrives whether or not your framework was honest enough to predict it.
Why null results never get published
The incentive is structural and it is not subtle.
A research shop that returns "N/A — insufficient information" eleven times in a row gets its subscription cancelled. A research shop that returns eleven confident scorecards, two of which are eventually right, gets a waitlist.
The scoring layer is monetized. The null path is not. So the null path gets deprioritized in sprint one and deleted in sprint two, and nobody notices, because the deletion is invisible — you only see it when the input is empty, and the input is almost never empty.
Almost never. That is the trap. A pipeline can run fourteen months at a 99% ingestion success rate and then hit a markup migration, a rate limit, a paywalled API, and produce a fully-formed document about nothing. The failure is rare enough that it never gets tested and common enough that it eventually fires.
Compare that to how desks that actually make money operate. During the January 2024 ETF approval window I ran premium/discount spreads between the spot ETFs and Coinbase. No nine-dimension scorecard. No narrative layer. I measured three numbers: the premium, the borrow cost, the settlement latency. I sized to the smaller of the two constraints. Fifty-plus trades over two weeks, roughly $8,000, risk-free in the technical sense.
That trade worked because its input was a number. Numbers cannot be empty. They can be wrong, but they cannot be absent, and wrong is a state you can detect with a second data source. Absent-dressed-as-present is not.
CONTRARIAN
The consensus framing is that AI research agents hallucinate. That framing is wrong in a way that keeps people from fixing anything.
They do not hallucinate. They comply. The hallucination lives in the schema design and in the analyst who commissioned a document with forty-five mandatory fields and then read it as though the fields were optional. Every fabricated risk matrix I have audited was the correct output of an incorrect specification.
The second-order point is harder to accept: the industry treats N/A as a failure state. Every dashboard flags it red. Every investment committee memo treats it as incomplete work. Every scoring rubric penalizes it.
But N/A is the only output in the entire crypto research stack with an error rate of zero. It is not useful. It is not alpha. It will not get you a seat on a panel. It is, however, correct — and correctness is a floor, not a ceiling. This sector has spent four years optimizing the ceiling while quietly pulling load-bearing bricks out of the floor.
And the floor is where the alibi was.
Here is the part I actually believe, and it is the part that makes people uncomfortable at dinners: most research in this market is written after the position, not before it. The document is not a decision input. It is a justification artifact, produced to be shown to someone — a partner, an LP, a compliance file, yourself at 3am. And a justification artifact with forty-five filled fields is enormously more persuasive than one with forty-five N/As. From the industry's perspective that is not a bug. It is the entire product.
Which is why the empty pipeline is genuinely dangerous to the business model. It does not merely fail to produce alpha. It removes the alibi. It says, in the plainest available language: you do not know, and you are about to size anyway.
Liquidity vanishes when the music stops. So does conviction. The difference is that liquidity comes back.
TAKEAWAY
The next eighteen months will produce a thousand agent frameworks and almost no null-aware ones, because the null path has no demo. Nobody funds a screenshot that reads insufficient information.
Build it anyway, and build it first. Three things, in order. A provenance schema with four fields per information point, enforced at the extraction boundary so nothing downstream can invent one. An abstain state as a first-class output, wired into your interface and your position-sizing logic, not buried in a log. And a null-distribution gate on every strategy before a dollar moves — a thousand shuffled datasets, an empirical Sharpe distribution, a hard stop if you land inside it.
None of that is alpha. All of it is the floor.
So here is the question I would put to anyone running capital in this market. How many positions in your book right now were opened on a document that would have been forty-five N/As if the pipeline had been honest?
You don't have to answer. The P&L already did.