Ten Empty Fields: What an AI Research Pipeline Proves When It Refuses to Fabricate

SamPanda • • NFT

Evidence suggests the most useful document produced by the AI-research boom so far contains no research at all. I received it last week: a second-stage deep-analysis output on an unnamed crypto asset, generated by a pipeline that had been fed a first-stage extraction package. Twelve fields were required. Ten came back empty. No title. No source. No publication timestamp. No sector classification. No stated authorial stance. No summary. Most critically, the information-point list — the field that every downstream conclusion was supposed to cite — was blank.

The first stage had also left its own instructions inside the output: "identify the project from the information points above," "judge source quality." These are prompts addressed to a stage that had already executed. They are the fossil record of a system talking to itself, and they are the clearest evidence I have seen this year that the extraction layer was never validated before it was promoted to production.

The second stage stopped. It printed a validation table, flagged ten empty rows, labeled the missing information-point list a fatal gap, and stated plainly that any deep analysis built on zero input would be a projection onto invented content. Then it published the minimum input set required to restart: any one of the original article, three or more sourced factual statements, or a title plus a project name plus a one-line event description.

I have read several hundred pages of AI-assisted due diligence in the last twelve months. This was the first document that told the truth. That is not a compliment to the pipeline. It is an indictment of the category.

The stack is standard now. Crawler, extractor, analyst, memo. A crawler fetches the page. An extractor converts prose into structured fields. An analyst stage consumes those fields and produces a memo. The economics are unambiguous: a language model costs cents per page, a junior analyst costs a salary, and the layer that was cut to capture that difference is precisely the layer whose only job was to say "we do not know." In 2019, an associate who could not locate a source flagged it, and the flag was the deliverable. In 2026 the flag has no owner, because there is no longer a human position between extraction and rendering.

The schema is where the damage happens. A template field is an invitation, and a helpful model reads an empty slot as a task rather than an error. In database terms, this pipeline had no NOT NULL constraint on its primary key. It had a text box. Completion pressure does the rest: output volume is proportional to the size of the template, not the substance of the input, so an empty input yields a long report. I have seen this pattern now across eleven vendor decks this quarter — every one of them dense, none of them traceable.

The chokepoint is not the model. It is the absence of a provenance contract between stages.

The report I received made that contract explicit. Every conclusion was required to resolve to the form "basis: information point N." Read that requirement carefully, because it is not stylistic. It is structural. A claim that cannot be indexed to a numbered fact is not evidence; it is a horoscope with a formatting pass. When the index is empty, the honest output is null, and the honest renderer displays null as unknown rather than as a default value of zero.

The most sophisticated line in the entire document — and I want to be precise about this, because it is the reason I kept it — was the disclaimer attached to every unassessable field: "N/A means evaluation was impossible due to missing information, and must not be interpreted as a risk-neutral judgment about the project." That distinction is absent from most paid research I audit. Human desks routinely collapse "not assessed" into "no issue found," because the second one gets printed and the first one gets a phone call from the client.

I watched that exact collapse at scale during the Terra review. Anchor's dashboard fields were all populated. Yield displayed, TVL displayed, inflows displayed — all of them real, all of them accurate, none of them wrong. The failure was in the schema, not the data. No field asked whether the 19.5% was revenue or debt. There was no column for "source of yield," so the report could be simultaneously factually correct and completely false. Empty-field discipline would not have saved that protocol. A required field labeled "funding source of payout" would have ended the conversation in one afternoon.

Trust is a variable; proof is a constant. A pipeline that cannot produce a null produces a liability, because the null is the only output that survives contact with a counterparty's lawyer.

This is why I treat research pipelines the way I treat contract systems. I run five checks. First, nullability: does the schema permit empty, and does the renderer surface empty as unknown? Second, field-level provenance: can every sentence in the memo be walked back to a URL, a publisher, and a timestamp? Third, reverse-instruction detection: does any stage output contain instructions addressed to itself? That symptom indicates contract drift between stages and almost always correlates with silent data loss upstream. Fourth, source independence rather than source count — five articles quoting one tweet are one source, and pipelines that count URLs will score that echo chamber as high confidence. Fifth, denominator presence: if the memo contains no revenue figure, no TVL figure, and no unlock schedule, the FDV discussion is decoration.

My own scar tissue points the same direction. In the 2026 audit of the first autonomous AI-agent wallet protocol, the vulnerability was not in the model's capability. It was in the assumption that the reward function's output could be treated as a policy without checking its determinism under adversarial market conditions. A logical race condition permitted unbounded minting under a narrow set of states. The code behaved exactly as written; the trust placed in it did not. Opaque generators and unaudited extraction layers fail the same way, and they fail quietly, because there is no transaction to trace and no gas to follow.

At FTX, the trace was the product. Five chains, fourteen wallet clusters, every hop documented to a standard that survived a courtroom. That work was possible only because the evidentiary chain was reconstructable after the fact. A research memo built on unindexed prose has no such chain. When the thesis breaks, there is nothing to walk backward through.

Now put a price on the error. A false negative costs an hour of analyst time and a delayed memo. A false positive compounds: it enters an allocation decision, it becomes a footnote in the next report, it becomes a quote in the next pitch. In a trending market, price corrects a fabricated thesis within days, which is why the AI-research category has survived its own sloppiness. In a range-bound market, that correction mechanism is switched off. Chop is for positioning, and positioning is where fabricated signals do the most damage — a protocol can lose 40% of its liquidity providers in a week while every surface metric stays flat, because nothing in the market is loudly falsifying anything. The half-life of a synthetic thesis in a sideways tape is measured in months.

Here is what the bulls get right, and they deserve the concession. The framework that halted was better engineered than most human desks. Requiring every conclusion to resolve to a numbered fact, and forbidding the conflation of unassessed with safe, is a stricter epistemic standard than the sell-side applies to most of what it publishes. The refusal itself proves guardrails are buildable. Extraction quality is an engineering problem, and engineering problems get solved on predictable curves.

They are wrong about what is being sold. Generation is now cheap, near-free, and commoditized past the point of differentiation. Verification is not. The bulls price research as a generation problem and expect generation's cost curve; verification scales with human attention and adversarial testing, neither of which halves every eighteen months. A pipeline that refuses a third of its inputs is commercially unusable, and I would not defend it. But the resolution to that is not more confident prose — it is promoting the null to a first-class output, routed to a human who fills the gap, rather than to a renderer that fills it with priors.

Within eighteen months, field-level provenance will be a line item in research-vendor due diligence, sitting next to SOC 2 and sitting there for the same reason. The procurement question that will separate the categories is a single request: show me a report where your system's answer was "unknown." A provider that cannot produce one does not operate a pipeline. It operates a generator with a template around it.

Trust is a variable; proof is a constant. When the fields come back empty, someone has to sign the memo anyway — and the only defensible signature is the one that writes "unassessed" and means it.