The pipeline returned a report. It had nine sections, a Howey four-factor table, a risk matrix with six categories, and a supply schedule broken into team, early investors, community, and treasury. Every cell read N/A.
The upstream extraction stage had returned an empty array. Zero information points. The downstream stage rendered the document anyway, because an empty array is a valid array, and a template does not check whether it has anything to say.
I have audited enough broken data feeds to recognize this shape. In 2017 I ran 14 ICO whitepapers through a fixed compliance checklist and rejected 11 for undefined utility. The checklist worked because it had a stop condition. This pipeline had none.
Verification precedes valuation; always. A system that cannot refuse to produce output is not an analytical system. It is a formatting engine.
Crypto research has quietly become a two-stage machine. Stage one parses a source — an article, a governance forum post, a tokenomics doc — into a structured payload: title, source, an enumerated list of information points with IDs, a one-line thesis with the author's stance, domain tags, named protocols, time-sensitivity flags, and a source-quality grade. Stage two consumes that payload and emits nine analytical dimensions: technical, tokenomics, market, ecosystem, regulatory, team, risk, narrative, and supply-chain transmission.
The design is sound in principle. It forces provenance. Every conclusion in stage two is supposed to trace back to an enumerated information point. That constraint — conclusion must map to a specific IP — is the only thing separating research from opinion.
But the contract between the two stages is implicit. Stage two checks types. It does not check cardinality. A payload with zero information points, a blank thesis, and unclassified domain tags passes every type check ever written.
Failure one is schema-level. [] is a valid array. A JSON validator sees an empty list and returns true. It does not know that the downstream engine is about to fill 60 table cells with the string "N/A" and call the result a risk assessment. The fix is mechanical: assert length greater than zero on the information-point list, assert non-null on the thesis, assert a minimum count of named protocols. Gate before generation. Terminate, do not degrade.
Four checks, run before any generation step:
- Information-point count greater than zero.
- Thesis field non-null and non-empty.
- At least one resolvable protocol identifier.
- Per-point source tag present on every entry.
Four booleans. Any false halts the batch.
Failure two is visual. An N/A-filled template is visually indistinguishable from a completed report at a glance. It has headers, borders, and the same word count. Density of formatting substitutes for density of evidence. I call this template hallucination. The document is not wrong in any single assertion — every cell is honestly labeled — yet the aggregate reads as diligence performed.
Failure three is the most expensive. N/A is not "no risk." It is "not assessed." A retail reader scanning a risk matrix where every row says N/A will frequently conclude the asset cleared screening. The distinction between an evaluated-clean result and an unevaluated result is the entire product of a due-diligence process. Collapsing it destroys the output's meaning while preserving its appearance.
Failure four is sourcing asymmetry. Stage one requires a source tag on each information point for a reason. A project's own claim of 10,000 TPS and a third party's measured 1,200 TPS are both "information points." Only one can support a conclusion at high confidence. Strip the per-point source attribution and the downstream engine has no way to downgrade — so it either trusts everything or trusts nothing. Both are wrong.
Failure five is confidence laundering. The downstream engine grades its own conclusions — high, medium, low — without inheriting any upstream grade. When the input carries no source-quality flag, the output still carries a confidence label. A number with no denominator. Remove the inheritance chain and confidence becomes decoration, printed identically on verified and empty payloads alike.
I learned this the hard way in 2022. During the Terra collapse I executed a withdrawal protocol across three DeFi venues in 45 minutes and preserved 85% of a €15,000 book. That worked because every trigger was binary and every input was verified against a live source. When a feed went stale, the bot did not interpolate. It halted. The halt was the feature.
The industry measures research by coverage. Four hundred assets monitored. Nine dimensions per asset. Weekly cadence. Coverage is a vanity metric. The scarce resource is refusal.
A pipeline that cannot output nothing will eventually output something false — not through malice, but through arithmetic. If the generation stage always runs, the fraction of outputs with zero evidentiary base is exactly the fraction of inputs that failed upstream parsing. In a batch job, that number is never zero. In a 400-asset coverage claim, a 5% empty-output rate is 20 reports per cycle that read as diligence and contain none. OCR failures, paywalls, image-only pages, non-Latin scripts, empty body extraction — each one produces a valid empty payload and a completed nine-dimension report.
The counterintuitive part: the more polished the template, the higher the risk. Polished templates invite trust. A reader who receives a rough, obviously incomplete note applies skepticism automatically. A reader who receives a bordered risk matrix applies none. Formatting is a trust signal, and trust signals are attack surface.
My own workflow now runs a pre-flight gate on every ingestion batch. Information-point count must exceed zero. Thesis must be non-null. At least one named protocol must be resolvable. If the gate trips, the batch quarantines and a human reviews the raw fetch. In 2025 I ran 10,000 backtested trades through an AI agent and cut manual emotional interference by 90% — but the agent never decided whether a data feed was valid. That boundary stayed human. Human-in-the-loop is not a slogan about oversight. It is a specific allocation of which decisions cannot be automated.
Run the audit on your own stack this week. Take the last hundred research outputs and ask one question: how many are structurally indistinguishable from a null input — same layout, same cell count, same confidence language, zero traceable facts?
If you cannot answer that number, your alpha is a formatting artifact. Verification precedes valuation; always. The next question is harder: what would your process look like if it were allowed to say nothing at all?
