Why Trading Data Lies in Subtle Ways

Trading research often begins with a clean table: timestamp, open, high, low, close, volume. Real markets do not arrive as clean tables. They arrive as messages from different systems, on different clocks, through connections that fail.

Cleaning that stream is not clerical preparation before the “real” modelling. It determines what the model is allowed to know and whether a historical trade could have happened.

Event time is not receive time

An exchange may timestamp an event when it processes it. Your collector records when it receives the event. A news source has a publication time, an update time and the time your crawler finds it. A blockchain transaction has a signed time only indirectly, a mempool observation time where available and a block-inclusion time.

These timestamps answer different questions. A backtest that sorts everything by the timestamp most convenient in hindsight can leak future information into past decisions.

A defensible event record keeps, where available:

• venue and instrument;
• event or sequence identifier;
• exchange event time;
• local receive time;
• local processing time;
• collector and connection version; and
• raw payload or an integrity hash.

Sequence gaps corrupt order books silently

If a feed sends incremental updates, missing one cancellation or trade can leave the local book wrong for every subsequent decision. The dangerous failure is not always a crash. It can be a plausible-looking book built on an invalid state.

Collectors should know when continuity is lost, stop treating the book as valid, acquire a fresh snapshot using the venue’s documented procedure and record the gap. Hiding or interpolating missing order events makes later research impossible to audit.

Candle data can know too much

A five-minute candle labelled 12:00 often summarizes trades from 12:00 through 12:04:59. A decision made at 12:00 cannot use that candle’s final high, low, close or volume.

Other candle traps include:

• using the current, still-forming bar as final;
• mixing UTC and local exchange boundaries;
• forward-filling a missing price as though a trade occurred;
• silently replacing originally published data with a corrected history; and
• using one venue’s candle to simulate fills on another.

The rule is simple: every feature must be reproducible using only inputs available before the recorded decision.

Today’s asset list deletes yesterday’s failures

Testing the current top 100 tokens across five years creates survivorship bias. Many current tokens did not exist for the whole period, while failed, delisted and illiquid projects disappear from today’s list.

The same problem affects exchanges and trading pairs. A venue can remove a pair, change a contract, migrate a symbol or alter minimum order sizes. A point-in-time universe should preserve what was actually tradable at each date—including eventual failures.

Symbols are not identities

Ticker strings are reused. Wrapped and native assets differ. A perpetual contract is not its spot reference. Corporate actions are rare in crypto compared with equities, but redenominations, migrations, forks and token swaps create their own continuity problems.

Use stable internal instrument IDs linked to time-bounded venue metadata. Never join long histories on a ticker alone.

Hyperliquid’s current API documentation illustrates why metadata matters: spot and perpetual instruments use different asset representations, and its asset-ID guide warns that mainnet and testnet IDs differ. The exact scheme is venue-specific; the general lesson is to resolve instruments through authoritative metadata rather than screen labels.

News and social histories are especially fragile

An article can be edited after publication. Search results and engagement counts change. Deleted posts vanish. Community annotations appear later. A language model queried today may contain information unavailable at the simulated decision time.

For point-in-time research, preserve the retrieved content, event time, retrieval time, URL, content hash and transformation. If an input cannot be reconstructed historically, mark it unavailable. Do not ask a model to “remember what was known then.”

Training and production must see the same world

Headlands’ Elements of Statistical Learning review highlights practical systems that reduce divergence between training and inference. That gap is a frequent source of false confidence.

Examples include:

• research uses corrected historical candles while production uses raw ticks;
• training calculates a feature after a bar closes while production estimates it mid-bar;
• a notebook fills missing values globally using future observations;
• production rounds orders to venue increments but the simulation does not; and
• live signals run after network delay while the backtest acts at the event timestamp.

A strong system shares feature definitions and records the exact data version used in both environments.

A minimum data-quality ledger

For every dataset, publish or retain:

1. source and license;
2. collection period and timezone;
3. venue, product and instrument mapping;
4. raw versus normalized schema;
5. snapshot/update reconstruction rules;
6. known gaps and reconnects;
7. corrections and deduplication policy;
8. point-in-time universe construction; and
9. content hash or immutable version.

For every test run, record the data version rather than merely the date it was downloaded.

What can go wrong?

• Two collectors use unsynchronized clocks.
• Duplicate WebSocket messages count twice.
• A missing cancellation creates fake depth.
• Current token metadata is applied to old markets.
• Delisted assets are removed from the sample.
• News revisions are presented as original text.
• A model provider silently changes the model behind the same name.

What this does not prove

Clean data does not make a strategy good. It makes the result interpretable. A negative result from valid point-in-time data is more useful than an impressive result produced by leakage.

Our existing guide, How to Backtest an AI Crypto Strategy Without Fooling Yourself, turns these controls into a testing workflow. Next in this series: Should a Trading Model Predict Up or Predict a Price?.

Crypto market data FAQ

Why can clean-looking market data be misleading?

Tables can hide missing messages, clock differences, revised history, invalid order books and future information that was unavailable when a simulated decision occurred.

What is the difference between event time and receive time?

Event time records when a source processed or published information; receive time records when a collector obtained it. Both are needed to model what was knowable.

Why are sequence gaps dangerous in an order book?

A missed cancellation or trade can leave every later local-book state wrong while the book still looks plausible. The collector must invalidate and rebuild it.

How can candle data leak future information?

A candle labelled at its opening time contains high, low, close and volume from the entire interval, which were not known at the start.

What is survivorship bias in crypto backtests?

Using today's asset list excludes failed, delisted and illiquid projects, producing a historical universe that was not actually available at each test date.

Why should a system not identify assets by ticker alone?

Tickers can be reused across native, wrapped, spot and derivative products. Stable internal IDs linked to time-bounded venue metadata avoid false joins.

How should historical news data be preserved?

Store retrieved content, publication and retrieval times, URL, hash and transformations because articles, engagement and annotations can change later.

Why must training and production use consistent features?

Research results can be invalid when training uses corrected or end-of-bar data while production receives delayed raw events and calculates features differently.

What should a data-quality ledger record?

Record sources, licences, timezones, instrument mappings, schemas, reconstruction rules, gaps, corrections, universe construction and immutable versions or hashes.

Does high-quality data guarantee a profitable strategy?

No. It makes results interpretable and reproducible. A strategy can still have no edge even when its data is complete and point-in-time correct.

Risk disclosure: Historical datasets can be incomplete, revised or inconsistent across venues. Backtests based on them can materially misstate executable performance. This article is educational and not investment or financial advice.