Historical Data

Introduction

Historical data is the raw material from which a backtest reconstructs past market conditions. But historical data is not the market itself. It is a representation produced by exchanges, vendors, corrections, corporate-action processing, symbol mapping, aggregation rules, and database design.

The expert question is therefore not simply whether enough years of data are available. It is whether the historical dataset represents the information, instruments, prices, liquidity, sessions, and decision environment that a trader could actually have experienced at the time.

1. Historical Data Is a Reconstruction

A historical database is a reconstruction of the past. Different sources can produce different representations of the same period because they apply different adjustment, consolidation, correction, and timestamp rules.

This matters because a backtest does not observe history directly. It observes the vendor's representation of history.

2. What Was Actually Tradable?

A historical study should define the tradable universe at each point in time. Today's list of liquid or successful securities is not automatically equivalent to the list that existed historically.

For stock strategies, relevant questions include which securities were listed, when they became tradable, when they were delisted, and whether historical corporate events changed their identity or price series.

3. Survivorship and Historical Universe

Survivorship bias occurs when instruments that disappeared from the market are systematically absent from the historical sample. The result can make a strategy look stronger because the dataset disproportionately retains securities that survived.

Historical universe construction is therefore a data problem as well as a backtesting problem.

4. Point-in-Time Information

Historical research must preserve what was knowable at each decision point. A value can be historically correct today while still being unavailable to the trader at the time.

This is especially relevant to fundamentals, classifications, index membership, analyst data, corporate information, and datasets that are later revised.

5. Price History: Raw, Adjusted, and Total-Return Concepts

Different price series answer different research questions. Raw prices can describe the historical trading environment. Adjusted series can make long-term price comparisons more coherent after corporate actions. Total-return data can incorporate distributions into return measurement.

The researcher must define which concept is being tested and ensure that indicators, entries, exits, and performance calculations use compatible data.

6. Corporate Actions Across Time

Splits, mergers, spin-offs, rights events, symbol changes, and distributions can alter the relationship between historical prices and present-day identifiers.

A backtest that fails to handle these events consistently can create artificial gaps, false signals, or incorrect return calculations.

7. Delistings, Mergers, and Acquisitions

A historical instrument can cease to exist in its original form. A strategy that remains invested until the final available record must have an explicit rule for delisting, acquisition, suspension, or conversion.

Ignoring these events can systematically remove difficult outcomes from the sample.

8. Market Sessions and Trading Calendars

Historical data must respect the actual trading calendar. Weekends, holidays, early closes, overnight sessions, and exchange-specific schedules affect indicators and holding periods.

A calendar mismatch can create false gaps or cause the engine to interpret a non-trading interval as missing market data.

9. Timeframe and Bar Construction

Daily, hourly, and intraday data represent different levels of information. A higher-timeframe bar may conceal the order in which price levels were reached.

For example, if both a stop and a target fall inside one daily candle, OHLC data alone may not establish which level was reached first. The execution model must therefore specify how such ambiguity is handled.

10. Historical Volume and Liquidity

Historical volume does not automatically tell you how much capital could have been deployed without market impact. Volume can be consolidated, venue-specific, revised, or represented differently across providers.

Liquidity assumptions should therefore be explicit rather than inferred from a single volume column.

11. Data Granularity and Research Questions

The required granularity follows from the strategy. A monthly trend study may tolerate daily observations. A stop-sensitive intraday strategy may require tick or quote information.

Using coarse data for a fine-grained question creates uncertainty that cannot be repaired by sophisticated statistics afterward.

12. Historical Data Revisions

Some datasets are revised after the original publication. Revisions can improve accuracy, but they can also change what the researcher believes was knowable at the time.

For point-in-time research, the distinction between today's corrected history and the information set available then is critical.

13. Data Vendor Cross-Validation

A useful research control is to compare selected periods, instruments, corporate events, and extreme moves across independent sources.

The objective is not to force sources to agree. It is to identify material differences and determine whether the strategy depends on them.

14. Freezing a Historical Dataset

Once a dataset has been validated, the exact version used for a study should be recorded. Otherwise a future vendor correction can change the backtest while leaving the published result apparently unchanged.

Useful metadata includes source, extraction date, dataset version, adjustment policy, timezone, session calendar, universe definition, and transformation rules.

15. Building a Point-in-Time Research Set

Define the instruments that were eligible at each historical date.

Use information that was available at the relevant decision time.

Document price and corporate-action treatment.

Apply the correct historical trading calendar and timezone.

Match data granularity to the strategy's decision and execution horizon.

Record missing periods and exceptional events.

Version and freeze the final research dataset.

16. Expert Case Study — Ten Years of Daily Data

A trader tests a stock-selection strategy on ten years of daily data and obtains a strong result. The dataset contains today's index constituents, adjusted prices, and a current symbol list.

The result cannot yet answer the intended question: what would the strategy have done using the information and universe available at each historical date? The researcher must reconstruct historical eligibility, point-in-time information, corporate events, and instrument identity, then rerun the study.

The lesson is fundamental: a long history is not automatically a realistic history.

17. Common Mistakes

Equating more years of data with better evidence.

Using today's universe to represent every historical period.

Ignoring delisted instruments.

Mixing raw and adjusted data without a defined purpose.

Treating daily OHLC as if it revealed intrabar execution order.

Ignoring historical revisions.

Failing to document the exact dataset version.

Assuming two vendors represent history identically.

18. Expert Exercise — Reconstruct the Trader's Information Set

Choose one historical strategy test. For a specific historical date, write down exactly what the trader would have known, which instruments were eligible, what prices were observable, what session was open, and what execution information was available.

Then compare that point-in-time reconstruction with the dataset your backtest currently uses. Identify every place where today's database contains information that would not have been available at the time.

19. Knowledge Check

Why is historical data a reconstruction rather than the market itself?

Why must a historical universe be defined point in time?

How can corporate actions alter historical price interpretation?

Why can a daily OHLC bar be insufficient for stop-versus-target ordering?

Why does data granularity have to match the research question?

Why are historical revisions important for point-in-time research?

What metadata should be recorded when freezing a dataset?

Why is a long historical period not sufficient evidence by itself?

Copying, reproducing, adapting, distributing, or publicly displaying any part or all of the Academy’s textual content without the author’s prior written consent is prohibited.