Strategy Research Framework

Introduction

A strategy research framework is the structure that turns trading ideas into controlled research. Without a framework, strategy development easily becomes a sequence of charts, indicators, parameter changes and attractive backtests. The trader may work hard while gradually losing track of which question was actually being tested.

A professional framework does not make research rigid. It makes research traceable. Every major decision should have a reason, every test should have a defined purpose, and every result should be interpretable in the context in which it was produced.

The objective is not to find the most complicated strategy. It is to create a repeatable process for asking good questions, testing them honestly, validating useful evidence and rejecting weak hypotheses.

1. Research Starts With a Question

Good research begins with a question that can be tested.

Weak question:“Can I make this chart pattern profitable?”

Better question:“Does this predefined pattern produce a different distribution of forward R-multiples than the chosen baseline under these market conditions?”

The second formulation defines what is being compared and what evidence could support or weaken the hypothesis.

2. Observation, Hypothesis and Mechanism

Separate three things:

Observation — something appears repeatedly in the data.Hypothesis — a specific claim about what happens.Mechanism — a reason why the effect might exist.

For example, an observation might be that breakouts following unusually narrow ranges appear to travel farther. The hypothesis could specify the exact range definition, breakout rule and measurement horizon. The mechanism might involve volatility expansion.

The mechanism is useful because it helps determine where the edge should and should not work.

3. Define the Research Object

Before testing, define exactly what the research object is.

Specify:• instrument universe;• timeframe;• session definition;• entry;• exit;• holding period;• position sizing;• transaction costs;• slippage;• liquidity constraints;• data source;• evaluation metrics.

Changing several of these at once makes it difficult to know what caused a result to change.

4. Establish a Baseline

A strategy result has little meaning without a reference point.

Possible baselines include:• buy-and-hold;• random entry with the same exit;• the strategy without the proposed filter;• a simpler version of the same rule;• a benchmark appropriate to the research question.

The baseline is not required to be a perfect model of randomness. Its purpose is to answer: “What changes when I add this proposed edge?”

5. Build the Simplest Test First

Start with the smallest rule set that can answer the research question.

If the hypothesis concerns a breakout, test the breakout before adding six confirmation indicators.

Every extra condition introduces another research decision. A complicated strategy can produce an impressive historical fit simply because it has many opportunities to adapt to the sample.

Simplicity at the beginning improves interpretability.

6. Separate Discovery From Evaluation

Exploration and evaluation should have different jobs.

Discovery is where you:• search;• compare;• investigate;• reject;• formulate new hypotheses.

Evaluation is where you:• freeze the specification;• use controlled data;• measure predefined outcomes;• document the result.

Mixing the two turns every evaluation into another opportunity for optimization and increases Data Snooping risk.

7. Maintain an Information Boundary

The research framework should make it obvious which information was available at each stage.

A simple structure is:

Development data → frozen specification → validation data → decision.

If validation results lead to rule changes, the affected validation set has become part of the development process. A fresh validation period may then be necessary.

The boundary is not bureaucracy. It is what gives the test evidentiary value.

8. Use a Research Log

A research log records what was believed before the result was known.

Record:• hypothesis;• date;• dataset version;• rule version;• parameters;• reason for the test;• primary metric;• secondary metrics;• result;• interpretation;• next action.

This prevents memory from rewriting the research history after a successful backtest.

9. Define Primary and Secondary Metrics

Do not allow the preferred metric to change after every test.

A primary metric should represent the central research question. Secondary metrics describe supporting properties.

For a trading strategy, a useful set may include:• expectancy;• Profit Factor;• average R;• maximum drawdown;• drawdown duration;• trade count;• distribution of R-multiples.

The point is not to collect every metric available. It is to measure the dimensions that matter for the hypothesis.

10. Think in R-Multiples

R-multiples create a common language for comparing trades across different position sizes.

If one trade risks $100 and another risks $500, +2R means +$200 in the first case and +$1,000 in the second. The underlying trading outcome is expressed relative to the initial risk.

This helps research:• compare different instruments;• compare different position sizes;• analyze distributions;• evaluate expectancy;• study drawdown in risk units.

11. Control Multiple Testing

Strategy research often contains many hidden tests.

Examples:• several indicators;• multiple lookback periods;• different exits;• many instruments;• several timeframes;• different filters;• alternative cost assumptions.

Even if each individual test seems reasonable, a large number of attempts increases the chance of finding an apparently successful result by chance.

A research framework therefore records the search process, not only the final winner.

12. Avoid the Optimization Trap

Optimization can be useful when it is a predefined part of the research process. It becomes dangerous when the objective is simply to improve the historical score until it looks impressive.

Warning signs include:• very precise parameter values;• large performance changes from tiny parameter changes;• repeated optimization on the same sample;• adding rules after observing failures;• a large gap between development and OOS results.

Optimization should answer a defined question, not replace the research question.

13. Validation and Walk-Forward

Once a candidate strategy is specified, validation asks whether it survives unseen information.

Walk-forward analysis extends this idea across multiple chronological cycles. It can show whether a research process continues to work as market conditions evolve.

Neither a single OOS test nor walk-forward analysis guarantees future profitability. They provide evidence about the stability of the research result under defined conditions.

14. Statistical Evidence and Practical Evidence

Statistical analysis helps quantify uncertainty. Trading decisions also require practical analysis.

A strategy may show a statistically detectable effect but have:• tiny expectancy;• high costs;• low capacity;• unacceptable drawdown.

Another strategy may show an attractive effect in a small sample but have insufficient evidence.

A research framework should keep these questions separate rather than compressing them into one score.

15. Research Across Regimes

A framework should specify which market conditions are relevant to the hypothesis.

Possible dimensions include:• volatility;• trend;• liquidity;• market direction;• sector behavior;• macro/event environments.

The goal is not to force uniform performance everywhere. It is to understand where the proposed mechanism appears to operate and where it does not.

16. Version the Strategy

Treat each meaningful rule change as a new strategy version.

For example:Strategy v1.0 — original breakout.Strategy v1.1 — stop definition changed.Strategy v1.2 — volatility filter added.

Do not overwrite the old result. Preserve it.

Versioning makes it possible to answer an important question: did the strategy improve because the underlying idea improved, or because the research process gradually adapted to the same historical sample?

17. Case Study — The Indicator Monster

A trader starts with a simple breakout. The first test is mediocre. A moving average is added, then RSI, then volume, then an ADX filter, then a time-of-day restriction.

After each change the historical result improves. The final version has excellent development performance.

Because the research log contains every step, the trader can see what actually happened: a long sequence of adaptive decisions was made on the same data.

The framework does not automatically reject the strategy. It changes the research conclusion: the final result now requires particularly disciplined fresh validation because the development process contained substantial search freedom.

18. Case Study — A Clean Research Chain

Another trader observes a recurring setup and writes a precise hypothesis. A simple baseline is tested first. The data definition is frozen. A development sample is used for refinement, then the rules are frozen.

A separate validation sample is evaluated using predefined metrics. The result is weaker than development but remains positive, with similar R-distribution and acceptable drawdown.

Because the research chain is traceable, the trader can distinguish discovery, development and validation instead of treating the entire history as one large backtest.

19. Professional Research Framework

A practical workflow is:

1. Define the question.2. Record the observation.3. State the mechanism and hypothesis.4. Define the research object.5. Choose the baseline.6. Define metrics.7. Build the simplest test.8. Record the research version.9. Explore only within the development process.10. Freeze the candidate.11. Validate on unseen data.12. Analyze distribution and drawdown.13. Check relevant regimes.14. Document the conclusion.15. Decide whether a new hypothesis is justified.

The framework should be repeatable enough that another researcher could understand what was done without relying on memory.

20. Common Failure Modes

• Starting with an indicator instead of a research question.• Changing several variables simultaneously.• Ignoring the baseline.• Testing many variants without recording them.• Using the validation set for repeated development.• Reporting only the best period.• Optimizing until the equity curve looks smooth.• Ignoring costs.• Ignoring R-multiple distribution.• Treating statistical significance as profitability.• Treating one positive backtest as proof.• Deleting failed versions from the research history.

21. Expert Exercise — Build Your Own Framework

Choose one trading idea you currently consider interesting.

Create a research sheet containing:• research question;• observation;• mechanism;• precise hypothesis;• universe;• timeframe;• entry/exit;• costs;• baseline;• primary metric;• secondary metrics;• development period;• validation period;• failure criteria;• version number.

Then write down what you are explicitly NOT allowed to change after seeing the validation result.

This final line is important: it defines the boundary between research and post-result adaptation.

22. Knowledge Check

1. Why should research begin with a question rather than an indicator?2. What is the difference between observation, hypothesis and mechanism?3. Why is a baseline useful?4. Why should discovery and evaluation have different roles?5. What information should a research log preserve?6. Why is R useful as a research language?7. How does multiple testing create Data Snooping risk?8. When does optimization become a research problem?9. Why should strategy versions be preserved?10. What makes a research framework reproducible?

Key idea: a strong strategy research process is not the process that finds the most impressive backtest. It is the process that makes it difficult to fool yourself.

Copying, reproducing, adapting, distributing, or publicly displaying any part or all of the Academy’s textual content without the author’s prior written consent is prohibited.