Feature Engineering

Introduction

Feature engineering is the process of transforming raw trading data into variables that a model or rule can use. In a trading context, a feature may describe price movement, volatility, volume, trend, relative position, market context, or a relationship between several observations.The advanced challenge is not creating more features. It is creating features that represent a meaningful hypothesis without leaking future information or giving the model unnecessary degrees of freedom.A feature is therefore not automatically an edge. It is an input to a research process. Its usefulness must be evaluated through a controlled methodology, realistic data, and Out-of-Sample evidence.

1. What a Feature Actually Is

A feature is a measurable input supplied to a model or decision rule. Examples include a one-day return, distance from an EMA, ATR-normalized movement, relative volume, rolling volatility, or the position of price within a recent range.A feature can be raw, transformed, normalized, ranked, aggregated, or combined with another variable. What matters is that its calculation is precisely defined and reproducible.In trading research, the feature's timestamp is just as important as its mathematical formula. A perfect formula calculated with information unavailable at the decision time is still an invalid feature.

2. Start With the Trading Hypothesis

Feature engineering should begin with a trading question, not with a desire to find predictive variables.Suppose the hypothesis is that unusually strong short-term momentum persists for a limited period. Potential features could describe recent return, relative strength, volatility-adjusted return, or distance from a reference trend.The hypothesis determines what should be measured. Starting with hundreds of arbitrary indicators and asking which one predicts the future creates a much larger Data Snooping problem.

3. Raw Data Versus Derived Features

Raw market data include observations such as open, high, low, close, volume, timestamps, and instrument identifiers. Derived features transform these observations into information that may be easier for a model to use.For example, return can be derived from price, range from high and low, and normalized movement from return divided by a volatility estimate.Every transformation adds a definition that must be documented. If two researchers calculate the same concept differently, they are no longer testing exactly the same research object.

4. Returns, Changes, and Differences

Price levels often contain scale information that is not directly comparable across instruments. Returns and changes can provide more comparable representations.A simple return can be written as R_t = P_t/P_{t-1} – 1. Log returns can also be useful when their mathematical properties suit the research problem.The important question is not which representation is universally best. It is whether the chosen transformation matches the hypothesis and remains consistent between development, validation, and live use.

5. Normalization and Scaling

Features can have very different numerical scales. One variable may range between -0.05 and 0.05 while another may range from 0 to several million.Scaling can make model estimation more stable, especially for algorithms sensitive to feature magnitude. Common approaches include standardization and range-based transformations.However, normalization itself can leak information if its parameters are calculated using the full dataset. In time-series research, scaling parameters should respect the same information boundary as the strategy.

6. Rolling Features and Time Windows

Rolling features summarize recent history. Examples include rolling mean, volatility, momentum, volume statistics, and distance from a moving average.The window length is itself a research decision. A 10-bar measurement and a 100-bar measurement are not interchangeable hypotheses.Rolling calculations must also be aligned correctly. A feature intended for decision time t should not accidentally include observations from after t. This is one of the most common ways a seemingly innocent feature becomes a source of Look-Ahead Bias.

7. Volatility-Adjusted Features

The same raw price movement can mean very different things in different volatility environments. Dividing a movement by a volatility measure can make the feature more comparable across regimes.For example, a 2% move may be extreme in one environment and ordinary in another. An ATR- or return-volatility-adjusted measure attempts to capture this distinction.Normalization by volatility does not automatically create predictive power. It simply changes the representation so that the hypothesis can be tested in a more comparable way.

8. Relative and Cross-Sectional Features

Many trading questions are comparative rather than absolute. A stock's return may be meaningful relative to an index, sector, or peer group.Cross-sectional features can include relative return rank, distance from a benchmark, or relative volume. These features require careful universe definition because the set of available instruments can itself introduce Survivorship Bias.The ranking procedure should also be point-in-time correct. The algorithm must know which instruments belonged to the eligible universe at the decision time.

9. Feature Interactions

A feature may become informative only in combination with another condition. For example, momentum may behave differently during high and low volatility, or a breakout feature may behave differently when liquidity is unusually low.Interactions can therefore represent conditional hypotheses. But every additional interaction increases model flexibility.The research question should be explicit: why should the interaction exist, and what evidence would distinguish it from a pattern found by chance?

10. Feature Redundancy and Correlation

Different features can encode nearly the same information. Price above EMA, moving-average distance, and a trend-strength measure may overlap substantially.Redundant features can make a model look richer without adding independent information. They can also increase sensitivity to noise and complicate interpretation.Feature selection should therefore consider not only individual predictive performance but also whether the feature adds information beyond variables already present.

11. Feature Selection Is a Research Decision

Selecting features is not a neutral preprocessing step. It is part of the strategy research process.If a researcher tests 500 candidate variables and keeps the 12 that produced the best historical results, the final feature set contains information from the research sample even if the model itself was never directly optimized on those 500 variables.This is why Data Snooping and Overfitting can occur before a machine-learning model is trained. The researcher's feature-selection process can already encode the historical sample into the system.

12. Missing Data and Availability

A feature can only be used when the underlying information was actually available. Missing observations must not be silently converted into a value that changes the meaning of the variable.For example, a missing volume observation, unavailable fundamental release, or delayed market-data field may have a different meaning from a true zero.Define missing-value behavior explicitly. In some cases the correct response is to exclude the observation; in others, a dedicated missing indicator or state may be appropriate.

13. Feature Leakage

Feature leakage occurs when information that would not have been available at the decision time enters the feature.Leakage can happen through future bars, revised fundamentals, full-sample normalization, target-derived variables, universe construction, or preprocessing performed before a chronological split.Leakage is especially dangerous because the model may appear highly predictive. The correct question is always: could this exact feature value have been known at the moment the trade decision was made?

14. Features and Statistical Significance

A feature that appears associated with future returns is not automatically statistically convincing. The observed relationship depends on Sample Size, variability, multiple research decisions, and the statistical procedure used.Statistical Significance should therefore be interpreted in the context of the research process. A low p-value discovered after thousands of feature searches is not equivalent to the same result from a pre-specified hypothesis.Practical significance matters as well. A statistically detectable effect may be too small to survive costs, slippage, or execution constraints.

15. Features, Expectancy, and the R-Multiple Distribution

The ultimate trading question is often not whether a feature predicts a continuous target with a high correlation. It is whether using the feature improves the distribution of trade outcomes under a defined trading process.A feature may increase win rate but reduce average win size. Another may reduce win rate while increasing the average R-Multiple of winners. Expectancy captures the combined effect.Evaluate feature changes at the level of the actual trading decision, including exits, costs, and position sizing where relevant.

16. Case Study — The 37th Feature

A researcher starts with a simple momentum model. Twenty features are added based on reasonable market concepts. Development performance improves. The researcher then tests dozens of alternative lookback lengths and keeps the configuration with the highest historical expectancy.Out-of-Sample performance falls sharply. The feature itself was not necessarily useless. The problem was that the research process used many degrees of freedom to select a configuration from the same historical evidence.A better approach would have been to define the feature family, limit the search space, record the research decisions, and evaluate the final specification on genuinely unseen data.

17. Case Study — A Useful Volatility Feature

A trader believes that momentum signals behave differently when volatility is unusually high. Instead of adding several unrelated filters, the trader defines one volatility-normalized momentum feature before testing.The feature improves development expectancy modestly and also improves Out-of-Sample results, while the R-Multiple distribution remains plausible and costs do not erase the effect.The evidence is still not proof of permanence. It is stronger evidence because the feature represents a defined hypothesis and survives a separate evaluation boundary.

18. Build Features for Live Use

A feature is only useful operationally if it can be calculated in the same way when the strategy is live. The live pipeline must reproduce the historical definition, timing, missing-data handling, and scaling rules.Avoid research features that require information unavailable or impractical in production. A theoretically attractive feature that cannot be computed reliably at decision time is not an executable trading input.The final feature specification should therefore include formula, inputs, timestamp, update frequency, missing-data behavior, and expected range or validity checks.

19. A Professional Feature Engineering Workflow

A disciplined workflow can be organized as follows:1. Define the trading hypothesis.2. Identify the economic or behavioral reason for the feature.3. Define the exact data inputs and information boundary.4. Create the simplest feature that tests the idea.5. Document transformations and parameters.6. Check missing data, timestamps, and availability.7. Evaluate redundancy and feature interactions deliberately.8. Test development performance without repeatedly redesigning around results.9. Protect Out-of-Sample data.10. Evaluate statistical and practical significance.11. Inspect Expectancy, R-Multiples, drawdown, and stability.12. Reproduce the feature in the live pipeline before deployment.The objective is not to find the largest possible feature set. It is to build a small set of interpretable inputs that express well-defined hypotheses and survive controlled validation.

20. Expert Exercise — Build a Feature Dossier

Choose one existing trading idea and create a dossier for three candidate features.For each feature record: hypothesis, exact formula, data inputs, timestamp, lookback, normalization, missing-data rule, expected relationship, possible leakage path, and reason for inclusion.Then test each feature separately before combining them. Record every research decision, including discarded variants. Finally, evaluate the combined specification on an Out-of-Sample dataset that was not used to select the features.The key question is not “Which feature produced the highest backtest?” It is “Which feature provided evidence that remained credible after the research freedom was controlled?”

21. Knowledge Check

1. What is a feature in quantitative trading?2. Why should feature engineering begin with a hypothesis?3. Why can normalization create leakage?4. What makes rolling features vulnerable to Look-Ahead Bias?5. Why can relative features introduce Survivorship Bias?6. Why is feature selection itself a research decision?7. What is feature leakage?8. Why is correlation with future returns not sufficient evidence of trading usefulness?9. Why should feature changes be evaluated through Expectancy and R-Multiple outcomes?10. What must be reproduced when a research feature is moved into live trading?A strong answer recognizes that feature engineering is not a search for magical indicators. It is the disciplined translation of a trading hypothesis into measurable, time-correct, reproducible inputs whose usefulness must survive controlled research.

Az Akadémia szöveges tartalmának részben, vagy egészben történő másolása, többszörözése, átdolgozása, terjesztése, valamint nyilvános bemutatása a szerző előzetes, írásos hozzájárulása nélkül tilos.