How to Validate Technical Indicator Periods and Thresholds
Understand overfitting when indicator periods and trading thresholds are tuned to past performance. Review chronological validation, stability across nearby settings, and cost and execution assumptions.
Indicator periods and thresholds are not validated simply by choosing the numbers with the best-looking historical performance. Separate the data used to set the rule from the data used to check it afterward, and evaluate the new window without retuning the settings. Check whether nearby settings produce similar results and account for actual execution costs.
Complete the rules before selecting numbers
A rule is effectively incomplete if you choose only a moving average period and leave the exit method or signal-confirmation time undefined. First write out the stock universe, data-adjustment method, buy and sell conditions, handling of simultaneous signals, position sizing, and costs. Also define how to handle trading halts and missing data. Differences in implementation can change performance even with identical parameters, so keep the raw data alongside the calculation settings. Check whether reconstructing historical candidates using only stocks that survive today misses the risks of stocks that were excluded or disappeared at the time.
Split data chronologically and record the number of trials
Choosing settings in an earlier window and checking fixed rules in a later one reduces the use of future data in past selections. If you change the numbers after seeing the later results, that window has also become part of the selection data. A new, unseen window is then needed. If you tested hundreds of combinations of periods and thresholds, record the search range and number of comparisons rather than only the final choice. Assessing how likely the best result was to look good by chance requires knowing how many alternatives were examined.
One bar and small cost differences can change the result
If a signal became known only after a bar closed, assuming execution at that bar’s most favorable price can create impossible performance. Incorporate feasible order timing, slippage, fees, and other costs, and examine how results change as costs rise. If a single optimal period performs exceptionally well but nearby numbers perform sharply worse, suspect excessive fitting to the data. Examine profit-and-loss paths and the largest loss stretches in rising, falling, and sideways markets. Good historical validation neither guarantees future returns nor eliminates changes in market conditions.
Worked example
Hypothetical example: Suppose periods of 18, 20, and 22 were compared and 20 was selected in the earlier window. Changing the final setting to 22 because results in the later window were 3%, −2%, and 4%, respectively, means the later window is no longer a clean validation sample. Also, in a test consisting of one purchase and sale of a fixed quantity, buying at 100 won and selling at 101 won yields a pre-cost profit of 1 won. Assuming total round-trip costs of 0.4 won per share reduces net profit to 0.6 won. These are hypothetical calculations illustrating validation procedures and cost effects, rather than actual performance.
What to check
- Document signal, position-sizing, exit, and cost rules first.
- Separate selection and validation windows in chronological order.
- Keep the combinations tried and failed results as well.
- Check sensitivity to nearby settings and execution and cost assumptions.
Sources and primary materials
This is educational material about research methods. For an actual company decision, verify the filings and primary sources for the relevant period.