Walkforward ReOpt Parameters Mismatch Example
[Note: the re-optimization date issue described in this article can be avoided by using Calendar Period Types, but the parameter issue can still occur due to OHLC data corrections.]
I recently experienced the issue where a re-optimization of one of my Daily bar ES strategies using a 252/252 day walkforward had a very different outcome after re-optimization. When this occurs, I feel obligated, as a developer, to ensure that an error has not crept into MultiWalk’s walkforward engine. In all the years of working with MultiWalk, this has ever only happened once (the June 2023 beta release). Without fail it has always been due to other situations. But since I was testing the new June 2023 release, I needed to make sure the error was fully resolved.
If you have a strategy that is showing different results, this is an excellent article that clearly shows what is happening and the dramatic effect that small changes in the data can make to the equity curve and parameterization of the strategy.
As I dug deeper into the discrepancy, I found that TradeStation had a discrepancy between their LIVE data server and their DELAYED data server. I spoke with TradeStation support and they suggested that it should resolve by the next day, which it did. TradeStation had made data corrections, however they have multiple data servers and the corrections don’t always proprogate to all the servers at the same time. The issue I describe in this article will occur when TS makes any data corrections. TS will perform data corrections even if you never clear your cache.
Changes in data, for whatever reason, can create small to large divergences to the strategy’s results depending on the severity of the data changes.
To understand why the data can have such a large effect on the walkforward strategy, it is important to remember that MultiWalk uses trading days, not calendar days, to bracket both the walkforward in-periods and out-periods. Therefore the first and last date of each period is based on what is found in the DATA, not on a calendar. So if you define the trading period to be 252 trading days (as I do in the example below) and TS does not have all the data within the calendar period, the first and last date, and all the data inbetween, will not align with calendar dates. There are missing dates, so 252 trading days cannot align. When (if) TS corrects the data, the beginning and ending dates of the 252 trading days will change and, consequently, the price and trade dynamics in that rangeThe following is a data window dump and comparison of the data from the live server (left side) and delayed server (right side):
Here is the effect on the first walkforward period that had missing data (on the left) and complete data (on the right):
This had a cascading effect the got more extreme with each re-optimization.
Here is an analysis of the strategy walkforward — using the LIVE (incomplete/incorrect) data server on the left and DELAYED (complete/corrected) data server on the right:
Notice that the OOS periods date shifted due to the missing data. 252 trading should approximately align on calendar yearly boundaries, which they do on the right-hand side that has correct data. When TS corrected the data, the walkforward dates correctly began and ended in May of each year. However, with each missing day in the different periods on the left-hand side, the 252 trading day groups became increasingly mis-aligned to the point that by the 15th re-optimization date, the out-period was including August 2021-2022 data instead of May 2021-2022 data.
Notice the highlighted example on Iteration 6 above. The data change made the optimization parameters change resulting in a OOS different outcome of $7,408 vs -$1,173 net profit. That’s a dramatic difference for missing only 4 data points (daily bars) in Iteration 6!
The cascading effect continued for the rest of the walkforward, resulting in very different equity results and parameter dates. There were more missing data points (daily bars) in Iterations 7-13. The effect on the walkforward is dramatic.
This is why Kevin Davey has the practice of adding new re-optimization parameters to the original/previous optimization list of parameters and adjusting the time period to fit.