< All Topics

MultiWalk’s fast walkforward emulation engine

MultiWalk’s Fast Walkforward Emulation Engine

How does MultiWalk calculate the final walkforward strategy?

Why are these equity results sometimes different from TradeStation when the strategy is put on a chart?

The short explanation…

When a walkforward strategy is created, it will change the values of the optimized parameters using a defined in-sample and out-of-sample time schedule. Understanding what occurs during these in/out transition periods is important. It effects the behavior of the strategy and the way that it trades.

Each transition period of a walkforward strategy — what we call “re-optimization periods” — can have a profound effect on two things:

  1. Trade position (long, short or flat)
  2. Behavior of functions used in the strategy (such as Average(), RSI(), etc.)

In a perfect world where computer computational resources were a infinite, full optimizations would be performed every walkforward transition period and positions automatically adjusted to accomodate the new walkforward period.

In our non-perfect world, we need to employ estimation methods that are representations of the actual behavior of the strategy.

MultiWalk uses a built-in fast approach to calculating the walkforward strategy.  This avoids needing to re-optimize every walkforward in-period, which is compounded because of MultiWalk’s unique ability to test across many instruments and time bar intervals.  However, it also means that the equity results will not exactly match TradeStation.  The differences can be small or huge.

For this reasons, MultiWalk has a final walkforward calculation option that can be enabled:

When you enable this option, MultiWalk will run the walkforward strategy through TradeStation so that equity results in MultiWalk and TradeStation will be identical. If you are seeing differences between MW and TS results and have ruled out all the other possible reasons that MW and TS equity curves do not match, then enable this option.

If the results are poor, then you should probably consider other non-walkforward techniques to trade your strategy, or simply abondon this strategy idea and move on to the next one. In other words, if the results are poor in TradeStation, then the strategy is not tradable and should not be traded, even if results look great in MultiWalk.

Using this option will dramatically increase the amount of time of MultiWalk’s walkforward process. Therefore, it is probably best to use this option only when you have a small set of walkforwards and want to see how they would actually trade in TradeStation. I, personally, do not enable it for long project runs that will be testing across many symbols and many bar time intervals.

The longer explanation…

A much earlier version of MultiWalk that I never released publically used TradeStation’s optimization engine to compute the walkforward results. It took many days to run just a few walkforward scenarios across several symbols and time frames. Accessing the optimization process using TradeStation’s application programming interface was just too slow to be useful. The next version of MultiWalk used a hybrid approach – optimizing the strategy in TradeStation once for the full analysis period and then running the walkforward in an external program (StratOpt in this case). I released this version to some fellow traders. While much faster, it was very cumbersome to use and, like the first version, this limited its usefulness. StratOpt was simply not designed to be called by another program to run walkforwards and controlling it externally presenting too many issues. Plus, there were other issues with StratOpt, especially the way it handled fitness functions, that led me to eventually abandon its use.

But it gave me an idea and I decided to write my own walkforward processor using an approach that would avoid TradeStation’s slow processing speed without sacrificing too much precision.

The speed gains were astonishing.

Here is a simple example:

Let’s say I have a strategy with 1 parameter to optimize, and it looks like this:

    Input: xbar(5);
    If close > close[xbar] then buy next bar at market;
    If close < close[xbar] then sellshort next bar at market;

If I defined 28 values of xbar for the walkforward project (xbar = 5 to 32 step 1), MultiWalk would create 28 different equity curves (one for each xbar iteration) on the entire analysis time period using TradeStation to actually run and trade the strategy. And let’s say I want to do this for 11.5 years using a 126 trading day (6 month) walkforward out-period. That would mean doing 26 out-of-sample/reoptimization periods for a total of 728 evaluations (28 * 26).

MultiWalk looks at the 28 equity curves for each one of the 26 out-periods, picks the best one each time, and then puts all those “best” out-periods together into one final equity curve. Think of it like “cut-and-pasting” the correct out-period time segment from those original 28 optimized equity curves into the final walkforward equity curve. This is all done without the need of actually running the strategy again for all those 728 out-period evaluations. To have TradeStation run the strategy 728 times (rather than just 28 times) increases the computational time astronomically.

The reuse of the 28 optimization equity curves was my solution to the time problem. Other walkforward software packages (such as StratOpt) use similar solutions. It works well for the majority of cases – meaning there is a very good match between the walkforward equity graph in MultiWalk and Tradestation. But, there will be cases when it does not match as well, which is why we use TradeStation as a final verification of the walkforward results.

So let’s put this example in an actual MultiWalk project that performs a 504/126 period walkforward on crude oil using 360 minute bars.

MultiWalk uses TradeStation to make one single pass to optimize the strategy (28 times) over the entire analysis time period. It then reuses the data from those optimized iterations over and over again during the walkforward process (728 evaluations).

This “single pass” is what you see when the MultiWalk progress task log shows “FullOptimization”:

Notice the time difference between the optimization task and the walkforward task. Full optimization of the 28 iterations took 32 seconds and the 728 walkforward evaluations took less than 1 second!

If MultiWalk were to calculate the strategy in the same way that TradeStation does (the long approach of the early version of MultiWalk), it would need to call TradeStation for all 728 evaluations rather than just 28.

Given that one year takes TradeStation approximately 2.1 seconds to compute in this example, the total time commitment to use TradeStation to compute the walkforward would look like this:

In order to maintain accuracy and consistency with the walkforward parameters, TradeStation would need to run the strategy starting from the same begin date (Jan 2007) and re-optimize every 6 months (126 trading days).

As you can see, the overall time has now increased from 27 seconds to over 8 minutes! That’s a massive time increase – over 1400% !!

So rather than having TradeStation do 728 evaluations, MultiWalk sacrifices some precision by having TradeStation do 28 optimizations upfront and then performing its own exponentially faster walkforward evaluation method.

Conclusion

The trade-off between speed and precision is always a delicate balance. MultiWalk’s time saving measure comes at a cost and can lead to occasional and mostly minimal differences between Tradestation and MultiWalk.  However, without MultiWalk’s walkforward emulator, the massive time requirement to perform a 100% accurate walkforward would prevent us from attaining out trading goals.

Table of Contents