Incubation period in MultiWalk

One of the fundamental aspects of Kevin Davey’s process is the use of an incubation as a final robustness test against over-optimization. It is a forward-testing phase where a newly developed trading system is monitored on live market data without trading real money. The goal is to see how the system performs in real-time conditions and ensure that the promising results from backtesting and walk-forward testing hold up in live (but simulated) markets.  For Kevin Davey, the incubation period is typically 6-9 months.

MultiWalk has a built-in feature to simulate the incubation period without having to wait 6-9 months of real-time testing.  In this regard, incubation in MultiWalk is similar to the traditional practice of segmenting data into two data sets for testing:  in-sample and out-of-sample.  The only difference between the two practices (Kevin’s incubation and IS/OOS) is that incubation occurs in real-time, whereas IS/OOS tests are performed on historical data.

In MultiWalk, the incubation (i.e., OOS) period can be defined.  However, in order to adhere to the real-time aspect of incubation in MultiWalk, the OOS period must be hidden from all reports until you are ready run the trading strategy on the unseen OOS data, just as though you were running the incubation in real-time.  This takes strong diligence not to “cheat” and unhide the data prematurely.  Once you unhide the incubation period, you must treat the process as though it ran in real-time.  There is no going back to make changes to the strategy or walkforward parameters.  It either passed or failed incubation.

Using Incubation Period

There are two options associated with the incubation period in MultiWalk:

1. Exclude and hide from walkforward results
2. Disable/Enable

Please note that functionally there is no difference between the “Exclude” option and the “Disable” option.  Both will prevent MultiWalk from calculating incubation results or from displaying them in reports.

The primary reason to use the “Exclude” option is to make it clear that you are reserving data from your analysis for Kevin Davey’s incubation period instead of waiting 6-9 months in real-time for incubation to occur.

Once you are ready to run the incubation period on your strategy, simply uncheck the “Exclude and hide” option and re-run the project.  But recognize that you can only do this once!  If the results are poor, then you cannot go back and make changes to the strategy until they are “good”.  Doing so violates the incubation period and “burns” that out-of-sample period of time.  At that point you would either scrap the strategy idea and move on, or need to run the strategy in a real-time incubation period to verify the new, changed, strategy.

So be careful — it takes discipline to run the incubation period once and only once!

WALKFORWARD and INCUBATION versus IS and OOS

In order to remain consistent with Kevin’s terminology, MultiWalk uses the terms “WALKFORWARD” and “INCUBATION” on the walkforward definition screen.  However, to remain concise, the reports will use the more traditional terms “IS” and “OOS”.

In other words, the reports in MultiWalk refer to WALKFORWARD and INCUBATION time periods as IS and OOS:

• WALKFORWARD period = IS (in-sample)
• INCUBATION period = OOS (out-of-sample)

Please see the Walkforward Reports And Filters  article for more information on how IS and IS+OOS periods are managed in MultiWalk’s report system.