< All Topics

Running large AlgoBB/Fixed Input Parameter projects

Running large Algo Building Block (also called Fixed Input Parameter) projects can be take a long time — many days — even with extremely fast CPUs.  Part of the issue is simply the massive number of optimizations that need to be performed and part of the issue is TradeStation’s poor thread management that does not take advantage of greater thread support with modern CPUs.  MultiWalk, unfortunately, runs “under” TradeStation, so inherits these limitations. (Note: also see this related article on running multiple instances of regular MultiWalk projects to reduce run time.)

Another issue of long running projects is the potential for TradeStation becoming unstable, ultimately leading to TradeStation crashes.  Click here to read more on this issue.  This article both serves two purposes: (1) to demonstrate how these large projects can run faster using smaller projects and (2) demonstrate how to void TradeStration crashes.

For example, this Algo Building Block project was running with ONE stock/ETF symbol (KRE) and three bar intervals (Daily, 240min, 120min) for a total of 312 Algo BB sub-projects.  After about two hours of running, TradeStation crashed:

This was not due to any internal fault in MultiWalk.  The cause was some internal issue within TradeStation when optimizating a stock or ETF symbol (I tried several).  This project works fine on all futures symbols.

I was able to kill all TradeStation processes and resume the project, which then completed about 15 minutes later.

However, as a test, I decided to split the project up into two smaller projects to see if I could avoid the crash.

The results demonstrated that, yes, the crash could have been avoided with two smaller projects.  But most important was that the run was almost TWICE as fast compared to running one large project.  Run two projects simultaneously was faster than running one larger project alone.  I believe the primary reason for this is that:

  1. The project was not demanding on RAM, so worked well within TradeStation’s 32-bit RAM limitation and,
  2. I was able to work around TradeStation’s thread limitations and make better use of my CPU’s greater thread capacity.

So keep in mind that by running simultaneous projects that we know need to be careful of exceeding TradeStation’s 32-bit memory limitation.  More threads will consume more memory.

Splitting large projects up into smaller projects is easy.  Here’s an example of how I split this one large project into two smaller projects.

1.  Start with the large project in a TS workspace.  This AlgoBB project will create 312 sub-projects.  I want to reduce that by half by splitting it into two AlgoBB projects.

2. Press Ctrl-Shift-C to copy MultiWalk project to your Window’s paste buffer.

3. Press Ctrl-Shift-V to paste the MultiWalk project into TradeStation’s workspace.  You should now have two MultiWalk projects — each an exact duplicate of the other.

The large project was set to use 3 threads, which was the original number used for this project from Kevin.  So I kept it the same.

Both smaller projects are also set to 3 threads.  TradeStation appears to throttle the number of threads for each process, to a maximum of 8 with a “sweet spot” of around 4 threads.  It would appear that by splitting the project into two MultiWalk processes that we gain some benefit of additional thread space.  In other words, the projects complete more quickly by running two simultaneously rather than one large project alone.

I am going to make two changes to each project:

  1. I’ll change the range of the fixed input parameter iStratVersion, 1-2×1 for one project and 3-4×1 for the other project.
  2. Add the sub-folder text “Run1” and “Run2” to each project so they are distinct from each other.

4. For PROJECT 1:

5. For PROJECT 2:

6. Click Run Project on BOTH Run1 and Run2 projects so that they both run simultaneously.

The large project run took approximately 2 hours 15 minutes to run.  But both smaller projects running simultaneously took about 1 hour 13 minutes to run — 46% less time, or almost twice as fast!

 

Table of Contents