Complex Testing

From Quantwiki
Jump to: navigation, search

Often strategies will fare differently depending on when they enter the market. In order to analyze this phenomenon, QSTK has features which support testing over multiple periods. The report functionality of QSTK then allows a user to analyze this information.


Robust Strategy Back Testing

From the command line, the backtester may test strategies in a more rigorous fashion by doing a robust back test. By testing a given strategy over a several time periods, one may determine the statefulness of a strategy and other useful information. This may be accomplished by making the following call from the command line:

python '' '1-1-2004' '1-1-2005' 7 28 1000 'out.pkl'

The first two arguments specify a time period over which to test the strategy multiple times. The third and fourth paramaters are an offset and duration for each test respectively. The backtester uses these arguments to test the strategy multiple times over a time frame and output a list of funds time series. This list may then be analyzed using QSTK report functionality to determine some interesting metrics about the strategy.


Using the module provides several more options and finer control over how you test a strategy. This function tests a strategy a number of times. It takes in a number of tests along with the offset of the tests. Each test is as long as the period between the start and end date.

from quicksim import quickSim as qs
import datetime as dt
startday= dt.datetime(2004,1,1)
endday = dt.datetime(2005,1,1)  


Also provided through the module, this function tests a strategy throughout a period, with each test taking a set duration. Tests begin at each offset, and end after the duration is up until the given period is at an end.

from quicksim import quickSim as qs
import datetime as dt
startday= dt.datetime(2004,1,1)
endday = dt.datetime(2005,1,1)  


Once the back tester has tested a user strategy over several iterations, QSTK can plot the tests and display the average total return for the series of tests. It also will plot the confidence bounds, given by the standard deviations on the total return plot. This may be done by a command line call to the script in Bin.

python -r 'funds.pkl'

This will create a report.html file containing two graphs, showing the results of the tests. Below is some sample output from a stateful bollinger strategy.
Stateful funds.png Stateful analysis.png
The analysis graph is particularly useful, since the spread of the confidence bounds provides an indication of the degree of statefulness of a strategy. A stateless strategy and its analysis is shown below.
Stateless funds.png Stateless analysis.png

Personal tools