Complex Testing

From Quantwiki
Jump to: navigation, search

Often strategies will fare differently depending on when they enter the market. In order to analyze this phenomenon, QSTK has features which support testing over multiple periods. The report functionality of QSTK then allows a user to analyze this information.

Contents

Robust Strategy Back Testing

From the command line, the backtester may test strategies in a more rigorous fashion by doing a robust back test. By testing a given strategy over a several time periods, one may determine the statefulness of a strategy and other useful information. This may be accomplished by making the following call from the command line:

python quickSim.py 'strategy.py' '1-1-2004' '1-1-2005' 7 28 1000 'out.pkl'

The first two arguments specify a time period over which to test the strategy multiple times. The third and fourth paramaters are an offset and duration for each test respectively. The backtester uses these arguments to test the strategy multiple times over a time frame and output a list of funds time series. This list may then be analyzed using QSTK report functionality to determine some interesting metrics about the strategy.

strategy_backtest1

Using the module provides several more options and finer control over how you test a strategy. This function tests a strategy a number of times. It takes in a number of tests along with the offset of the tests. Each test is as long as the period between the start and end date.

from quicksim import quickSim as qs
import datetime as dt
startday= dt.datetime(2004,1,1)
endday = dt.datetime(2005,1,1)  
fundsmatrix=strat_backtest1('strategy.py',startday,endday,10,7,1000)		

strategy_backtest2

Also provided through the module, this function tests a strategy throughout a period, with each test taking a set duration. Tests begin at each offset, and end after the duration is up until the given period is at an end.

from quicksim import quickSim as qs
import datetime as dt
startday= dt.datetime(2004,1,1)
endday = dt.datetime(2005,1,1)  
fundsmatrix=strat_backtest1('strategy.py',startday,endday,7,28,1000)	

Analysis

Once the back tester has tested a user strategy over several iterations, QSTK can plot the tests and display the average total return for the series of tests. It also will plot the confidence bounds, given by the standard deviations on the total return plot. This may be done by a command line call to the report.py script in Bin.

python report.py -r 'funds.pkl'

This will create a report.html file containing two graphs, showing the results of the tests. Below is some sample output from a stateful bollinger strategy.
Stateful funds.png Stateful analysis.png
The analysis graph is particularly useful, since the spread of the confidence bounds provides an indication of the degree of statefulness of a strategy. A stateless strategy and its analysis is shown below.
Stateless funds.png Stateless analysis.png

Personal tools