Event-driven Python backtesting engine - Intro

This post is part of a series that will cover how to build an event-driven backtesting engine. I’ve used Python, but you can adapt the concepts to any language.

This project is inspired from this series on Quantstart, with my own additions and refinements. As we’re going over each of the building blocks of the engine, I’ll also show you how to write tests to cover the features.

By the end of this series you should be able to:

reproduce something similar in your language of choice, following an event-driven architecture
adapt the backtester to a live trading environment or a simulated one with minimal changes

Before diving into the code, here’s a schematic of how events flow in the engine:

flowchart TD

A(Market feed from broker or data provider) --> B(Data management layer)
B--> D(Event Manager)
D-->E(Non-market event)
D-->F(Market event)
F-->G(Process trading signal)
G-->|Trade|H[No]
G-->|Trade|I[Yes]
I-->J(Open new position)
J-->K(Order manager)
K-->L(Calculate optimal position size and risk level)
L-->M(Send order to broker)
M-->N(Book new order into portfolio)
N-->O(Portfolio: monitor positions, calculate risk, calculate PnL)
O-->P(Risk above threshold)
O-->Q(Risk below threshold)
P-->R(Close position)
Q-->S(Keep position open)-->O

In terms of code organisation, I’ve arranged each significant part from the diagram above into its own file, i.e data.py for the data layer, events.py for the events, etc.

For the impatient, the final code looks like this, backtester.py:

Details


# trading_engine/backtester.py

import datetime
import queue

from data import HistoricCsvDataHandler
from strategy import BuyAndHoldStrategy, StatArbStrategy
from portfolio import BuyAndHoldPortfolio
from broker import SimulatedExecutionHandler
from utils import *


if __name__ == "__main__":
    parsed_args = cli_parser()
    asset_class = asset_class_selector(parsed_args)
    data_dir = data_dir_setup(asset_class)

    # main, and only, event queue (FIFO)
    events = queue.Queue()

    # input symbols
    symbols = symbols_filterer(parsed_args.tickers[0], data_dir)

    # engine components
    bars = HistoricCsvDataHandler(events, data_dir, symbols)
    buy_hold_strategy = BuyAndHoldStrategy(bars, events)
    buy_hold_portfolio = BuyAndHoldPortfolio(bars, events, datetime.datetime(1960, 1, 1).strftime("%Y-%m-%d"))
    broker = SimulatedExecutionHandler(events)

    # main execution loop
    while True:
        if bars.continue_backtest:
            bars.update_bars()
        else:
            print("End of backtesting...")
            buy_hold_portfolio.create_equity_curve_dataframe()
            print("\n **** Portfolio statistics **** \n")
            for stat in buy_hold_portfolio.output_summary_stats():
                print(stat)
            break

        while True:
            try:
                event = events.get(False)
            except queue.Empty:
                break
            else:
                if event is not None:
                    if event.type == "MARKET":
                        buy_hold_strategy.calculate_signals(event)
                        buy_hold_portfolio.update_timeindex(event)

                    elif event.type == "SIGNAL":
                        buy_hold_portfolio.update_signal(event)

                    elif event.type == "ORDER":
                        broker.execute_order(event)

                    elif event.type == "FILL":
                        buy_hold_portfolio.update_fill(event)

And you would run it with:

$ python backtester.py --ticker AAPL

to run a backtest on Apple’s stock.

For a portfolio of multiple stocks you can run:

$ python backtester.py --tickers AAPL MSFT AMD INTC

However, there is a benefit to following each part of the series to better understand how it all comes together in the snippet above.

In the next article we will briefly cover at a high level how each parts fits into the whole.

Feel free to share the article on socials: