Live Algorithmic Trading with Python¶
How To Make Your Trading Strategies Work On Live Markets¶
© Ran Aroussi
@aroussi | aroussi.com | github.com/ranaroussi
August, 2017
Agenda¶
- Working with the 3 types of market data:
- EOD market data
- Timed market data snapshopts
- Live streaming data
- How to execute and monitor trades in live markets
- Live examples using Interactive Brokers and Oanda FX
Working with Market Data¶
Working with Market Data¶
- EOD market data (for daily strategies)
- Timed market data snapshopts (usually for intraday strategies)
- Live (or paper) streaming data (for anything, really 😁)
EOD Market Data Sources¶
(live example to follow)
- Historical market data providers
- Live market data feed providers
- Your broker
EOD Market Data Sources¶
- Historical market data providers
- Quandl
- Yahoo! Finance
- EODData.com
- Tick data providers (QuantQuote.com, TickData.com, ...)
- Live market data feed providers
- IQFeed
- Rithmic
- CQG
- ...
- Your broker
Downloading EOD Market Data¶
Quandl Example: EOD Stocks¶
In [3]:
import quandl
# quandl.ApiConfig.api_key = "YOUR_KEY_HERE"
df = quandl.get('WIKI/AAPL')
df.tail()
Out[3]:
Quandl Example: Continuous Futures Contract¶
In [4]:
df = quandl.get('CHRIS/CME_CL1')
df.tail()
# see more python methods @ quandl.com/tools/python
# Useful databases:
# - https://www.quandl.com/data/WIKI-Wiki-EOD-Stock-Prices
# - https://www.quandl.com/data/CHRIS-Wiki-Continuous-Futures
Out[4]:
Yahoo! Finance Example¶
(Sadly, this service became very unreliable in terms of availability)
In [5]:
from pandas_datareader import data as pdr
df = pdr.get_data_yahoo("AAPL", start="2010-01-01")
df.tail()
Out[5]:
Reading CSV Time-Series Files¶
In [6]:
# read csv from eoddata.com
df = pd.read_csv('./AAPL_20170102_20170823.csv')
df.tail()
Out[6]:
Reading CSV Time-Series Files¶
In [7]:
# parse date column and set as index while reading
df = pd.read_csv('./AAPL_20170102_20170823.csv',
parse_dates=['Date'], index_col=['Date'])
df.tail()
Out[7]:
Downloading Data from Oanda¶
Oanda Example (using REST API)¶
In [8]:
api_token, account_id = open('./oanda.conf').read().split(',')
In [9]:
import oandapy
oanda = oandapy.API(environment="practice", access_token=api_token)
In [10]:
data = oanda.get_history(instrument="EUR_USD",
granularity='D',
start='2017-01-01')
# convert candles to dataframe
df = pd.DataFrame(data['candles'])
df.set_index('time', inplace=True)
df.index = pd.to_datetime(df.index)
df.tail()
Out[10]:
Getting Snapshot Data¶
- Get price of EUR/USD every 5 seconds
- Stop after 5 ticks (for demo porposes)
In [13]:
reqs = 0
while True:
price = oanda.get_prices(instruments="EUR_USD")
print(price) # price available via price['prices'][0]
reqs += 1
if (reqs == 5):
break
time.sleep(1)
Downloading Data from Interactive Brokers¶
Downloading Historical Data from Interactive Brokers¶
Before anything...
- Requires IB's TWS or IB Gateway running on your machine with API permissions enabled
- For order execution, you also need to check "Bypass Order Precautions for API Orders" under "Precautions"
- All IB examples will be using the ezIBpy library
Interactive Brokers Example¶
- Connect to TWS
- Create a callback method
- Create a "contract"
- Request historical data
- Access historical data
In [14]:
import ezibpy
import time
# initialize ezIBpy
ibConn = ezibpy.ezIBpy()
In [15]:
# create callback method
completed = False
def ibCallback(caller, msg, **kwargs):
global completed
if caller == "handleHistoricalData":
completed = kwargs['completed']
In [28]:
# connect to TWS
ibConn.connect(port=4001)
ibConn.ibCallback = ibCallback
# create a contract
contract = ibConn.createStockContract("AAPL")
# request historical data
ibConn.requestHistoricalData(resolution="1 day", lookback="1 Y")
# wait until data is downloaded
while not completed:
time.sleep(1)
# if we got here, download is completed
print("download completed")
# cancel request and disconnect from TWS
ibConn.cancelHistoricalData(contract)
ibConn.disconnect()
In [18]:
# access historica data
df = ibConn.historicalData['AAPL']
df.tail()
Out[18]:
Working with Live, Streaming Data¶
Oanda Streaming Data¶
Oanda Streamer¶
In [19]:
# create streamer
class OandaBasicStreamer(oandapy.Streamer):
def __init__(self, *args, **kwargs):
oandapy.Streamer.__init__(self, *args, **kwargs)
self.ticks = 0
def stream(self, account_id, symbols):
oandapy.Streamer.start(self,
accountId=account_id,
instruments=symbols
)
def on_error(self, data):
self.disconnect()
# logic goes here:
def on_success(self, data):
if 'tick' in data:
self.ticks += 1
print(data)
In [23]:
stream = OandaBasicStreamer(environment="practice", access_token=api_token)
try:
stream.rates(account_id, instruments="EUR_USD")
except (KeyboardInterrupt, SystemExit):
print('Stopped by user...')
Plotting Oanda's Streaming Data¶
In [29]:
import plotly.plotly as ply
import plotly.graph_objs as pgo
import plotly.tools as tls
stream_ids = tls.get_credentials_file()['stream_ids']
In [34]:
def init_plotly_streaming_chart(token, maxpoints=100,
title="Untitled", auto_open=False):
# create a new streaming chart
stream = pgo.Stream(
token=token,
maxpoints=maxpoints
)
# initialize streaming chart
tracer = pgo.Scatter(x=[], y=[], stream=stream)
figure = pgo.Figure(data=pgo.Data([tracer]),
layout=pgo.Layout(title=title))
return ply.plot(figure, auto_open=auto_open)
In [40]:
chart_url = init_plotly_streaming_chart(
token=stream_ids[0],
maxpoints=50,
title="Oanda Streaming Demo")
print(chart_url)
In [36]:
# create streamer
class OandaPlotter(OandaBasicStreamer):
# logic goes here:
def on_success(self, data):
if 'tick' in data:
midprice = (data['tick']['bid'] + data['tick']['ask']) / 2
streamer.write(dict(x=self.ticks, y=midprice))
self.ticks += 1
In [41]:
tls.embed(chart_url)
Out[41]:
In [42]:
stream = OandaPlotter(environment="practice", access_token=api_token)
streamer = ply.Stream(stream_ids[0])
streamer.open()
try:
stream.rates(account_id, instruments="EUR_USD")
except (KeyboardInterrupt, SystemExit):
print('Stopped by user...')
streamer.close()
Interactive Brokers Streaming Data¶
In [43]:
import ezibpy
import time
# initialize ezIBpy
ibConn = ezibpy.ezIBpy()
# create callback method
def ibCallback(caller, msg, **kwargs):
if "tick" in kwargs:
tick = ibConn.marketData[msg.tickerId]
tick['datetime'] = tick.index
print(tick.to_dict(orient='records')[0])
# connect to TWS
ibConn.connect(port=4001)
ibConn.ibCallback = ibCallback
In [44]:
# create a contract
contract = ibConn.createFuturesContract("ES",
exchange="GLOBEX", expiry="201709")
# request market data for the contract
ibConn.requestMarketData()
try:
while True:
time.sleep(1)
except (KeyboardInterrupt, SystemExit):
print('Stopped by user...')
# cancel request and disconnect from TWS
ibConn.cancelMarketData(contract)
ibConn.disconnect()
Plotting IB's Streaming Data¶
In [45]:
chart_url = init_plotly_streaming_chart(
token=stream_ids[1],
maxpoints=50,
title="IB Streaming Demo")
print(chart_url)# create streamer
In [46]:
ticks = 0
def ibCallback(caller, msg, **kwargs):
global ticks
if "tick" in kwargs:
tick = ibConn.marketData[msg.tickerId]
streamer.write(dict(x=ticks, y=tick['last'].values[0]))
ticks += 1
In [47]:
tls.embed(chart_url)
Out[47]:
In [48]:
streamer = ply.Stream(stream_ids[1])
streamer.open()
ibConn.connect(port=4001)
ibConn.ibCallback = ibCallback
contract = ibConn.createFuturesContract("ES",
exchange="GLOBEX", expiry="201709")
ibConn.requestMarketData()
try:
while True:
time.sleep(1)
except (KeyboardInterrupt, SystemExit):
print('Stopped by user...')
# cancel request and disconnect from TWS
ibConn.cancelMarketData(contract)
ibConn.disconnect()
streamer.close()
Executing & Monitoring Orders¶
Executing Orders on Oanda¶
In [49]:
# market order
order = oanda.create_order(account_id,
instrument="EUR_USD",
units=1000,
side='buy',
type='market'
)
pretty_print(order)
In [50]:
# limit order
trade_expire = datetime.datetime.now() + datetime.timedelta(hours=1)
order = oanda.create_order(account_id,
instrument="EUR_USD",
units=1000,
side='buy',
type='limit',
price=1.1,
expiry = trade_expire.isoformat("T")
)
pretty_print(order)
In [51]:
# get trade, positions, orders information
positions = oanda.get_positions(account_id)
trades = oanda.get_trades(account_id)
orders = oanda.get_orders(account_id)
positions, trades, orders
Out[51]:
In [52]:
# close posiiton (market)
order = oanda.create_order(account_id,
instrument="EUR_USD",
units=1000,
type='market',
side='sell'
)
pretty_print(order)
In [53]:
# get trade, positions, orders information
positions = oanda.get_positions(account_id)
trades = oanda.get_trades(account_id)
orders = oanda.get_orders(account_id)
positions, trades, orders
Out[53]:
In [54]:
# close pending order
orderId = orders['orders'][0]['id']
oanda.close_order(account_id, orderId)
pretty_print(order)
In [55]:
# get trade, positions, orders information
positions = oanda.get_positions(account_id)
trades = oanda.get_trades(account_id)
orders = oanda.get_orders(account_id)
positions, trades, orders
Out[55]:
Executing Orders on Interactive Brokers¶
In [56]:
import ezibpy
# initialize ezIBpy
ibConn = ezibpy.ezIBpy()
# create callback method
def ibCallback(caller, msg, **kwargs):
if caller == "handleOrders":
order = ibConn.orders[msg.orderId]
print('Order > ', order)
In [57]:
# issue market order
ibConn.connect(port=4001)
ibConn.ibCallback = ibCallback
contract = ibConn.createFuturesContract("ES",
exchange="GLOBEX", expiry="201709")
order = ibConn.createOrder(1)
orderId = ibConn.placeOrder(contract, order)
time.sleep(1)
ibConn.disconnect()
In [58]:
# get trade, positions, orders information
ibConn.positions, ibConn.portfolio, ibConn.orders
Out[58]:
In [59]:
# bracker limit order
ibConn.connect(port=4001)
ibConn.ibCallback = ibCallback
contract = ibConn.createFuturesContract("ES",
exchange="GLOBEX", expiry="201709")
order = ibConn.createBracketOrder(
contract, quantity=1, entry=2400, target=2500., stop=2300.)
time.sleep(1)
ibConn.disconnect()
In [60]:
# get order information
pretty_print(ibConn.orders)
Live Trading Example¶
Live Trading "Template"¶
- Subscribe to streaming data (resample data if necessary)
- Run strategy logic
- Execute & monitor orders
In [61]:
# build ohlc _bars
_bars = pd.DataFrame()
_ticks = pd.DataFrame()
_res = '1T'
_window = 100
def ibCallback(caller, msg, **kwargs):
global _bars, _ticks, _res, _window
if caller == "handleOrders":
order = ibConn.orders[msg.orderId]
print('Order > ', order)
elif "tick" in kwargs:
# read tick
tick = ibConn.marketData[msg.tickerId]
# add to _ticks df
_ticks = pd.concat([_ticks, tick])
_ticks.index = pd.to_datetime(_ticks.index)
# on_new_bar(_ticks.copy())
# return
# resample _ticks to 1min _bars
ticks_as_bars = _ticks['last'].resample(_res).ohlc()
ticks_as_bars['volume'] = _ticks['lastsize'].resample(_res).sum()
# replace latest bar data with resampled _ticks
_bars = _bars[_bars.index != ticks_as_bars.index[-1]]
_bars = pd.concat([_bars, ticks_as_bars[-1:]])
# remove old _ticks
pre_trimmed_ticks = len(_ticks)
_ticks = _ticks[_ticks.index >= ticks_as_bars.index[-1]]
# run strategy logic
if len(_ticks) != pre_trimmed_ticks:
on_new_bar(_bars[-_window:].copy())
In [62]:
def on_new_bar(bars):
position = 0
if contractString in ibConn.positions:
position = ibConn.positions[contractString]['position']
# calculate sma
bars['sma'] = bars['last'].rolling(2).mean()
# get lastest bar as dict
bar = bars.to_dict(orient='records')[-1]
order_qty = 0
if bar['last'] > bar['sma'] and position <= 0:
order_qty = abs(position) + 1
elif bar['last'] < bar['sma'] and position >= 0:
order_qty = -abs(position) - order_qty
# send order?
if order_qty != 0:
print('order qty ==>', order_qty)
order = ibConn.createOrder(order_qty)
ibConn.placeOrder(contract, order)
In [64]:
ibConn.connect(port=4001)
ibConn.ibCallback = ibCallback
contract = ibConn.createFuturesContract("ES",
exchange="GLOBEX", expiry="201709")
contractString = ibConn.contractString(contract)
ibConn.requestMarketData()
try:
while True:
time.sleep(1)
except (KeyboardInterrupt, SystemExit):
print('Stopped by user...')
# cancel request and disconnect from TWS
ibConn.cancelMarketData(contract)
ibConn.disconnect()
To Be Continued...¶
This was the part 3 out of the 4-part webinar series
Up Next...¶
Prototyping Trading StrategiesBacktesting & OptimizationLive Trading- Using Machine Learning in Trading
Q&A Time¶
Live Algorithmic Trading with Python¶
Thank you for attending!¶
© Ran Aroussi
@aroussi | aroussi.com | github.com/ranaroussi
August, 2017