advertisement

Let's use reinforcement learning in AI to build a digital forex trading robot. In this article, we will create and apply a number of enhanced learning framework to learn how to create a Bitcoin trading robot. In this tutorial, we will use the OpenAI gym and PPO robot from the stable-baselines library, which is a branch of the OpenAI base library. Thank you very much for the open source software provided by OpenAI and DeepMind to deep learning researchers in recent years. If you haven't seen their incredible achievements with AlphaGo, OpenAI Five, AlphaStar, and other technologies, you may have been living in isolation this past year, but you should check them out. AlphaStar Training: https://deepmind.com/blog/alphastar-mastering-real-time-strategy-game-starcraft-ii/ Although we won't create anything impressive, it is still not easy to trade Bitcoin robots in day trading. However, as Teddy Roosevelt once said, There is no value in anything that is too simple. Therefore, we must not only learn to trade ourselves, but also let robots trade for us. Plan Create a gym environment for our robot to perform machine learning Renders a simple and elegant visual environment. Train our robot to learn a profitable trading strategy If you are not familiar with how to create gym environments from scratch, or how to simply render the visualization of these environments. Before continuing, feel free to search Google for an article of this type. These two actions will not be difficult even for the most novice programmers. Getting started In this tutorial, we will use the Kaggle dataset generated by Zielak. If you would like to download the source code, it will be provided in my Github repository, along with the .csv data file. Well, let's get started. First, let's import all the necessary libraries. Make sure to use pip to install the missing libraries. import gym import pandas as pd import numpy as np from gym import spaces from sklearn import preprocessing Next, let's create our class for the environment. We need to pass a Pandas data frame number and an optional initial_balance and inner_window size, which will indicate the number of steps in past time observed by the robot at each step. By default, the commission of each transaction is 0.075%, that is, the current Bitmex exchange rate, and the serial parameter defaults to false, which means that our data frame number will be traversed by random fragments by default. We also call dropna() and reset_index() on the data, first we delete the row with the value NaN and then reset the index of the frame number, because we have deleted the data. class BitcoinTradingEnv(gym.Env): """A Bitcoin trading environment for the OpenAI gym""" metadata = {'render.modes': ['live', 'file', 'none']} scaler = preprocessing.MinMaxScaler() spectator = None def __init__(self, df, lookback_window_size=50, commission=0.00075, initial_balance=10000 series = False): super(BitcoinTradingEnv, self).__init__() self.df = df.dropna().reset_index() self.lookback_window_size = lookback_window_size self.initial_balance = initial_balance self.commission = commission self.serial = serial # Actions of the format Buy 1/10, Sell 3/10, Hold, etc. self.action_space = spaces.MultiDiscrete([3, 10]) # Look at OHCLV values, net worth and trading history. self.observation_space = spaces.Box(low=0, high=1, shape=(10, lookback_window_size + 1), dtype=np.float16) Our action_space is represented here as a group of 3 options (buy, sell or hold) and another group of 10 quantities (1/10, 2/10, 3/10, etc.). When we choose to buy, we will buy amount * word of own balance of BTC. To sell, we will sell the quantity * value of self.btc_held in BTC. Of course, keep will ignore the quantity and do nothing. Our observation_space is defined as a continuous floating po.

advertisement

advertisement