Deep LOB trading: Half a second please!

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. In view of the referees’ feedback and my own reading of your paper, I invite you to address all issues noted below. I consider these issues to be major in nature, requiring more than a superficial or minor revision. In particular, there are important deficiencies in the methodological section that seriously hinder the understanding of the work as well as the results obtained. Their robustness is also unclear, so I have doubts as to whether the conclusions are supported by the results presented.

It is observed that the thickness of the market prices is correlated with the trading intensity inversely. As a larger trading intensity decreases the market impact in execution which leads a decrease in price movements; it causes a lower price that is presented in Fig. For the case of a quadratic utility function, we derive the optimal spreads for limit orders and observe their behaviors. For this purpose, we should obtain an appropriate solution to with the final condition and show that this solution verifies the value function . While the market maker wants to maximize her profit from the transactions over a finite time horizon, she also wants to keep her inventories under control and get rid of the remaining inventories at the final time T by the penalization terms. We consider an agent who takes a short position in a contingent claim and employs limit orders and market orders to trade in the underlying asset to maximize expected utility of termina…

Can maker-taker fees prevent algorithmic cooperation in market making?

The agent can also skew the bid and ask prices output by the Avellaneda-Stoikov procedure, tweaking them and, by so doing, potentially counteract the limitations of a static Avellaneda-Stoikov model by reacting to local market conditions. The agent learns to adapt its risk aversion and skew its bid and ask prices under varying market behaviour through reinforcement learning using two variants (Alpha-AS-1 and Alpha-AS-2) of a double DQN architecture. The central notion is that, by relying on a procedure developed to minimise inventory risk (the Avellaneda-Stoikov procedure) by way of prior knowledge, the RL agent can learn more quickly and effectively. A wide variety of RL techniques have been developed to allow the agent to learn from the rewards it receives as a result of its successive interactions with the environment. A notable example is Google’s AlphaGo project , in which a deep reinforcement learning algorithm was given the rules of the game of Go, and it then taught itself to play so well that it defeated the human world champion.

Kumar , who uses Spooner’s RL algorithm as a benchmark, proposes using deep recurrent Q-networks as an improved alternative to DQNs for a time-series data environment such as trading. Gašperov and Konstanjčar tackle the problem be means of an ensemble of supervised learning models that provide predictive buy/sell signals as inputs to a DRL network trained with a genetic algorithm. The same authors have recently explored the use of a soft actor-critic RL algorithm in market making, to obtain a continuous action space of spread values . Comprehensive examinations of the use of RL in market making can be found in Gašperov et al. and Patel .

Related work on machine learning in trading

At the end of the day, the market maker will be loaded with BTC, and his total inventory will have a smaller value. Standard MM benchmarks like the AS approximations are ill-suited for our framework since they take into account neither the existence of the bid-ask spread nor the discrete nature of the underlying LOB . We consider a class of MM strategies linear in inventory and including inventory constraints.

The inventory position is flipped, and now the bid offers are being created closer to the market mid-price. The value of q on the formula measures how many units the market maker inventory is from the desired target. The basic strategy for market making is to create symmetrical bid and ask orders around GALA the market mid-price. Making may be dangerous when markets or networks are unstable. » Maps states to probability distributions over the action space.

Table 3

Hawkes es and their applications to high-frequency data modeling. Handbook of High-Frequency Trading and Modeling in Finance, 9, p.183. The results clearly indicate that higher limit order transaction costs, quite expectedly, lead to fewer realized transactions , as the DRL controller becomes increasingly discriminating about when and if to post limit orders in the presence of high transaction costs.

Our algorithm works through 10 generations of instances of the AS model, which we will refer to as individuals, each with a different chromosomal makeup . In the first generation, 45 individuals were created by assigning to each of the four genes random values within the defined ranges. These individuals run through the orderbook data, and are then ranked according to the Sharpe ratio they have attained. For each subsequent generation 45 new individuals run through the data and then added to the cumulative population, retaining all the individuals from previous generations. The 10 generations thus yield a total of 450 individuals, ranked by their Sharpe ratio. Note that, since we retain all individuals from generation to generation, the highest Sharpe ratio the cumulative population never decreases in subsequent generations.

A discount factor (γ) by which future rewards are given less weight than more immediate ones when estimating the value of an action (an action’s value is its relative worth in terms of the maximization of the cumulative reward at termination time). There are many exciting models out there with different approaches, and with HFTs dominating the market-making scene in the last years, there is a lot for our team to explore. This will set “boundaries” to the calculated optimal spread, so hummingbot will never create your orders with a spread smaller than the minimum nor bigger than the maximum. As usual, you can create a new strategy on Hummingbot using the create command. Since this is a market-making strategy, some configurations will be similar to the pure market-making strategy, so we will cover what is different in this article. Reading the paper, you won’t find any direct indication of calculating these two parameters’ values.

Section 5 describes the experimental setup for backtests that were performed on our RL models, the Gen-AS model and two simple baselines. The results obtained from these tests are discussed in Section 6. The concluding Section 7 summarises the approach and findings, and outlines ideas for model improvement. In order to analyze the experimental results, we work on the models that we have DOGE avellaneda-stoikov paper derived using different metrics. It is salient to mention that the market maker modifies her qualitative behavior in various situations, i.e., changing inventory levels, utility functions. By our numerical results, we deduce that the jump effects and comparative statistics metrics provide us with the information for the traders to gain expected profits.

Stochastic Control/Reinforcement Learning for Optimal Market Making

The https://www.beaxy.com/ bid and ask quotes are obtained from a set of formulas built around these parameters. These formulas prescribe the AS strategy for placing limit orders. The rationale behind the strategy is, in Avellaneda and Stoikov’s words, to perform a ‘balancing act between the dealer’s personal risk considerations and the market environment’ [ibid.].

You might have noticed that I haven’t added volatility(σ) on the main factor list, even though it is part of the formula. That is because volatility value depends on the market price movement, and it isn’t a factor defined by the market maker. If the market volatility increases, the distance between reservation price and market mid-price will also increase.