Stock trading in Elixir

[rm-rf-etc]

Thanks @GunnarPDX, that was quick. Looking forward to testing it when I have some time (probably this weekend).

@CinderellaMan

From everything that I’ve done over the years, I think the system needs to be really simple to succeed. People will build on top of it so it needs to be as simple as this is the data and you do whatever you want with it (write your strategy). In my course (which looks already much different than Igthorn: GitHub - Cinderella-Man/hedgehog at video_07 · GitHub [at this moment at the 7th episode]) I took a PubSub approach. If you think about it - the only things that consumer needs to know is the data struct that will be broadcasted and the topic name. It works for anything that you want to stream - just prefix your stuff with type of what you stream. You would have topics like “stocks:AAPL”, “forex:GBPUSD”, “crypto:ADAUSDT” - nothing would interfere with each other. Then you write your specific consumers for crypto data, forex data, or stock data. There will be multiple consumers for single topic - one storing data to Postgres (or timescaleDB - or both[would guess that would be 2 consumers though!] as this could be beneficial as well!), one making some intermediate indicators and publishing them forward to topics like “forex:GBPUSD:ATR” or something like that. Obviously one of the consumers will be your strategy.

I’ve been thinking about this quite a bit. I think this is the right idea. Although I’m not sure how I feel about the indicators going to topics. The way I’ve approached it is that I have a state struct that contains the OHLC price chart and the output from whatever indicators we want to use:

  %LiveTrade{
    charts: %{
      m1: %{
        max_list_length: 200,
        bars: [],
        macd: %Indicators.MACD{}
        rsi: %Indicators.RSI{}
        ... any others you want here ...
      },
      m5: %{
        max_list_length: 40,
        bars: [],
        macd: %Indicators.MACD{}
        rsi: %Indicators.RSI{}
        ... any others you want here ...
      }
    }
  }

So with this, the timeframe of the price chart is inherently the same for any of the indicators attached. :charts can contain any number of timeframes we want, as the module just loops over all the keys under :charts. So the whole system works as a state machine, the previous state is passed into the first function along with the new candlestick object, the function merges it into the chart, and then this state object is piped into each indicator function which then generates the updated indicator list and writes it to the state struct.

So I’m thinking about a few things from here.

1. The chart max_list_length is used to keep the list from getting too large for memory, so the data then needs to be written to the DB, and I haven’t worked on that piece yet.
2. Sometimes some of the real-time data shows up after a delay (https://polygon.io/blog/aggregate-bar-delays/) and for some use cases, we want to update bars that have already been on the chart for a while. In this case, we would have to basically rollback to the part of the chart which is going to be modified, merge the delayed data in, and then iterate forward until we’re back to the latest OHLC packet. I haven’t worked on this problem yet either.
3. In the case of backtesting, we have to make sure we have all the necessary data in the DB before we start. In some cases, we may have parts of the data and gaps in between, so we would want to only download the data needed for filling the gaps, and then allow the backtest to run. So I’m thinking about how we might solve this with a PubSub approach. And I’m also thinking about how it would work if built on just Stream and Task.async. Perhaps the primitive would be Stream objects, and then the layer above could implement the PubSub parts.