Optimizing WebSocket Compression in Plug Cowboy: Reducing CPU Overhead on High-Traffic Socket Server

Questions & Help Questions
, ,
5

Thought more about this, you can get the compress once behaviour out of the box with phoenix channels which uses something called fastlane within the serializers. (see this forum post explained by LostKobrakai How to scale Phoenix Pubsub event publishing? - #2 by LostKobrakai ). When you broadcast to a channel it will serialize once and reuse that for each listener. You could write a custom serializer which would compress the payload with msgpack or zstd (train a dictionary on a bunch of your data and use that on both sides which would compress way better on smaller payloads). An example of a custom serializer is something like this; (source: blog/binary_data_over_phoenix_sockets/web/transports/message_pack_serializer.ex at b164ae5e8fb4701ee40925aca9aef2297b80be95 · Kaizen-Gaming/blog · GitHub )

defmodule BinaryDataOverPhoenixSockets.Transports.MessagePackSerializer do
  @moduledoc false

  @behaviour Phoenix.Transports.Serializer

  alias Phoenix.Socket.Reply
  alias Phoenix.Socket.Message
  alias Phoenix.Socket.Broadcast

  # only gzip data above 1K
  @gzip_threshold 1024

  def fastlane!(%Broadcast{} = msg) do
    {:socket_push, :binary, pack_data(%{
      topic: msg.topic,
      event: msg.event,
      payload: msg.payload
    })}
  end

  def encode!(%Reply{} = reply) do
    packed = pack_data(%{
      topic: reply.topic,
      event: "phx_reply",
      ref: reply.ref,
      payload: %{status: reply.status, response: reply.payload}
    })
    {:socket_push, :binary, packed}
  end

  def encode!(%Message{} = msg) do
    # We need to convert the Message struct into a plain map for MessagePack to work properly.
    # Alternatively we could have implemented the Enumerable behaviour. Pick your poison :)
    {:socket_push, :binary, pack_data(Map.from_struct msg)}
  end

  # messages received from the clients are still in json format;
  # for our use case clients are mostly passive listeners and made no sense
  # to optimize incoming traffic
  def decode!(message, _opts) do
    message
    |> Poison.decode!()
    |> Phoenix.Socket.Message.from_map!()
  end

  defp pack_data(data) do
    msgpacked = MessagePack.pack!(data, enable_string: true)
    gzip_data(msgpacked, byte_size(msgpacked))
  end

  defp gzip_data(data, size) when size < @gzip_threshold, do: data
  defp gzip_data(data, _size), do: :zlib.gzip(data)
end

Maybe a completely different direction but giving you different options :smiley: