How to test OpenTelemetry(API)?

Questions & Help Questions
, ,
1

Say I have some code like this:

require OpenTelemetry.Span
require OpenTelemetry.Tracer
def hello do
  OpenTelemetry.Tracer.with_span "hello method" do
    val = :world

    OpenTelemetry.Span.set_attributes(result: val)

    val
  end
end

I’d like to be able to test:

  • that a span with the name “hello method” was created
  • that it had the attribute result: :world
  • what the parent id was, if any
  • whether it was recorded
  • whether it was sampled

With Telemetry you can :telemetry.attach and give it a fake handler. Not sure what the best approach is for OpenTelemetry. Anyone have ideas?

2

In the application level, I think all we need is to test that OpenTelemetry’s functions are being called, rather than test how they perform and what they produce.

With Mox it might be something like

# define mock in helper
Mox.defmock(OpenTelemetry.SpanMock, for: OpenTelemetry.Span)

# in test case
expect(OpenTelemetry.SpanMock, :set_attributes, fn [result: :world] -> :ok end)
MyModule.hello()

With this we are making sure that MyModule.hello() invokes OpenTelemetry’s set_attributes function with our desired attribute. (Similarly we can write an expectation for Tracer.with_span).

For the rest, (wether it’s recorded, sampled) I would rely on tests in library and assume that it works properly.

In other words, in my application I just make sure that I gave 3rd party lib required info - the rest is not my responsibility :sweat_smile:

3

Yeah, I think that’s good advice for client apps. I didn’t know Span defined a behavior and hadn’t thought about mocking the set_attributes method, so that’s good to know!

I’m actually working on a library that defines and links spans so it’s nice to have a bit more control. A couple folks on slack pointed me toward the pid and ets exporters. The example they linked registers the pid exporter to send spans to the test process and then assert that the received span has the correct attributes.

https://github.com/opentelemetry-beam/opentelemetry_phoenix/blob/main/test/opentelemetry_phoenix_test.exs#L99-L166

setup do
    :application.stop(:opentelemetry)
    :application.set_env(:opentelemetry, :tracer, :ot_tracer_default)

    :application.set_env(:opentelemetry, :processors, [
      {:ot_batch_processor, %{scheduled_delay_ms: 1}}
    ])

    :application.start(:opentelemetry)

    :ot_batch_processor.set_exporter(:ot_exporter_pid, self())
    :ok
  end

  test "records spans for Phoenix web requests" do
    OpentelemetryPhoenix.setup()

    :telemetry.execute(
      [:phoenix, :endpoint, :start],
      %{system_time: System.system_time()},
      %{conn: @conn, options: []}
    )

    :telemetry.execute(
      [:phoenix, :router_dispatch, :start],
      %{system_time: System.system_time()},
      %{
        conn: @conn,
        plug: MyStoreWeb.UserOrdersController,
        plug_opts: :index,
        route: "/users/{user_id}/orders",
        path_params: %{"user_id" => "123"}
      }
    )

    :telemetry.execute(
      [:phoenix, :endpoint, :stop],
      %{duration: 444},
      %{conn: stop_conn(@conn), options: []}
    )

    expected_status = OpenTelemetry.status(:Ok, "Ok")

    assert_receive {:span,
                    span(
                      name: "GET /users/{user_id}/orders",
                      attributes: list,
                      status: ^expected_status
                    )}

    assert [
             {"http.client_ip", "10.211.55.2"},
             {"http.host", "mystore"},
             {"http.method", "GET"},
             {"http.scheme", "http"},
             {"http.status", 200},
             {"http.target", "/users/123/orders"},
             {"http.user_agent",
              "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:79.0) Gecko/20100101 Firefox/79.0"},
             {"net.host.ip", "10.211.55.2"},
             {"net.host.port", 4000},
             {"net.peer.ip", "10.211.55.2"},
             {"net.peer.port", 64291},
             {"net.transport", "IP.TCP"},
             {"phoenix.action", "index"},
             {"phoenix.plug", "Elixir.MyStoreWeb.UserOrdersController"}
           ] == List.keysort(list, 0)
  end
4

Nice!

Honestly, I haven’t worked with OpenTelemetry, and don’t know if OpenTelemetry.Span defines behaviour (seems like it’s not)
But for mocking with Mox if behaviour is not defined by lib I would write it and have the lib’s original module as one of the implementations.

Thank you for following up in here with those ideas!