nature_whistle - telemetry driven alerting with recovery notifications

Introducing nature_whistle: Telemetry-driven alerting with recovery notifications

Hi everyone,

I’d like to share my first open-source Elixir package: nature_whistle.

The goal is simple: turn Telemetry events into actionable alerts, including recovery notifications when things return to normal.

Most monitoring tools do a good job of telling you when something breaks. But they often leave out an equally important part: knowing when things are healthy again without manually checking dashboards.

nature_whistle tries to solve that gap.


What it does

  • Listens to Telemetry events from the BEAM, Phoenix, Ecto, Oban, or custom metrics

  • Triggers alerts when thresholds are crossed

  • Sends a “calm” notification when metrics return to normal

  • Supports Slack, Microsoft Teams, generic webhooks, and a console notifier

  • Uses retries with exponential backoff for HTTP-based notifications

  • Lightweight design built on ETS + Telemetry (no heavy runtime overhead)


:light_bulb: Example usage

Here’s a simple example of how a monitor might look:

# 👉 PUT THIS CODE IN THE config.exs of your application after adding NatureWhistle.Application as one of your application's children

config :nature_whistle,
  monitors: [
    %{
      event: [:vm, :memory],
      threshold: 80,
      notifier: NatureWhistle.Notifiers.Slack,
      message: "Memory usage is high!"
    }
  ]

You can replace this with your real-world config showing:

  • Phoenix request latency

  • Ecto query time

  • Oban job failures

  • or any custom telemetry event


Why I built it

I originally got the idea after watching a talk by an Elixir developer talk on Telemetry in Elixir. It made me think about how much valuable operational data is already available in Telemetry events — and how easy it could be to build a small layer on top to turn them into meaningful alerts.

That exploration led to nature_whistle.


Design goals

  • Minimal footprint (ETS + Telemetry only)

  • Easy integration into existing apps

  • Extensible notifier system

  • Works with both infrastructure and application-level metrics


Links

GitHub:

Hex:


Questions for the community

  • What metrics do you currently alert on in your Elixir systems?

  • Do you currently use recovery notifications, or only failure alerts?

  • What notifier integrations would be most useful to you?

  • Would this fit alongside your existing monitoring setup, or replace part of it?


Feedback, criticism, and suggestions are very welcome.

3 Likes

Made some modifications on how to configure the package on your app’s config.exs file. This is how it should be done

config :nature_whistle,
  alerts: [
    %{
      id: :high_memory,
      event: [:vm, :memory, :total],
      measurement_key: :total,
      threshold: 1_073_741_824,        # 1 GB
      alert_message: "🚨 High memory: %{value} MB",
      calm_message: "✅ Memory back to normal: %{value} MB",
      cooldown_ms: 300_000,            # 5 minutes
      resolution_ms: 60_000,           # 1 minute
      notifier: :slack
    }
  ],
  notifiers: [
    slack: [webhook_url: "https://hooks.slack.com/services/..."],
    console: []   # optional, always available as fallback
  ],
  retry: [
    max_attempts: 3,
    base_delay_ms: 1000,
    max_delay_ms: 30_000
  ]