Image - an image processing library based upon Vix

News Announcing
,
189

image_vision was also updated this week with some cool new models; all callable in a couple of lines without ML expertise.

Image.FaceDetection

Fast face detection with bounding boxes, confidence scores, and the five canonical landmarks (eyes, nose tip, mouth corners). The default model is YuNet 2023-March. It’s quite small at ~340 KB and runs on the CPU.

Includes crop_largest/2 as the entry point for face-aware crop bias (think gravity: :face).

Image.Background

Provides foreground/background separation.

  • remove/2 returns the input with the background made transparent;

  • mask/2 returns the foreground mask alone for custom compositing.

The default model is BiRefNet lite (MIT, ~210 MB) via Ortex. There’s a mix task for downloading it.

There is a livebook for exploring background removal.

Image.Captioning

Provides a natural-language description of an image. caption/2 returns strings like "a man riding a horse with a bird of prey".

The default model is BLIP base (BSD-3-Clause, ~990 MB) via Bumblebee. Heavy enough that it isn’t auto-started by default — flip autostart: true or add it to your supervisor.

There is a livebook for exploring image captioning.

Image.ZeroShot

Classify an image against arbitrary labels you supply at call time with no retraining.

  • classify/3 returns ranked {label, score} pairs;
  • label/3 returns just the best;
  • similarity/3 computes CLIP-space cosine similarity between two images.

The default model is OpenAI CLIP ViT-B/32 (MIT, ~600 MB). The default prompt template "a photo of {label}" boosts accuracy on bare-noun labels; override or disable as needed.

There is a livebook for exploring zero-shot classification.