Image Processing API Architecture

Are the transformations constantly changing, or will the Ruby app be requesting relatively consistent sizes and transformations? If they don’t change much, consider putting Cloudfront in front of the requests to cache the assets after the first request with the specific size/transformation.

The advantages are much less load on your api (only needs called when the asset doesn’t exist) and you reduce your S3 egress costs because Cloudfront will cache it for a year (or whatever lifetime you set). The fallback URL for Cloudfront would hit your Elixir app to do the initial resize or transformation and the result would then be cached for subsequent requests. This might alleviate the need for any throttling/backpressure in your Elixir app.

If the resize/transformation requests vary, then the above won’t help you much, but there are addons in AWS, such as Imagizer AWS Marketplace: Nventify Imagizer for Amazon S3, which are tailor-made for this (why reinvent the wheel). Imagizer lets you use either their SAS platform, or run it on your own EC2 instances.