Tiered caches will help with the build time a lot. On CircleCI, even with small test runners, you are able to cache the hex deps, the compiled code, and PLT files if dialyzer is your thing.
I do this for all my applications, but here is an example for an open source project:
https://github.com/sorentwo/oban/blob/master/.circleci/config.yml
