Another reason for slow compilation is if given module depends on other slow modules. [1] Is a great resource for understanding compilation. In general, you want to avoid compile-time dependencies between modules as much as possible.
On our Phoenix app, switching from import MyApp.Router.Helpers to alias MyApp.Router.Helpers, as: Routes speed up compilation and limited re-compilations greatly, see: [2]
Elixir 1.6 will ship with improvements to mix xref which should make it easier to investigate this stuff.
[1] http://milhouseonsoftware.com/2016/08/11/understanding-elixir-recompilation/
[2] Do not set compile time dependencies on the router · Issue #2530 · phoenixframework/phoenix · GitHub
[3] elixir/CHANGELOG.md at main · elixir-lang/elixir · GitHub
