Hey folks!
Been through it all with elixir compilation. We were using Concourse for a bit, which allowed me to run images that had our application pre-compiled. I would rsync the files into the cloned branch, and only the changed files would compile.
For many reasons we moved to github actions, and since then I have been unable to create a setup that doesn’t compile/get dependencies every time. We also have issues with branches that change our structure.sql file, as the test runs depend on seeding a database, and new relations tend to not be applied unless we we “bust the cache” by incrementing a version number in the build cache string.
This all has created the headache of using up our minutes quite rapidly as our team (or a pull request) grows, and dipping deeply into our actions overage budget.
Below is part of a sample action
jobs:
ingest:
runs-on: ubuntu-18.04
services:
redis:
image: redis
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 6379:6379
postgres:
image: postgres:11.5
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: postgres
ports:
- 5432:5432
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
steps:
- uses: actions/checkout@v2
- name: Setup elixir
uses: erlef/setup-elixir@v1
with:
elixir-version: ${{ env.ELIXIR_VERSION }}
otp-version: ${{ env.OTP_VERSION }}
- name: Retrieve Mix Dependencies Cache
uses: actions/cache@v2
id: mix-cache
with:
path: backend/deps
key: ${{ env.OTP_VERSION }}-${{ env.ELIXIR_VERSION }}-test-deps-cache-{{ hashFiles('**/mix.lock') }}
restore-keys: |
${{ env.OTP_VERSION }}-${{ env.ELIXIR_VERSION }}-test-deps-cache-
- name: Retrieve Build Cache
uses: actions/cache@v2
id: build-cache
with:
path: backend/_build/**
key: ${{ env.OTP_VERSION }}-${{ env.ELIXIR_VERSION }}-app-build-cache-v5-${{ hashFiles('**/mix.lock','**/structure.sql') }}
restore-keys: |
${{ env.OTP_VERSION }}-${{ env.ELIXIR_VERSION }}-app-build-cache-v5-
- name: Install Mix Dependencies
run: |
cd backend/
mix local.rebar --force
mix local.hex --force
MIX_ENV=test mix deps.get
- name: Compile Dependencies
if: steps.build-cache.outputs.cache-hit != 'true'
run: |
cd backend
MIX_ENV=test mix compile
- name: Run Tests
run: |
cd backend/
MIX_ENV=test mix ecto.reset
mix cmd --app app mix test --color
Other examples I’ve found around the forums don’t seem to improve anything.
Does anything stand out here? How can I make these not get deps/compile on every run?
Finally, we also split our tests up so that each have their own actions file. If a PR is open and X app has files changed, the related action runs. If multiple apps have changes, those tests run. Our main package, all of the tests run. Is there possibly a way for a PR to not run all of these tests if a subsequent commit to that PR doesn’t change the apps that have already been tested?
Happy to share more details/clarify, apologies for any longwindedness or confusion.
