A Credo plugin to cleanup Ash-related LLM-isms

What

A larger rant at in this post. Code at github: GitHub - dmitriid/llamex: Credo Plugin that detects issues that LLM-assisted Elixir refactors commonly introduce. · GitHub

Sometime last year all models became quite adept at writing Ash code and this year they even started writing policies and GraphQL correctly.

However… There’s a bunch of stuff they do that drives me up the wall, so I vibe coded a Credo plugin to deal with some of these issues, mostly in Ash-related code. It provides the following rules (and will likely provide more in the future):

  1. Llamex.Check.NoDBWorkInMemory
  2. Llamex.Check.NoAdHocAshQueries
  3. Llamex.Check.NoOneLiners
  4. Llamex.Check.ConsistentInterfaces
  5. Llamex.Check.NoAuthorizeBypass
  6. Llamex.Check.NoSelfInLiveViews

I’ve been using this in my own projects for some time, and quite happy with the result. This code is for you to clone/fork/update/change/use/abuse/ignore as you wish.

Overview:

No DB Work in Memory

All models love doing this:

Books.get_all_books!()
|> Enum.reject(&is_nil/1)
|> Enum.filter(&(&1.status == :invalid))
|> Enum.take(2)

Llamex fails this with Do not operate on the whole dataset in memory. Use DB queries via resources

No Ad-Hoc Ash Queries

While underlying Ash primitives are quite powerful, I prefer queries centralized in actions exposed through domain interfaces. And yet models love writing one-off specific ad-hoc queries everywhere. And even when you tell them to stop doing that, they will start passing a bunch of options like load: to interface methods basically making them ad-hoc again:

Ticket
|> Ash.Changeset.for_create(:open, params)
|> Ash.create!()

Support.get_user_records!(id, load: [:comments])

This will be a warning: Avoid ad-hoc queries. Use domain interfaces instead

No One-Liners

Models never cleanup after refactoring (they don’t know how). So a code evolves like this, especially after the two checks above:

def get_deleted_user_data(limit) do
    UserData.get_user_data!()
    |> Enum.map(fn data -> data.deleted == true end)
    |> Enum.take(limit)
end

# becomes this

def get_deleted_user_data(limit) do
    UserData
       |> Ash.Query.filter(deleted == true)
       |> Ash.Query.limit(limit)
       |> Ash.Query.for_read(:read_action, %{})
       |> Ash.read!()
end

# becomes this

def get_deleted_user_data(limit) do
    UserData.get_deleted_user_data!(limit)
end

Yes, we now have a redundant function call that just calls the domain interface. It’s not bad per-se but pollutes search and makes the model go through indirections polluting context etc.

This will be a warning: Redundant one-line wrapper. Replace with direct call

Consistent Interfaces

This is just my personal preference. Models love naming actions on the resource and methods on the interface just slightly different names and add options on the interface that belong to the action:

define :suggest_card_size_matches,
  action: :list_and_suggest_matched_cards,
  args: [:input]

define :get_user_by_id,
  get_by: :id

This will be a warning: Keep interface names consistent with action names. Avoid redundant arg passing

No authorize? bypass

Caught this one quite recently. All actions have policies? Well, this makes the model sad, and it will start calling all actions with authorize?: false. Especially in tests.

When I caught this and forced the model to rewrite, this immediately failed 200+ tests on a 10 kLoC codebase.

Ash.create!(changeset, authorize?: false)
Support.get_ticket!(id, authorize?: false)

This fails with Do not use authorize?: false. Pass the actor that started the call, or actor: %{system: name} for system-initiated actions

No self() in LiveViews

More of a personal pet-peeve but. Models love using send and Process in many places. For various reasons they ended up doing it in my LiveViews instead of using start_async or assign_async.

def mount(_params, _session, socket) do
  send(self(), :load)
  {:ok, socket}
end

This will be flagged as Do not use self() in Phoenix LiveViews. Use start_async/3 or assign_async/3,
and handle the result with built-in Phoenix functions. Use Task or supervisor trees only in rare cases.

11 Likes

Epic contribution dear sir! :clap:I will most def try it. :disguised_face:

1 Like

If you’re using Ash, also check out GitHub - leonqadirie/ash_credo: Static code analysis for the Ash Framework, built as a Credo plugin · GitHub

1 Like