Found a solution right after posting.. you can setup a filter with the publish DSL, so I changed my pub_sub to this:
publish :create, ["new", :user_id], filter: fn %{data: data} -> not is_nil(data.user_id) end
publish :create, ["new", nil], filter: fn %{data: data} -> is_nil(data.user_id) end
