Why is my random number lookup slow using lists?

Lists do not support random access and access is linear, so to get nth element you need n steps. In other words the Enum.at/2 for list will be implemented as:

def at([], _), do: nil
def at([val | rest], 0), do: val
def at([_ | rest], n) when n > 0, do: at(rest, n - 1)