# Access behaviour for Explorer.DataFrame

**URL:** <https://forum.elixirforum.com/t/access-behaviour-for-explorer-dataframe/66691>\
**Category:** Questions\
**Tags:** learning-elixir, nx, explorer\
**Created:** [October 13, 2024, 1:19am UTC](https://forum.elixirforum.com/t/access-behaviour-for-explorer-dataframe/66691 "2024-10-13T01:19:55Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![mooreryan](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/mooreryan/32/35965_2.png) [@mooreryan](https://forum.elixirforum.com/u/mooreryan)\
**Post date:** [October 13, 2024, 1:19am UTC](https://forum.elixirforum.com/t/access-behaviour-for-explorer-dataframe/66691/1 "2024-10-13T01:19:55Z")

</div>

I have a couple of questions about Explorer’s `DataFrame` and the `Access` behaviour.

In the [selecting columns](https://hexdocs.pm/explorer/Explorer.DataFrame.html#module-selecting-columns-and-access) section of the Explorer manual, it mentions that the Access behaviour is implemented for DataFrames.

Because of this I would expect that accessing a column that doesn’t exist with the bracket notation to return `nil` and using `DataFrame.fetch` to try an access a column that doesn’t exist to return `:error`.

However, this is not the case. Here are some examples.

## Examples

```elixir
df =
  DataFrame.new(
    a: [1, 2, 3],
    b: [10, 20, 30]
  )

```

And here are some ways to access it.

### Using brackets

```elixir
iex> df["a"]
#Explorer.Series<
  Polars[3]
  s64 [1, 2, 3]

iex> df["c"]
** (ArgumentError) could not find column name "c". The available columns are: ["a", "b"].
If you are attempting to interpolate a value, use ^c.

```

Trying to access column `c` raises an `ArgumentError`, but I would expect that to return `nil` given the `Access` behaviour.

### Using `fetch`

Using `fetch` was also surprising:

```elixir
iex> Explorer.DataFrame.fetch(df, "a")
{:ok,
 #Explorer.Series<
   Polars[3]
   s64 [1, 2, 3]
 >}

iex> Explorer.DataFrame.fetch(df, "c")
** (ArgumentError) could not find column name "c". The available columns are: ["a", "b"].
If you are attempting to interpolate a value, use ^c.

```

The first makes sense (`{:ok, val}`) but the second, raises an `ArgumentError`, where I would expect it to return `:error` given the `Access` behaviour.

I assume that this is the intended way for it to work given that it is included in the test suite. See this [test](https://github.com/elixir-explorer/explorer/blob/ca2ca1f80e4cb994f3182e8d59e9608bad95385e/test/explorer/data_frame_test.exs#L2612) for example, which shows that an `ArgumentError` is expected to be raised.

(The fact that it is an `ArgumentError` is also interesting given that [`fetch!/2`](https://hexdocs.pm/elixir/Access.html#fetch!/2) from Access raises a `KeyError` exception rather than `ArgumentError`.)

### Other Access behaviour functions

Another interesting thing is that not all of the functions in the `Access` behaviour are available. E.g.,

```elixir
iex> Explorer.DataFrame.fetch!(df, "c")
** (UndefinedFunctionError) function Explorer.DataFrame.fetch!/2 is undefined or private.

```

## Summary

To summarize, here are my questions:

- Why does accessing columns in a `DataFrame` not behave in the way that the `Access` behaviour docs imply that it should behave?
- Why aren’t all the `Access` behaviour functions available on `DataFrame`?
- How are you supposed to check if a column exists in a `DataFrame` other than using a `try` block?

---

<div class="post-metadata">

**Author:** ![josevalim](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/josevalim/32/1787_2.png) [@josevalim](https://forum.elixirforum.com/u/josevalim)\
**Post date:** [October 13, 2024, 2:44pm UTC](https://forum.elixirforum.com/t/access-behaviour-for-explorer-dataframe/66691/2 "2024-10-13T14:44:22Z")

</div>

> [@mooreryan](#):
>
> Why does accessing columns in a `DataFrame` not behave in the way that the `Access` behaviour docs imply that it should behave?

The Explorer team has decided that it makes more sense to raise (catching errors early) than return nil. How much this is violation of the Access behaviour is a good question. Here is what we can say:

1. Keyword lists are used to model optional keys, but they raise if the key is not an atom
2. Maps always return nil for missing keys, but they also have a convenience API for being strict, such as `map.foo`
3. Nx tensors raise if you use access with an invalid dimension

I’d have to put more thought into it but, in the face of the two conflicting positions below, I’d probably stick with the first one:

- We should let data structures decide what is best for them
- Access should impose it to return `nil`

> [@mooreryan](#):
>
> Why aren’t all the `Access` behaviour functions available on `DataFrame`?

The Access behaviour requires you only to implement `fetch` and `get_and_update`, the other functionality is made available through the `Access` module itself. It is similar to a GenServer, where you implement `handle_call` but you must invoke `GenServer.call`.

> [@mooreryan](#):
>
> How are you supposed to check if a column exists in a `DataFrame` other than using a `try` block?

We should add an API for it.

---

<div class="post-metadata">

**Author:** ![billylanchantin](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/billylanchantin/32/28428_2.png) [@billylanchantin](https://forum.elixirforum.com/u/billylanchantin)\
**Post date:** [October 13, 2024, 3:15pm UTC](https://forum.elixirforum.com/t/access-behaviour-for-explorer-dataframe/66691/3 "2024-10-13T15:15:58Z")

</div>

> [@mooreryan](#):
>
> How are you supposed to check if a column exists in a `DataFrame` other than using a `try` block?

You can also do:

```elixir
df = Explorer.DataFrame.new(a: [1, 2, 3])

if "a" in df.names do
  # ...
end

```

It’s not perfect because `Access` is more inclusive. For example:

```elixir
# Both of these work
df["a"] #=> #Explorer.Series<Polars[3] s64 [1, 2, 3]>
df[:a] #=> #Explorer.Series<Polars[3] s64 [1, 2, 3]>

# Only the string version works
"a" in df.names #=> true
:a in df.names #=> false

```

So it’s a bit of a leaky abstraction. But depending on what you’re doing it may be good enough.

---

<div class="post-metadata">

**Author:** ![mooreryan](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/mooreryan/32/35965_2.png) [@mooreryan](https://forum.elixirforum.com/u/mooreryan)\
**Post date:** [October 13, 2024, 4:15pm UTC](https://forum.elixirforum.com/t/access-behaviour-for-explorer-dataframe/66691/4 "2024-10-13T16:15:17Z")

</div>

Thanks for the responses josevalim and billylanchantin!!

> [@josevalim](#):
>
> The Explorer team has decided that it makes more sense to raise (catching errors early) than return nil.

That makes sense. I was not aware that the return types for a behaviour were flexible.

> [@josevalim](#):
>
> I’d have to put more thought into it but, in the face of the two conflicting positions below, I’d probably stick with the first one:
> 
> - We should let data structures decide what is best for them
> - Access should impose it to return `nil`

This is interesting. I’m new to Elixir, so I can’t really say how it _ought_ to be, but at least for me, the `DataFrame` implantation of `Access` being different from the `Access` docs was surprising. But that could be addressed by adding something to `DataFrame` docs.

> [@josevalim](#):
>
> the other functionality is made available through the `Access` module itself. It is similar to a GenServer, where you implement `handle_call` but you must invoke `GenServer.call`.

Thanks for the explanation. That makes sense.

> [@billylanchantin](#):
>
> You can also do:
> 
> ```elixir
> df = Explorer.DataFrame.new(a: [1, 2, 3])
> 
> if "a" in df.names do
> # ...
> end
> 
> ```

Thanks, I will go with something like this for now.
