@evadne Glad you figured it out ![]()
FWIW I couldn’t get Polars to (which Explorer uses under the hood) to parse the timestamp directly either. So your cast approach is what I’d use too. I will throw a few more pointers out there.
DF.mutate/2 is nice because you don’t need to keep around the original column:
require Explorer.DataFrame, as: DF
path = "./ingestion_test.csv"
path
|> DF.from_csv!(dtypes: %{datetime_ns: :u64})
|> DF.mutate(datetime_ns: cast(datetime_ns, {:naive_datetime, :nanosecond}))
# #Explorer.DataFrame<
# Polars[1 x 1]
# datetime_ns naive_datetime[ns] [2025-02-03 09:03:33.286629]
# >
There is also the :lazy option. It will defer the computation so that you never create the intermediate column in the first place:
path
|> DF.from_csv!(dtypes: %{datetime_ns: :u64}, lazy: true)
|> DF.mutate(datetime_ns: cast(datetime_ns, {:naive_datetime, :nanosecond}))
|> DF.collect()
# #Explorer.DataFrame<
# Polars[1 x 1]
# datetime_ns naive_datetime[ns] [2025-02-03 09:03:33.286629]
# >
That option is only needed if you need to go real fast for some reason.
Also, we do have support for non-naive datetimes. But that doesn’t seem super relevant for this use case.






















