Scholar.Preprocessing.one_hot_encode does not work?

Questions & Help Questions
,
2

Started to do my own investigation of this. Looks like the issue is with the Nx.sort call here: scholar/lib/scholar/preprocessing/ordinal_encoder.ex at 8712e96189983e56f32050ba91874e115ed70e1d · elixir-nx/scholar · GitHub

It doesn’t properly work for scalar tensors:

Nx.sort(Nx.tensor(5))

Returns the same error:

** (ArgumentError) given axis (0) invalid for shape with rank 0
    (nx 0.7.3) lib/nx/shape.ex:1121: Nx.Shape.normalize_axis/4
    (nx 0.7.3) lib/nx.ex:14975: anonymous fn/3 in Nx.sort/2
    (nx 0.7.3) lib/nx.ex:5368: Nx.apply_vectorized/2
    #cell:wjcoh5dp2qo2qlha:4: (file)

So this seems like a bug (or feature?) in Nx itself.

However, Nx.sort does work with 1D tensors, and doesn’t error out. So attempting the original with a 1D tensor, I get a different error.

Scholar.Preprocessing.one_hot_encode(
    Nx.tensor([5]),
    num_classes: 27
)

** (ArgumentError) index -2 is out of bounds for axis 0 in shape {1}
    (nx 0.7.3) lib/nx/tensor.ex:196: Nx.Tensor.normalize_index/3
    (nx 0.7.3) lib/nx/tensor.ex:145: Nx.Tensor.fetch_axes/7
    (nx 0.7.3) lib/nx/tensor.ex:92: Nx.Tensor.fetch_axes/2
    (nx 0.7.3) lib/nx/tensor.ex:56: Nx.Tensor.fetch/2
    (elixir 1.17.1) lib/access.ex:322: Access.get/3
    (scholar 0.3.1) lib/scholar/preprocessing/ordinal_encoder.ex:59: Scholar.Preprocessing.OrdinalEncoder."__defn:fit_n__"/2
    (nx 0.7.3) lib/nx/defn/compiler.ex:218: Nx.Defn.Compiler.__remote__/4
    #cell:wjcoh5dp2qo2qlha:2: (file)

So there’s probably 2 different issues going on here. :slight_smile: I’m going to continue digging to see if I can find a resolution to this problem.