Ah that makes sense. Ortex embeds/second by sequence length:
This also achieves full CPU utilization on my MacBook in comparison to EXLA 
So it seems like Ortex does not benefit that much from using Nx.Serving, as the model does not need to be precompiled to certain input shapes, in contrast to when using EXLA, is that right?
