How to apply a L1/L2 penalty to layer's output in Axon?

Questions & Help Questions
, ,
5

Hey @meanderingstream and @seanmor5 :wave:

Thank you so much for you quick and kind replies, really appreciated :bowing_man:

I would suggest consider training a model with different optimizers and hyperparameter choices versus trying to add a regularizer on an individual layer.

Yup, I wanted to replicate an example in Keras, but as you suggested, better to try with other optimizers :+1:

I think adding a similar API just brings too much coupling between model creation/execution and model training. You are better off using weight decay that’s available in the optimization API to achieve the same regularization affect.

I see, makes total sense, thanks for sharing the reason :+1:

I’ll try nevertheless to follow your suggestions and see if I can implement it, it is a good occasion to look into Axon internals :nerd_face:

Thank you everyone, I wish you a great day! :blush: