Thank you very much for your help and guidance. You helped a lot 
It finally helped me to find out where I went wrong. Basically, I should have read the documentation better 
The thing is, I thought that socket.assigns.invoice reflected the state of the form and I started to update it. I rewrote the code and now I update the params fields from the form and compile the changeset that way. So now it works as I had intended.
Like here:
@impl true
def handle_event("validate", %{"_target" => ["invoice", "date"], "invoice" => invoice_params}, socket) do
%{"date" => date, "payment_term" => payment_term} = invoice_params
invoice_params =
invoice_params
|> Map.put("due_date", Date.add(Date.from_iso8601!(date), String.to_integer(payment_term)))
|> Map.put("date", Date.from_iso8601!(date))
changeset =
socket.assigns.invoice
|> Invoices.change_invoice(invoice_params)
|> Ecto.Changeset.put_assoc(:invoice_lines, socket.assigns.invoice.invoice_lines)
|> Map.put(:action, :validate)
socket =
socket
|> assign(:changeset, changeset)
{:noreply, socket}
end
