Hahaha, never mind, there was an old cache but it was because I am using gitlab review environments which after a git push automatically run. So what was happening that the server for some reason always was the first to be able to get the job and send out the email and not my dev laptop. Leaving this as it might save someone else from wasting two hours trying to figure out what was causing this :-/
