Very interesting, thanks for sharing!
Anyway, I think I can’t use it in my case. The whole stuff is going on in runtime (I am evaluating previously saved AST). Also, assert_raise is working correctly in this case, function raises CompileError as expected. The only small issue is this warning, which I can live with, but it is annoying 
