Get AI code generation tools to create correct Elixir code, or else

They… Really aren’t. After two weeks of running Copilot on a JS codebase

  • it managed to suggest autocompletion maybe 5% of the time
  • maybe another 5% of the time it was actually useful or correct

All other cases IDEA’s autocompletion was better, and actually correct.

Both Copilot and ChatGPT are barely competent for the smallest and easiest of use cases. For the absolute vast majority of cases it’s a junior developer misunderstanding the assignment and spitting out some code that you have to thoroughly review to catch all the mistakes.

Don’t mistake raving reviews on Youtube/Twitter that are result of an early honeymoon period for actual usability of the products.

9 Likes