They are quite useful as rubber duckies though, and for learning. Or for thinking through a problem, such as software architecture, or refactoring.
Concrete example: Breek.gr is built with PHP (backend) and NextJS (frontend). My skill in both parts of the stack was below-par (or rusty, for PHP) last September. After I picked up the frontend and backend codebases in October, I used ChatGPT to gradually explain general concepts of NextJS based on every TSX file I was touching, and so I gradually upskilled to “mid to OK”. Way easier than wading through the NextJS docs or using Google Search. I now use Grok a lot for new feature development in NextJS, but I don’t let it code from scratch.
After scaffolding the TSX, I gradually prompt Grok to add this or change that. Way more often than not, it works. It’s hopeless to give it the full target of what a complex page should do and expect it to spew out code that you can simply copy and paste into the IDE, because in a complex app there are too many interactions with other custom hooks etc. for any LLM to take into account. It’s great for refactoring some TSX files originally written by a junior dev in JSX and then “minimally converted” to TSX by using “any” etc.; so, I also use Grok to save time when defining interfaces.
Grok is passable to good in Elixir nowadays. However, I enjoy coding in Elixir, so I don’t use it much for that purpose. After all, half the fun of developing apps in Elixir is the “in Elixir” part! Or, perhaps I’m in this phase of my Elixir journey (and growing frustration with the complexity of NextJS) where every Elixir module I write feels like a breath of fresh air and an accomplishment.
With LLMs, my 2023 hunch remains valid: you need to understand enough about the subject matter to know when you’re being given BS. And you need to know enough about the codebase to provide guidance. Most of the context is in the user’s head, the rest is provided by the user to the chatbot. You need to do a lot of hand-holding, but in the process you also understand better, and sometimes redefine the problem; sometimes, it’s an “XY problem”.
My experience with LLMs so far has been very positive. It’s a boon both to development velocity (isn’t that what the Agilists call it?) and to my understanding of two pieces of the stack that I have no intention of building expertise in. With human QA (i.e., me) we end up with robust and performant code on both backend and frontend. For $30 per month, SuperGrok is a bargain.
