If you didn’t work with Claude 3 Opus or 3.5 Sonnet, it’s definitely worth a try. GPT-4o is somewhere in between. o1 is supposedly better, but I haven’t worked with it.
So were you extracting the EEP metadata from the .beam file and passing it in as text? If it provided enough information that an ordinary human programmer could write code to it, so probably can Claude 3.5 Sonnet, and others at that level.
My difficulty with documentation so far has been size. You really want your entire code + message history to fit in the context window, and some docs can be quite large.
For example, I tried experimenting with the OpenAI OpenAPI spec to see if I could use an LLM to generate the openai_ex library from the spec. Turned out to be too large for the context window (from what I recall, the spec alone was too large). I toyed with extracting one API endpoint at a time from the full spec and generating the corresponding code. I couldn’t quite manage to do a clean extraction with all the information, even with LLM assistance. I eventually gave up because it was too much work for such a simple API. It might be worth trying out in a higher value situation.
