GPT does better when it’s given more context as input. Right now, the stacktrace_gpt code doesn’t supply anything besides the stacktrace itself, which is rarely enough to understand the problem.
I’d recommend the same thing for GPT as I’d want when presented with a stacktrace: an answer for “what’s the code at / around the point where the failure happened?”
Several things that seem like they’ll make doing that challenging:
- the file may be too big for the LLM’s context window
- users may not want to upload proprietary source code to OpenAI’s servers
- the relevant line may not be the top one from the stacktrace, especially if it’s in stdlib code
