Thanks for thinking it through.
It’s possible that I am missing something in what you’re saying — I feel I am quite close to the problem and that has its own challenges but it seems to me that the focus on the zero-or-more semantics of many here is possibly a misdirect.
If we imagine a parser like:
program = many(elixir_statement)
elxir_statement = choice([case_expr, for_expr, def_expr, with_expr, …])
case_expr = sequence([literal("case"), expr, literal("do"), …, literal("end")])
# and so on
if the input is
case x# do … end
then the case_expr will fail but it’s not the case that for_expr, with_expr, def_expr or anything could match. From the moment it parsed "case " there was no other valid option and something is wrong with the input.
So the case_expr fails, causing the choice to fail and that rolls up into the many which expects its child parser to fail at some point because it treats that as “end of sequence” and then it returns success.
My problem is how to differentiate between the child parser of many failing in the expected “end of sequence” case or the unexpected “malformed choice” case because the error is different in each.
It seems Saša & co. didn’t attempt to solve this but rather worked around it through writing parsers differently.
Looking at your suggestion it seems to me we don’t want to attempt char(?d) because the input is invalid and char(?d) cannot match. It might be that some of these errors could be recovered to continue parsing (in my example of a # in an expression we could clearly attempt to continue parsing since the structure is sound but in other cases, it may not be). What I am trying to do is report the right error (i.e. we need to bubble an error out of many when that’s not its typical behaviour).
Thanks.
Matt
