We have encountered the same issue when developing an SQL parser. Basically, my takeaway was that parsers such as many and choice are mostly useless if we want to produce a precise error report. Perhaps there is a way to make it work, I’m not super familiar with various parsing techniques, so I’ll explain what we did instead.
We had multiple variations of the problem you describe here, e.g. when parsing arithmetic expressions, or handling joins and subqueries. Our strategy was to abandon many and choice. Instead, we would “commit” to a parsing path based on the next token. In your example grammar, if the next token is b, we’ll commit to parsing the pair bc. If that fails, we’ll emit an error. If that succeeds, we’ll try to parse another bc if the next token is b. I believe that this technique is called “lookahead”.
To make it work, we introduced two additional parsers. The most important one is called switch. Let’s see an example:
switch([
{char(?b), char(?c)},
{char(?d), char(?e)},
])
This will parse bc or de, returning a proper error if the second char is invalid. Basically the switch parser commits to the given “branch” if the first parser in some tuple succeeds. If the rest of that branch errors, that error is returned by the switch parser. If no branch can be selected, the parser will return a generic error (similar to choice). If the parser succeeds, it will emit {result_of_the_first_parser, result_of_the_second_parser}
Now in your example we don’t want to return an error if no branch can be selected. Instead, we want to just stop parsing. To make this happen, we added the support for the :else clause, and introduced another parser called noop which always succeeds consuming nothing from the input. So parsing bc or nothing can be now expressed as:
switch([
{char(?b), char(?c)},
{:else, noop()}
])
And now, we can parse zero or more bc pairs as:
defp bc_pairs do
switch([
{char(?b), sequence([char(?c), lazy(&bc_pairs/0)])},
{:else, noop()}
])
end
The emitted term will be a bit weird, something like:
{?b, [?c, {?b, [?c, nil]}]}
The final nil is emitted by the noop parser. This can be transformed into a list (e.g. ["bc", "bc", ...]) with the map parser.
I’m not sure if there’s a more elegant way, but this is what we did, and it worked fine. The code was definitely more complicated than the naive approach with many, but it was still understandable. Most importantly, our error reports were much more precise and informative.
For completness, we also wrote a parser called choice_deepest_error, which would work like a choice (aka one_of):
choice_deepest_error([
parser1(),
parser2(),
...
])
This parser emits either the result of parser1, parser2, etc. If no parser succeeds, it returns the error from the parser that consumed most of the input. I personally wasn’t a fan of this approach, because I found it hard to reason about. The “deepest error” isn’t necessarily the correct error. So personally I prefer the lookahead technique.
