# How to use yacc to test for ambiguity (Compilers)

There is no algorithm that decides definitively whether a given context free grammar is ambiguous, but that does not prevent us from using yacc pragmatically. If we submit a grammar (unannotated with precedence declarations) to yacc, and yacc accepts it, then we know the grammar is in the class LALR(1), and all such grammars are unambiguous. If yacc says No and reports parsing conflicts, then either the grammar is ambiguous, or it is unambiguous, but "too difficult" for yacc to handle. To find out which, we can look at yacc's diagnostic output and use it to find a situation where a conflict is encountered: that can help us to find a string that can be derived in more than one way.

Consider, for example, the following interesting grammar provided by Matteo Cati. It purports to be an unambiguous grammar for the classic Algol--60 dangling else problem.

stmt : inner | IF expr THEN stmt inner : BASIC | IF expr THEN inner ELSE stmt

In order to submit this to yacc, we must surround it with some apparatus declaring various tokens, and supply semantic actions, all of which can be written as the trivial unit expression `{()}`

. A complete input is shown towards the end of this note. Invoking `ocamlyacc`

with

$ocamlyacc -v cati.mly

produces the error message

1 shift/reduce conflict

and also (because of the `-v`

flag) a report file `cati.output`

that gives details of parsing automaton `ocamlyacc`

compiled, and where it discovered the conflict. In what follows, I've used (because why wouldn't I?) the output of my own yacc variant, but the output of `ocamlyacc`

is equivalent, even if the states are numbered differently. The whole of the report file is shown at the end of this note.

Examining the report, we see the conflict occurs in state 11:

State 11: shift/reduce conflict (shift 12, reduce 1) on ELSE State 11 inner --> IF expr THEN inner . ELSE stmt (4) stmt --> inner . (1) EOF reduce 1 ELSE shift 12

If the machine reaches state 11 and the lookahead is `ELSE`

, then the machine cannot decide between shifting the `ELSE`

(in the hope of reducing with the production `inner --> IF expr THEN inner ELSE stmt`

), or reducing immediately with the production `stmt --> inner`

(presumably so that it can then reduce with `stmt -> IF expr THEN stmt`

).

We can look for an ambiguous string by asking what sequence of moves in the machine can lead to state 11. Sadly, the simplifications from the full LR(1) process incorporated into yacc mean that the shortest such sequence does not yield an ambiguous string, but a longer sequence does.^{[1]} Consider the following string of tokens and variables:

IF expr THEN IF expr THEN inner ELSE IF expr THEN inner

Starting in state 2 (states 0 and 1 are present for housekeeping purposes, as is the magic token `*1*`

), we can trace out a trajectory in the machine as follows. In the report from yacc, state 2 contains the instruction

State 2: IF shift 4

And state 4 contains

State 4: expr goto 8

Continuing in this way, we can trace out the following path that ends at state 11, incidentally visiting state 11 along the way.

2 IF 4 expr 8 THEN 9 IF 4 expr 8 THEN 9 inner 11 ELSE 12 IF 4 expr 8 THEN 9 inner 11

The shorter sequence that stops after reaching state 11 for the first time also leaves the yacc-based parser unable to decide what to do next, but it does not allow both decisions to lead to a succesful parsing outcome.

Now, with the longer string, and with `ELSE BASIC`

as the remaining input, the conflict gives two possibilities: one option is to shift the `ELSE`

, in which case we can complete a parse by shifting the token `BASIC`

on top of it, then reducing in several stages as follows.

IF expr THEN IF expr THEN inner ELSE IF expr THEN inner | ELSE BASIC IF expr THEN IF expr THEN inner ELSE IF expr THEN inner ELSE | BASIC IF expr THEN IF expr THEN inner ELSE IF expr THEN inner ELSEBASIC| IF expr THEN IF expr THEN inner ELSE IF expr THEN inner ELSEinner| IF expr THEN IF expr THEN inner ELSEIF expr THEN inner ELSE stmt| IF expr THEN IF expr THEN inner ELSEinner| IF expr THENIF expr THEN inner ELSE stmt| IF expr THENinner|IF expr THEN stmt| stmt |

The other option is immediately to reduce by `stmt --> inner`

, beginning a sequence that goes like this:

IF expr THEN IF expr THEN inner ELSE IF expr THENinner| ELSE BASIC IF expr THEN IF expr THEN inner ELSEIF expr THEN stmt| ELSE BASIC IF expr THENIF expr THEN inner ELSE stmt| ELSE BASIC IF expr THEN inner | ELSE BASIC IF expr THEN inner ELSE | BASIC IF expr THEN inner ELSEBASIC| IF expr THEN inner ELSEinner|IF expr THEN inner ELSE stmt|inner| stmt |

The string

IF expr THEN IF expr THEN inner ELSE IF expr THEN inner ELSE BASIC

is not a string of tokens, but if `X`

is an expression, the sequence

IF X THEN IF X THEN BASIC ELSE IF X THEN BASIC ELSE BASIC

leads, with some reduction along the way, to the same configuration, and is therefore a witness that the grammar is ambiguous.

## Ocamlyacc input

%token BASIC X IF THEN ELSE %type<unit> stmt %start stmt %% stmt : inner {()} | IF expr THEN stmt {()} ; inner : BASIC {()} | IF expr THEN inner ELSE stmt {()} ; expr : X {()} ;

## Report file

This is the contents of the file `cati.output`

that is produced by Yacc when given the `-v`

flag. Rules 1 to 5 represent the input grammar; rule 6 is added by Yacc with a semantic action that makes the parser return the AST for the statement it has parsed, and rule 0 is added in order to establish the end-of-file token `EOF`

as a lookahead expected at the end of the input. A Yacc parser can have several start symbols, each giving a parser for part of the language that can be invoked from the main program; the magic token `*1*`

tells the parser which entry point has been chosen, and there would be other magic tokens `*2*`

, etc., and multiple variations on rule 6, in a parser with multiple entry points.

Following the summary of the rules appears a list of states in the LR(0) automaton compiled by Yacc. The administrative housekeeping just mentioned means that the real work of parsing starts in state 2. Each state has a number of LR(0) *items* (production rules with a marked place in the RHS, denoted by a dot). It also has a list of actions, each labelled with a lookahead token, and either `shift `

for some state *s**s*, or `reduce `

for a numbered rule *r**r*; for some states there is a default reduction rule labelled with a dot. States can also have `goto`

entries labelled with a nonterminal, and with the `shift`

entries, these are used to compute a trajectory through the state space in the way explained above.

Rules: 0: *start* --> *entry* EOF 1: stmt --> inner 2: stmt --> IF expr THEN stmt 3: inner --> BASIC 4: inner --> IF expr THEN inner ELSE stmt 5: expr --> X 6: *entry* --> *1* stmt State 0 *start* --> . *entry* EOF (0) *1* shift 2 *entry* goto 1 State 1 *start* --> *entry* . EOF (0) State 2 *entry* --> *1* . stmt (6) BASIC shift 3 IF shift 4 stmt goto 5 inner goto 6 State 3 inner --> BASIC . (3) . reduce 3 State 4 stmt --> IF . expr THEN stmt (2) inner --> IF . expr THEN inner ELSE stmt (4) X shift 7 expr goto 8 State 5 *entry* --> *1* stmt . (6) . reduce 6 State 6 stmt --> inner . (1) . reduce 1 State 7 expr --> X . (5) . reduce 5 State 8 stmt --> IF expr . THEN stmt (2) inner --> IF expr . THEN inner ELSE stmt (4) THEN shift 9 State 9 stmt --> IF expr THEN . stmt (2) inner --> IF expr THEN . inner ELSE stmt (4) BASIC shift 3 IF shift 4 stmt goto 10 inner goto 11 State 10 stmt --> IF expr THEN stmt . (2) . reduce 2 State 11: shift/reduce conflict (shift 12, reduce 1) on ELSE State 11 inner --> IF expr THEN inner . ELSE stmt (4) stmt --> inner . (1) EOF reduce 1 ELSE shift 12 State 12 inner --> IF expr THEN inner ELSE . stmt (4) BASIC shift 3 IF shift 4 stmt goto 13 inner goto 6 State 13 inner --> IF expr THEN inner ELSE stmt . (4) . reduce 4

- ↑ The longer sequence can be found immediately by using an LR(1) parser generator, such as the one at http://jsmachines.sourceforge.net/machines/lr1.html . Here is a suitable grammar:
S' -> S S -> I S -> if E then S I -> b I -> if E then I else S E -> x

The analysis reveals a parsing conflict, and the shortest string that leads to the state with a conflict is the one shown as an ambiguous input above.