Sunday, 23 June 2013

Minimizing Finite Automata

Consider the finite automaton shown in figure 1 which accepts the regular set denoted by the regular expression (aa + b)*ab(bb)*. Accepting states are colored yellow while rejecting states are blue.

Figure 1 - Recognizer for (aa + b)*ab(bb)*

Closer examination reveals that states s2 and s7 are really the same since they are both accepting states and both go to s6 under the input b and both go to s3 under an a. So, why not merge them and form a smaller machine? In the same manner, we could argue for merging states s0 and s5. Merging states like this should produce a smaller automaton that accomplishes exactly the same task as our original one.
From these observations, it seems that the key to making finite automata smaller is to recognize and merge equivalent states. To do this, we must agree upon the definition of equivalent states. Here is one formulation of what Moore [2] defined as indistinguishable states.
Definition. Two states in a finite automaton M are equivalent if and only if for every string x, if M is started in either state with x as input, it either accepts in both cases or rejects in both cases.
Another way to say this is that the machine does the same thing when started in either state. This is especially necessary when finite automata produce output.
Two questions remain. First, how does one find equivalent states, and then, exactly how valuable is this information? We shall answer the second question first by providing a corollary to a famous theorem proven long ago by Myhill [3] and Nerode [4].
Corollary. For a deterministic finite automaton M, the minimum number of states in any equivalent deterministic finite automaton is the same as the number of equivalence classes of M's states.
With one more observation, we shall be able to present an algorithm for transforming an automaton into its smallest equivalent machine.
Fact. Equivalent states go to equivalent states under all inputs.
Now we know that if we can find the equivalence classes (or groups of equivalent states) for an automaton, then we can use these as the states of the smallest equivalent machine. The machine shown in figure 1 will be used as an example for the intuitive discussion that follows.

Let us first divide the machine's states into two groups: accepting and rejecting states. These groups are: A = {s2, s7} and B = {s0, s1, s3, s4, s5, s6}. Note that these are equivalent under the empty string as input.
Then, let us find out if the states in these groups go to the same group under inputs a and b. As we noted at the beginning of this discussion, the states of group A both go to states in group B under both inputs. Things are different for the states of group B. The following table shows the result of applying the inputs to these states. (For example, the input a leads from s1 to s5 in group B and input b leads to to s2 in group A.)
in state:
a leads to:
b leads to:
Looking at the table we find that the input b helps us distinguish between two of the states (s1 and s6) and the rest of the states in the group since it leads to group A for these two instead of group B. Thus the states in the set {s0, s3, s4, s5} cannot be equivalent to those in the set {s1, s6} and we must partition B into two groups. Now we have the groups:
A = {s2, s7}, B = { s0, s3, s4, s5}, C = { s1, s6}
and the next examination of where the inputs lead shows us that s3 is not equivalent to the rest of group B. We must partition again.
Continuing this process until we cannot distinguish between the states in any group by employing our input tests, we end up with the groups:

A = {s2, s7}, B = {s0, s4, s5}, C = {s1}, D = {s3}, E = { s6}.
In view of the above theoretical definitions and results, it is easy to argue that all of the states in each group are equivalent because they all go to the same groups under the inputs a and b. Thus in the sense of Moore the states in each group are truly indistinguishable. We also can claim that due to the corollary to the Myhill-Nerode theorem, any automaton that accepts (aa + b)*ab(bb)* must have at least five states.
Building the minimum state finite automaton is now rather straightforward. We merely use the equivalence classes (our groups) as states and provide the proper transitions. This gives us the finite automaton pictured in figure 2.

Figure 2 - A Minimal Automaton
Here is the state minimization algorithm.

The complexity of this algorithm is O(n2) since we check all of the states each time we execute the repeat loop and might have to execute the loop n times since it might take an input of length n to distinguish between two states. A faster algorithm was later developed by Hopcroft [1].
  • J. E. Hopcroft, "An nlogn algorithm for minimizing the states in a finite automaton." in The Theory of Machines and Computations, ed. Z Kohave. New York, Academic Press, 1971.
  • E. F. Moore, "Gedanken-experiments on sequential Machines." in Automata Studies, ed. C. E. Shannon and J. McCarthy. Princeton University Press, 1956, 129 - 153.
  • J. Myhill, "Finite automata and the representation of events." WADD TR-57-624, Wright Patterson AFB, Ohio, 1957.
  • A. Nerode, "Linear automaton transformations." Proceedings of the American Mathematical Society, 9, 1958, 541 - 544.