Interaction Design Section   Living Section   About Bruce Tognazzini

The Butterfly Ballot: Anatomy of a Disaster

About the Author: I'm neither a Democrat nor a Republican. I'm an Independent, with strong Libertarian leanings. (I'd probably be a Libertarian, but the Libertarians are just too organized.) I have written what follows strictly so we can learn from the mistakes that caused the recent meltdown in the American election process.

The man who lost the popular vote both country-wide and in the state of Florida was recently elected President of the United States because a ballot in Palm Beach, Florida, was printed in such a way that when people attempted to vote for the Democratic candidate, Gore, many ended up voting for the conservative candidate, Buchanan, or for both Gore and Buchanan.

That the voters of America who cast their ballots for Gore feel hurt is understandable, but George Bush's supporters should not take too much comfort in their candidate's windfall. Unless swift and significant changes are made, the same thing could happen to them. In fact, in one of the many curious twists of this peculiar happening, four years ago it did.

I have used "disaster" in the title of this piece, not because having George Bush as president is in any way a disaster for the country, but because having elections decided by a handful of judges is not the way the American democracy was designed to work. ("My conservatives judges can beat up your liberal judges!") Had Al Gore eventually carried the day, the process would have been an equal disaster. The only thing different would be the half of the country distressed by it.

Disasters Don't "Just Happen"

One of the first things I learned in flying was that airplanes don't just fall out of the sky. (This, by the way, comes as welcome news to the student pilot.) Airplane accidents can almost always be traced back to a series of mistakes and events: The night was dark AND visibility was limited by fog AND the area being flown over was water, lacking in reference lights, AND John Kennedy, Jr. had only a few scant hours training in instrument flying AND he was flying illegally based on his visual flight rules license, perhaps causing him to hesitate before asking for help AND.... An equally elaborate series of critical mistakes and conditions caused singer John Denver's aircraft death.

The investigation of the near-meltdown at Pennsylvania's Three Mile Island nuclear power plant in March, 1979, listed a remarkable series of seemingly disconnected events that converged to almost cause a major disaster. Along with other causes were such human factors as lack of operator training and a confusing and misleading control layout. As with many disasters, the series of events that arose from a single trigger could have been short-circuited at any point, had anything gone right.

The Challenger Space Shuttle blew up because of O-Rings that failed in the face of excessive cold, but that's not the whole story as to what killed those seven American astronauts. What killed them was taking off after an all-night engineering meeting concluded that the O-Rings were OK. Charts displayed at the meeting demonstrated that a launch under current conditions could not help but kill all aboard. Why did they take off anyway? Because the design of the charts was so bad that no one could see this critical information. A redesign by Edward Tufte, after the fact, showed all.

The Elements of the Florida Ballot Disaster

The press has identified the problem in Palm Beach as being a badly-designed ballot. That specific design was certainly the most important ingredient in what occurred, but, like most disasters, other elements contributed their part.

•The Punchcard Voting System

The punchcard voting system was first used in the 1964 Presidential primary election, quickly gaining popularity.

Once the system came into widespread use, it became apparent that it had a most unusual characteristic: Every time the cards were fed through the reader, the reader came up with a different answer. The reason was simple: The now-infamous "chad," the bits of paper—the holes in the doughnut—that are supposed to be punched out by the user. Often, they are not.

Could the Registrar of Voters in Palm Beach have known of this system's problems? She should have. Even the simplest of web searches reveals that, as far back as 1988, the National Institute of Standards and Technology (NIST) strongly recommended the elimination of Votomatic-type cards just because of this chad problem.

Unfortunately, their recommendation came with no enforcement power, so the chad chugged on. Even as late as 1996, 37.3% of the registered voters in the United States were still casting their ballot using punched cards.

When the ballot disaster happened in Palm Beach, it was this chad with which the Democrat's hopes lay. They could do nothing to fix the ballot that had caused so many thousands of voters to double-punch the ballots, but they could perhaps get an accurate count of the vote that had been legitimately made, and they rightfully expected such a recount to find their candidate victorious. Why? Because the counties that used the punch card system were predominantly Democratic. Since most people had voted for Gore, it stood to reason that most improperly-punched ballots would end up being for their candidate.

With the country long-since on notice that the punchcard systems were inherently defective, Palm Beach, along with many other counties in America, continued to use them. Why? Because nothing had ever gone seriously wrong. Yes, there had been a smattering of contested elections brought about by the machines, in St. Louis, Atlanta, Los Angeles, but nothing yet that would alter the course of history. No reason for alarm.

•The Low-Income Paradox

Actually, a quiet change has been going on for some time. In fact, by the early 1990s, wealthy counties across America were pulling out the punchcard system with all deliberate speed. The poorer counties just felt they didn't have the funds.

Meanwhile, it was becoming apparent that the punchcard system had another key characteristic: The higher your socioeconomic class, the less likely you were to make an error using the punchcard system. Therefore, we were soon requiring punchcard ballots only in those areas where people were the least able to use them. Among those disadvantaged areas: Palm Beach County, Florida. It is very likely the Registrar of Voters wanted to change the machines, but where was the money?

•Design of the Palm Beach Ballot

The chad problem alone would not have tipped the balance in the 2000 election, since thousands upon thousands more voters went to the poles to vote for Al Gore in Florida. It took an atrociously bad design to really foul things up.

Not only does it appear that perhaps 4.000 people made the error of punching the second hole on the ballet in the mistaken belief that the second hole represented the second candidate, more than 19,000 people made the error of punching more than one hole, since both were directly alongside their candidate.

This is yet another disgraceful example of what happens when you don't bother to user test.

Why were the people who laid this out unable to see the problem, even without testing? Because they were not users, they were designers. As such, they were interested in all 10 candidates on the ballet (plus space for a write in), and they saw all ten candidates. They viewed the ballot as a 2X6 staggered matrix with a line of radio buttons in between the two sides. Their cute little arrows appeared to be enough to help people choose the right box from this matrix.

The voters saw things very differently. They were not interested in 10 candidates. They were interested in one candidate, the one they wanted to vote for. Their entire focus was on finding that candidate and punching the hole next to his or her name. In the case of Gore, that required scanning only two names down in the first column. There was never any reason at all for Gore voters to ever even see the right hand column, and we now have thousands of pieces of evidence that they, indeed, didn't. Rather than a staggered matrix, they saw a single column with a dedicated column of radio buttons adjoining.

So here I come doing a little Monday morning quarterbacking. Sure, it's easy enough to see now what a stupid design it was, now that they game is over.

That's what user testing is for. So you can play the game before you play the game. A competent designer, in my opinion, would have predicted this outcome just from a glance at the ballot, but even a beginner should have had enough sense to user test the design before release. That rather obviously was not done in this case, and a serious miscarriage of thousands of voter's wishes resulted.

(I've been attacked for using the term, "designer," to refer to the people involved in the layout of this ballot. I've seen no information on who actually laid the thing out. As a designer myself, I would hope this person was not a member of our noble profession. I use the term only because, whoever it was, they were acting in the role of designer.)

•The Republican Experience—Another Warning

Still and all, perhaps Palm Beach is not to be faulted for fouling up. After all, they had never had such a problem before. Or had they?

Yes, they had. In the 1996 election, the exact same problem occurred, for the exact same reason. In that instance, the Republican candidate, Robert Dole, had 14,000 ballots tossed out because of double-punching. Why the Republican in that election? Because Dole was the second name on the ballot, just as was Al Gore this time around.

Exactly how many massive failures of this kind does it take to break through the denial of the powers-that-be in Palm Beach, Florida?

•Jeb Bush Elected Governor

If Jeb Bush had lost the 1998 gubernatorial election, Al Gore would have been elected president. Why? Because candidate of the party of the incumbent governor appears first on the ballot. In 1996, Democrat Lawton Chiles was in the Governor's Mansion, and so Republican Dole got all the ruined ballots. With Jeb Bush as Governor, Al Gore found himself in second place on the ballot, and second in the count.

Disasters have these strange little complicating factors. It's one of the things that make them so interesting.

•Ensuring Disaster

Even if nothing had been done about the ballot itself, much of the effects of the design could have been mitigated. Voters could have been shown the problem with the ballot. The tiny arrow could have been pointed out and it could have been explained that it may not actually point right at the hole, that you've kinda got to slide the ballot around a bit. They could have been told "no double-punching!" and, of course, told how to get another ballot if they made a mistake. They could have been properly trained in how to remove hanging chad, as has been done in many areas around the country.

Unfortunately, all of this would have required training the precinct workers, and it appears that training was in short supply in Palm Beach County.

The elements were all in place. All it took was the predicted close election to set the disaster in motion.

What Needs to be Done about Election Ballots

We need:

•A uniform voting standard across America

People are mobile. They should not have to learn the idiosyncrasies of different systems in different places.

•Accurate machines

•Maintenance of privacy

•Means for post-election verification

What happened in Florida is nothing compared to what will happen when someone "hacks" the election. We will likely need to maintain a paper record of each voting session, even if the actual voting is carried out on a machine.

•Professional layout of individual county ballots

No one evidencing the lack of design ability seen in the Palm Beach ballot should be allowed within 20 feet of a ballot design. Designers should be required to prove competence and experience in forms layout and interaction design. No exceptions.

•User testing

How much user testing would have been needed in Palm Beach? It appears that one out of ten users have some problem with this design, with approximately one in one hundred failing completely. A professional usability tester running twenty subjects in a 10 minute test each would have amply shown that something was seriously amiss.

Would such a test be cost-effective? Certainly in Florida it would have. What happened there cost millions. Even in a plain old election, the benefits are manifold. Studies show that most users don't make mistakes when confronted with bad interfaces, they just slow way down. (The Florida results bear this out. One in ten people have trouble with the ballot, but only one in one hundred end up making an error.)

When people slow way down, you need more machines and more workers. That costs money, a lot more than a user test. A lot more.

The Lessons for Those of Us on the Web

Most human-machine interaction problems don't rise to the gravity of altering the outcome of a national election, but most share identical characteristics. The only thing unusual about what happened in Palm Beach is the shear magnitude of the result. These kinds of interaction meltdowns go on all the time. They are never just the simple result of a really bad design. One can, in looking over the historical record, find warning after warning that was ignored before the final crisis occurred.

Compuserve had an abysmal interface, much beloved by the early adopters of networking. The company knew it was abysmal, but they also knew they could make more money "this next quarter" by not fixing it. Then one day, AOL burst on the scene. Compuserve's users realized for the first time what they had been missing and quickly the company slipped into a steep and fatal decline, bowing out of the content market.

Horrible websites, built by amateurs, appear every day. The companies involved maintain a blissful ignorance throughout the development process, often turning away from the most glaring evidence that something is seriously amiss. In almost every case, someone has made a conscious decision early-on that the company could not afford the expense of user testing.

The penalty for this decision is death. The Web is no longer an amateur playground. Sites such as Amazon and Ebay have set people's expectations for how the web should work. If you try to feed them an incompetent design, they will turn away and you will be out of business.

If you hire an outside firm to do your website, ask for their user-testing plan. If they don't have one, run, do not walk, to another firm.

I make my living as a consultant, helping companies of all sorts set up the processes and procedures that result in successful design. I've seen the kind of transformation that iterative design methodology (design-test-design-test) can bring. It costs little in money and typically shortens time to market.

If you do not use iterative design as standard operating procedure for your in-house work, start doing so. If you don't know how, find out how. It is not that difficult. Much easier than looking for another job, and that is quickly becoming the alternative.

Don't miss the next action-packed column! Receive a brief notice when new columns are posted by sending a blank email to asktoglist-subscribe@yahoogroups.com. return to top

Contact Us:  Bruce Tognazzini   Copyright Bruce Tognazzini.  All Rights Reserved