AskTog:

If They Don't Test, Don't Hire Them

The single best indicator as to the overall competence of an interaction design team is their plan for user testing. If you are presented with no plan or a sort of vague “and we’ll eventually do some user testing,” you may want to back off and look at other resources. If, on the other hand, you are given a proposal outlining repeated design and test cycles, you are dealing with people who know exactly what they are doing.

This focus on testing may seem paradoxical. After all, isn’t user-testing a sign of weakness? Can’t a good designer do a proper job without parading a lot of test subjects through the design? Programmers seem to be able to do their job without a lot of testing, except for maybe QA at the very end. People in related fields, such as architecture, seem to do their jobs without a lot of testing. What’s with these Prima Donna designers?

Programmers, in fact, constantly test their work. It’s called debugging. They code, they compile, they test. They code, they compile, they test. They find problems. Plenty of them. And they keep the iterative process of code-compile-test going until the bugs are gone or until ship date, whichever occurs first.

Architects test, too. They start with reviews with the client to see if their rough sketches are filling the client’s needs. They then “wring out” their designs by walking potential users of the building through scenarios. They do this multiple times, until the design begins to work. At least the good ones do. I heard tell many years ago of a much-exalted architect who left out this rather important step when called upon to build a major new legitimate (live) theater. Everything seemed perfect until the costumes arrived for the premiere dress rehearsal. There was nowhere to put them. Why? Because this lavishly-appointed theater lacked one teensy little item: dressing rooms.

Iterative design, with its repeating cycle of design and testing, is the only validated methodology in existence that will consistently produce successful results. If you don’t have user-testing as an integral part of your design process you are going to throw buckets of money down the drain.

How user testing saves money

Problems are fixed before the product is shipped, not after.

Every company user-tests. For some, it is done as an integral part of the early prototyping process. Others do it during beta. Most companies still do it post-release. Their test subjects are unhappy customers, and their “usability professional” consists of a boiler-room of unhappy customer support staff.

It is far less expensive to have one person spend an hour revamping a light-weight prototype than for a room full of customer support people to spend two months fending off calls while your entire engineering team tries frantically to fix reported problems and get out release 1.1.

The team can concentrate on real problems, not imaginary ones.

This is perhaps the greatest money-saver. Developers know about most of the problems that will be found before the testing even begins. The problem is, they also know about a raft of other “problems” that aren’t real. And they have no tool other than testing to tell the difference.

The few products that actually end up working with little or no user-testing invariably have been way overbuilt, with thousands of lines of code written to prevent problems that would never have arisen. You are paying for that code to be written; you are paying for that code to be maintained.

Engineers code, instead of debating.

We’ve all been to those project team meetings where perhaps ten $100/hr engineers, designers, and marketing people sit around and debate how users are likely to respond. That’s $1000 an hour for uninformed opinion. One usability professional, applying the scientific method, can have a real answer in two hours for a tiny fraction of that amount. Not only that, it will be the right answer.

Time to market is sharply reduced.

The iterative design process cycles perhaps 8 to 12 times in a properly-tested product before reaching a reasonable stasis. I have routinely performed that many iterations in a two to three week period. At the end, what may have started as a brave stab in the dark has been honed into a solid, compact, successful design.

By having the design team be first off the blocks, you will usually have the major part of design ready to go before the implementers are ready for it. Where time to market is even shorter, designers are trained to work with the rest of the team, concentrating first on those areas that need to be implemented first.

Of course, major complex applications will require a higher level of activity for a longer time. Mileage will vary, etc. The point is that iterative design can be done swiftly and it can be done in parallel with the development effort so that time to market is actually reduced.

Finally, upon first release, your sales department has a rock-solid design they can go sell without having to pepper their pitches with how it is all going to actually work in release 1.1 or 2.0.

I have spent much of my twenty-five year career in software design troubleshooting projects that are in trouble, helping them get back on course. The single thread that ran through every one of them was a lack of user testing.

If your people don’t know how to user test, find someone who does. Whether you bring in an outside design firm or hire your own people, make sure they are telling you all about their plans to test, because if they don’t test, your customers will, and it will cost you a whole bunch more money.

Don't miss the next action-packed column!
Receive a brief notice when new columns are posted by sending a blank email to asktoglist-subscribe@yahoogroups.com.

return to top