Being Cheap with Error Messages

Jared Spool

September 17th, 2007

The user didn’t enter their billing address over at CollegeBoard.com, when registering for an SAT test. It made sense not to enter the billing address, since it matched the address they entered on the previous page.

The site, being partially clever, decided to pre-fill in the zip code, but no other address fields.

When the user pressed the “Next” button, expecting to go to the next stage of the registration process, they were confronted with this error message:

Error message at CollegeBoard.com
The error message from not filling out the form

!The ZIP that you entered doesn’t match the City/State you entered. Please verify the information.

Of course, this error message is wrong on so many levels. For one, we did enter a city & state that matched the zip — on the previous page. Second, it’s clear we hadn’t entered anything on this page, so why would the site say this? Third, the field isn’t “ZIP” — it’s “ZIP Code” — and City/State are two separate fields, not just one. And why is there an exclamation point as the first character?

Because the developers only had one error condition (the user entered all the fields but they didn’t match in their lookup table), they crafted an error message that doesn’t fit the scenario our user encountered. And since it doesn’t fit, it takes the user a fair amount of logic to debug.

What could the developers have done?

  1. Craft an additional error message when the City or State field has been left unentered that tells the user they need to enter the field.
  2. When encountering one or both fields unentered, ask the user if they’d like to use the elements from the previous page.
  3. Automatically pre-fill the address with the previous page’s value. (Of course, this creates more work for the user if it’s not the right address, because they have to correctly wipe out the old information.)
  4. Have a “Use previous page’s data” checkbox (probably with a better label, since the user thinks of the “previous page’s data” as something other than what we just called it).

All of these choices require the developers do extra work. That’s probably why it didn’t happen in the first place. But, being cheap with development resources rarely produce a quality experience for the user, so it’s not clear it’s cost effective in the long term.

7 Responses to “Being Cheap with Error Messages”

  1. James Says:

    I remember a rule of thumb that my HCI teacher at University used to spout regularly: “If we can save an end user a few seconds by spending a few hours more on development, then so be it. It’ll soon become hundreds of hours saved for our users.” – where I work now I take the same approach, and it always pays off.

  2. Michael Hughes Says:

    A fifth alternative (not exclusive of the other good suggestions you offer) is to just not pre-fill the zip code. I see this pattern a lot and I call it the “bridge that goes half way across the river.” If you are not going to take the user all the way, don’t take him half the way. It usually results in an experience that illustrates what could have been useful but isn’t and creates a level of dissatisfaction that would not have been there if not for the halfway attempt.

  3. Glenn Says:

    Have you seen the error message this site generates if you leave the comment form blank? Sparse and misleading.

  4. Bill Tyler Says:

    Though it’s not an endorsement of the error message, it at least reports something you can see. Just last Friday trying to place an order for a Canadian friend at hallmark.com (which doesn’t deliver north of the border)I ran into the “reverse” of this situation of a pre-filled zip code. It says more about how programmers get errors incomplete(ly wrong) and seems directly related/applicable.

    Before you can even check out on hallmark.com you must select a ship to address type (drop down) and delivery zip code –so they can compute the shipping cost for you (this is a Web 2.0 sort of site). OK. That’s nice to know though maybe annoying to do before I finish shopping.

    I go to check out, reviewed the order (canceling the card). On the billing/shipping screen I got “Please enter the correct information in the required fields.” validation error.

    The problem: NO required fields left unfilled. NO “incorrect” information. NO fields marked in error.

    I was stopped dead in the check out process.

    Normally I would have given up and gone elsewhere but I was doing this as a favor needed to get the order done to make a order deadline. So I tried different browsers, dumped the cart and started again, sent an email (which received a reply two working days after the event).

    Eventually I found the service number (on a previous web page –not where I was stuck– and with phone routing system that talked about only about “flower orders” when I was ordering a kid’s gift) and a CSR. He was about as stumped except for the idea that I should “re-enter” the city.

    That didn’t work at first until I spelled out SAINT (for my home city of St. Paul). Eureka! I could get the order in before the 1PM deadline. I told the CSR to report this as a bug and to be sure to add it to the CSR FAQ.

    ============================

    Now what should we make of that error:

    * Never EVER report an error without being certain there’s a field with an error the user can identify.

    At least in Jared’s case you know that the zip code was the “error.” In mine, they trusted the reporting to do it and it didn’t.

    * Have error messages that correctly identify this issue (for those who live St. Louis, St. Paul or Ft. Worth)

    Apparently no one in the design process lived in town that uses a common abbreviation. One hopes they’ll do the substitutions for me in the future.

    I went back and confirmed the developers spent some time checking that zip code against other fields. Putting in the wrong city could reports an error for the street address (assuming the street doesn’t exist in the zip code). If I put in surrounding community names –it was happy.

    More interestingly in the hallmark case is they pre-filled the zip code on the form complete with state (which were NON-editable), but not city. Why not pre-fill the city also and let me edit it if it’s something that I might want to change (like a suburb name)? That would have been even more helpful and probably prevented the error in the first place. Not to mention it would have made having to enter it in the first place easier (and let’s ignore what happens if you got the zip WRONG at the start).

    Speaking of Zip Codes…

    In a mailing/shipping situation, if you have a correct zip code, why should the user EVER enter city and state? About all the user can do is confirm a zip code typo (which for many city dwellers would be the first 1-3 digits). Either they match or they don’t –and the user has to debug it it’s checked and found to be “wrong” (as in this case). Do we make users enter three pieces of information because we’re trained to do that and might consider it an error if we lost that control?

  5. Eric Meyer Says:

    And, of course, the ultimate cheapitude: not letting users input a phone or fax number in any format other than The One True Pattern (as defined by the developers of the site in use). Better still: not telling the user what pattern they have to use when filling in the number, letting them submit the form, and returning an error stating “you must enter a valid phone number”. WITHOUT SHOWING WHAT CONSTITUTES A VALID PHONE NUMBER.

    Um, not that I’m bitter.

  6. Risparmiare sui messaggi di errore | Alessandro Raffa . IT Says:

    […] sui messaggi di errore Venerdì 28 Settembre 2007, Fucinaweb L’intervento di Jared Spool su User Interface Engineering sottolinea, se ancora ce ne fosse bisogno, la necessità di […]

  7. Risparmiare sui messaggi di errore | Creazione siti a Genova Says:

    […] L’intervento di Jared Spool su User Interface Engineering sottolinea, se ancora ce ne fosse bisogno, la necessità di […]

Add a Comment