About a month ago, Newman wrote a post titled 4 Points of Wisdom from Steve Krug’s ‘Rocket Surgery Made Easy’. He was reading the book and wanted to share his insights. At the time, I was immersed in the James Gleick book ‘The Information’. And if you’re a regular reader, you know that led us on a week long journey exploring how entropy is related to web design and user testing. Oh yes, it got serious.
Now it’s my turn to go through ‘Rocket Surgery Made Easy’ and I have 4 more points of wisdom that I learned from reading through the book.
#1. Prove vs. Improve
This was a bit of a revelation to me. When I think of user testing, I think of trying to make a website better. It never occurs to me that I’m trying to prove something. It just seems obvious that I would be trying to improve things. But when it comes to user testing, it’s possible to do both.
To put it in scientific terms, it’s quantitative testing vs. qualitative testing.
Quantitative testing involves creating a testing methodology adhering to a strict testing protocol to ensure non-biased results. If it sounds like a science test, with a hypothesis and all of that, you’d be right. And because there’s a hypothesis and you’re looking for valid data feedback, it is setting out to prove something.
Qualitative testing is much less formal. It’s not focused on proving anything. Instead, its focus is on making things better.
Certainly, there’s room for both types of testing but when it comes to actually doing the testing, for most businesses, it will be more time and cost effective to concentrate on improving things. And doing that is as easy as asking for an opinion.
#2. Why Down-and-Dirty Qualitative Testing Gets Results
Deep down, we all know that nothing is free. So what gives here? Why does it seem like the testing type that requires less rigor, time, effort, and money seem to be the one that actually works? Simply put: it’s because ‘good enough’ is good enough. And by the time you’ve exhausted insights from qualitative testing, you’ll be in a better position to do quantitative testing.
Krug lays out three reasons in particular why qualitative testing works:
1. All sites have problems
I’ve never come across a website that couldn’t use a little work. Apparently, neither has Steve Krug. One of the main reasons that water cooler user testing works is because there’s always room for improvement.
2. Most of the serious problems tend to be easy to find
In the dust of creating a website, it’s easy to get too close to the whole operation. A common issue comes from troubleshooting problems. These problems can be technical or organizational or even baked into the business plan. Eventually a solution is found and implemented. Sure, it managed to go all Matrix on all of the problems and managed to dodge all of the bullets but that doesn’t mean that the solution was the right one for the user.
When you find a non-interested party and get feedback, the major issues will crop up again and again. You can’t see the forest for the trees. They can.
3. Getting stakeholders involved in user testing gives them a reality check on who their users are
Another common mistake that’s made during the planning phase of web design is that it’s created for an ‘average user’. The problem is, the average user tends to bear little resemblance to their actual average user. The obvious remedy is to go find some ‘average users’. Technologically, we can do just that.
Use a tool like Inspectlet (which Newman has been using and blogging about) or one of the other ‘Heatmaps / Mouse Tracking Tools’ we have in the sidebar to record your user’s sessions. Share those videos with all the stakeholders and then stand back. Everybody will get a new insight about the ‘average user’ and will immediately want to talk about it. It’s pretty remarkable to watch, actually.
This has the wonderful effect of getting everybody to focus on the right thing: improving the user experience.
#3. Test other people’s websites
This is just brilliant. User testing can (and should!) happen even before the first napkin sketch is drawn. How? Test the websites of your competitors or of somebody else in the same field. Test sites that have features you’re thinking of implementing. A cup of coffee and a conversation could save you weeks of work.
Remember, this is not rocket surgery. It’s basically asking people their thoughts about a website. Nothing says that you only can ask people about your website. As Krug says, “someone has gone to the trouble of building a full-scale working prototype of a design approach to the same problems you’re trying to solve, and then they’ve left it lying around for you to use.”
#4. Test for ease of understanding
“People choose not on the basis of what’s most important, but on what’s easiest to evaluate.”
Or more simply stated, “Don’t Make Me Think!“.
Websites do two things: provide information for a user to consume and provide a way to filter out all of the other information.
You may know this as our ‘filter’ and ‘confirming’ pages that we talk about again and again. Web pages, until you get to an information page, whether that’s a YouTube video, or a contact page, or a product description page, or MLS search results, are known as ‘filters’. This is because their main role is to get you to the information you’re there to view. The most obvious of these filter pages is the front page.
On most front pages of website, they exist to shuffle visitors off to other pages. They are not a destination page in-and-of themselves. The essence of filtering is clarity. And clarity can be measured by watching how people interact with a page. The easier it is to navigate the page, the lower cognitive load and the higher the success rate.
People want ‘easy’ more than they want ‘better’. If your site isn’t easy, it won’t have a chance to show that it’s ‘better’ because finding another website is just as easy.
This testing can be done even at the napkin-sketch phase. Just ask somebody who isn’t involved with the project to tell you what they see in the sketch. Listen to what they have to say. Chances are they’ll say “oh, this looks like a site for ____ and what’s this ‘experience’ button?” or something to that effect and you’ll immediately hone in on what doesn’t make sense to your users.
I can see why this book is the go-to resource for easy and effective user tests. It maintains a laser-like focus on how to improve your website with user testing. It’s filled with good nuggets. Certainly enough to do at least a third installment of this series. I really can’t recommend it highly enough. It’s the perfect compliment for the various user testing tools that we sample here on the site.
The book I’m reading: Steve Krug’s “Rocket Surgery Made Easy“. It’s a short 168 page instruction manual for performing user-tests on websites. Following the success of his first book on web usability, “Don’t make me think”, Steve taught workshops for businesses and organizations to instruct them in how to implement the ideas in the book. “Rocket Surgery” is that workshop in book form.
First name basis with Steve
I like Steve enough to call him by his first name. His style is very personable and direct. He is obviously insightful and thoughtful, and yet communicates in a concrete style that makes the topic easy to understand. He does use clichés too much and speaks with quotes and axioms often – I cut him some slack because that’s what I do.
I think it’s easy to understand – where other, more pedantic manuals are not – is the personalification principle (I could be making this up b/c I can’t find a reference) . He speaks directly to the reader, as if engaged in conversation. He is writing for a clearly define audience – those wanting to use the ideas in the first book or wanting to user-test websites. He speaks with the audience, not at the audience. You become engaged with the material and you learn more.
I like the book and can recommend it whole heartedly. Here are just a few takeaways that I’d like to share and promote among other web usability practitioners.
Test everything, all the time
Test the sketch on the napkin, test the wire-frame, test the page mockups, test the landing pages, EVENT test competitor’s websites before you even have a sketch.(great way to do market analysis) He illustrates the common pitfall of testing too late and the reluctance to test in the early stages of design. Any product of the design process can be tested and those tests can lead to small corrections that avoid big mistakes later in the process
He says it’s never too early to start testing a website – I agree. A small thing in the beginning of a project can become a big thing later. Small and early is easy to fix while big and late is hard to fix. Don’t delay, test today. Don’t worry about not being perfect or not being ready, because…
Perfect is the enemy of Good
There has to be some truth to this or there wouldn’t be so many dang quotes about it. Personally, I have three or four I use and switch between – “Put the ball in the fairway” and “Put the ball in play” and “Base hits, not homeruns”. … it keeps going, “you can’t catch fish, unless your line is in the water” and “A good plan in action is better than the best plan on the shelf.”
It means that we should resist the temptation towards perfectionism and that we should think about gradual improvement instead of “all or nothing”. There is a Japanese term for it that is popular in management circles – “Kaizen“. This idea fits with Steve’s practical, simple style – it works and that’s good enough.
KISS – Keep it simple, smarty-pants
Simple and practical is serviceable and useful. Simple holds up under pressure and close inspection. Everything about the book and its procedures is to reduce the complexity in order to make user-testing more likely to occur. Simple tests, simple reports, and simple improvements are essential to building a user-testing culture in an organization. Why? Because, complexity doesn’t yield better results. This is especially true when compared to not doing any testing at all.
One of his axioms states “one morning a month, that’s all we ask”. He is making it clear that the investment is small and the return can be big. His ‘small, non-honkin’ report’ should be written in 30 minutes and read in 2.
Lo and Behold “Maxims”
Steve uses 6 maxims to summarize his advice to user-testers. I’ve listed them here with a few notes:
- A morning a month, that’s all we ask
- Start earlier than you think makes sense
- Recruit loosely and grade on a curve
- Make it a spectator sport
- Focus ruthlessly on a small number of the most important problems
- When fixing problems, always do the least you can do
And, here is my interpretation. (matched by number)
- We (user-testing group) need resources and attention (from the organization), but it’s not going to be much.
- Always be testing and test as much as you can afford.
- Finding the ‘right’, target market users isn’t as important as you may think – problems and improvements will still be found.
- The more stakeholders you can get to view the tests and be involved the better – seeing is believing.
- Create a priority list of observed problems and follow through on fixing them.
- Simple fixes are better than a total redesign. Subtraction is often better than addition.