We’ve been doing this learning-about-UX thing for a little over 4 months now. On today’s podcast we take a look back at the Top 10 things we learned in that time. If you’ve been following the site over the past week you know that we’ve already covered this material in two posts. But like any good discussion, we dig a bit further into each topic on the podcast.
Top 10 Key UX Concepts We’ve Learned (So Far)
10. Proactive vs. Reactive
9. UX Testing Resistance
8. The Power of Process
7. The Difference Between User Experience and Usability
6. The Critical Path
4. Prove vs. Improve
3. Signal vs. Noise / Information Theory / Entropy
2. Quick Tests Can Be Valuable
1. There’s a Great UX Community
Remember you can subscribe to our podcast on iTunes – new episodes each week!
Follow us on Twitter: @BUXofficial
Friend us on Facebook: facebook.com/BetterUserExperience
View us on Youtube: youtube.com/user/BetterUserExperience
When Ben and I started learning about UX back in August, we committed to a period of learning that ended on December 15th. That doesn’t mean we’re going away, but it does mean that we’re at the end of the time we set aside to get our feet wet in the discipline. As a result, we’ve been doing some reflecting on what we’ve learned and what the most valuable take-aways are from the past 4 months.
On Wednesday, Ben covered the first 5 key UX concepts and today we polish off the list with numbers 5 through 1.
When you don’t know what you’re doing, watching experts is like watching magic happen. How in the world does somebody know how to do that? I mean, how did he know how to do that!?
For visual design and page layout, the saving grace for me was CARP.
We talk most about CARP during the Feng-GUI vs… posts. In those posts we would evaluate a page using the principles of CARP and then run it through the Feng-GUI tool. This gave us a way to reflect on the visual elements on the page and a way for active testing those reflections. CARP is important for UX’ers because it allows for a way to communicate design decisions to stakeholders who don’t have a visual eye. You can tell them “This is not ‘Nam. There are rules!” to quote the Big Lewbowski. It gave form to the void – a method to the magic. When looking at a design, it gave me a way of looking at it to know not just if it was any good but why it was the way it was. It gave me a system for thinking about design.
4. Prove vs. Improve
In the beginning of learning about user testing, we thought user testing was essentially a scientific endeavor. And by that I mean that we thought that you had to have strict controls and that it really mattered how the tests were conducted. We felt a strong need to make sure that no bad data tainted the testing process.
But thanks to the Wisdom of Steve Krug and the idea that user testing doesn’t have to be about proving an idea, it’s about improving the website. And because all websites can be improved, in a sense it’s fish in a barrel.
There’s more to that concept: we are all experts at filtering and grouping, it’s about getting points of view not about error bars, and that a majority of the improvements on the website are going to be able to be discovered by out-of-context users.
This means that quick, down-and-dirty testing can be effective because to a certain extent, it’s all about just measuring stimulus response. It’s a freeing concept.
3. Signal vs. Noise / Information Theory / Entropy
It’s important to remember that websites are at heart, a communication medium. The computer is a communication device. The Internet connects us. But your web presence is responsible for communicating the message.
The ability to effectively communicate online is essentially what it’s all about.
There are two parts to communication: the communication channel and the message itself.
Information theory is concerned with the communication channel. The message is for all intents is irrelevant. The reason the message doesn’t matter is because it’s generic. Web content is text, video, and audio. The specifics of those content types don’t have any bearing on how the Internet works.
The channel itself has a maximum capacity (or maximum bandwidth). In that channel, there are three possible states:
- the amount of the channel used by the signal
- the amount of the channel used by noise
- the remaining unused channel capacity
If you assume that channel capacity is 100%, all you’re left with is signal and noise. Or, more abstractly, order and disorder.
That’s why we dragged the concept of entropy into the discussion in the first place (Ben’s awesome entropy in UX podcast). At a real level, defining a website’s critical path is the same thing as creating order from disorder.
We know that entropy is the tendency of things to go from order to disorder. But it’s also a measure of the system’s inability to do work.
A web site’s job is to do work. “Work” in this case is defined as “the process of accomplishing the website’s goals”. If you lay out your critical path, you should be able to generate a number for each page in the process that is probability that somebody will leave that page and not continue any farther on the critical path.
Lower that number and you will have reduced the entropy on the website. You will also have strengthened the website’s signal and lowered its noise. You will have strengthened your critical path and will have additional leads or revenue to show for it.
2. Quick Tests Can Be Valuable
As we talked a bit #7 Prove vs. Improve, the real shocker for user testing is that it can be done really quickly and with all kinds of leading questions. Essentially, think of a professional yet scientifically invalid test and it’s probably good enough for a basic user test.
The take away is another Krug-ism: “Test early and test often”.
1. There’s a Great UX Community
When we started this site it was a selfish endeavor to learn about web usability. I only use the word ‘selfish’ to mean that we did all of this for our own gratification. Yes, we hoped that what we put together would resonate with others. Yes, we hoped we’d find a cool existing community of people who are into UX which we could become part of and could learn from and contribute to. And yes, we hoped to reach out to a web ux company here and there.
What we found exceeded our best expectations. Universally, the UX community seems to be made up of good folks who love to learn, are striving to get better, and who generally have a groovy outlook on things.
We’ve been lucky enough to talk to Rafael from Feng-GUI, Paul from Usabilla, and Rachit from Inspectlet. They were very generous with their time and their wisdom and we appreciate them taking the time to talk with us. Check out our Blogroll and Twitter … followees to find out our community.
In the coming weeks and months we hope to talk to loads more. We want to talk to people passionate about building a better user experience. Learning how to build sites that give users a better experience is what motivates us to do this website, to write these posts and to do the podcasts that we do. We’re fighting entropy and we’ve learned that we’re not alone.
There’s a whole wonderful community of developers, designers, interface experts and ux tool makers who are all fighting the good fight. We’re honored to be a part of it.
[Note: In Monday’s podcast I said we were going to be doing a real user test today. We’ve decided to push that back to next week. We had two more items we wanted to get to first.]
When we first started learning about user testing back in August, I was under the presumption that user testing was essentially a controlled, systematic endeavor. But as Steve Krug and Jakob Neilsen will tell you, there’s more than one way to test a cat. (That’s a direct quote, I believe.) A lot can be gained just by getting feedback from somebody. They argue that the name of the game is improving your website (not testing a hypothesis) and that most major issues on a website are visible to most users.
Krug advocates companies set aside one morning a month to do three user tests. Then, debrief over lunch.
This rapid-testing idea was inspiring. It’s like everybody knows what’s not working on your site, and to learn the secret to what’s broken, all you have to do is ask them to tell you about it. Whoa!
But is that true?
Are we all walking around with an innate sense of what sucks on websites? And if so, why can’t we design a perfect one out of the gate? If we all have the magical power of user testing, why can’t we user-test our own site and save the time, cost, and hassle of asking other people?
The reason you can’t user test your own website is because you’re too close to it. You’re like a director who can’t enjoy his own movie in the same way as a regular viewer because you remember what happened off screen. When you look at your own website you remember the versions of the site that almost were and you know intimately the reasons and compromises that went into creating your current website. You can’t see the forest for the trees.
Your average user is focused on a completely different experience. They are focused on what game theorists call “utility-value maximization”. In plain English, it means that people are goal oriented on websites. It’s built into the way the Internet works. We click on links because we want to access new content. Each click is a statement of purpose. When a user visits your website, they are there for a reason and they have a goal in mind. Han Solo had it right about web design when he said, “Give the wookie what he wants.”
Have you ever helped a relative get a computer or get on the Internet or otherwise interface with a completely new piece of technology for the first time? Do you remember the first time you opened Photoshop or Illustrator or Dreamweaver or the first time you tried to center something with CSS and you had that feeling that you were in completely new territory?
That happens when you have no way to contextualize what you’re seeing with your previous experience.
People create mental models of all their behaviors. Have you ever caught yourself having a dumb conversation with somebody about the weather when (a) you don’t really know why you’re talking to them in the first place and (b) there’s nothing special about the weather? Why do we do that?
Or how about in movies and TV shows when people hang up the phone without saying goodbye first. That’s always weird to me. It’s because my mental model includes goodbye language at the end of phone conversations and in TV shows, it doesn’t advance the plot so it isn’t necessary.
Once you get your non-techie relative online, they too will begin to create a mental model of how the Internet works. Some of it will be based on fact. A lot of it will be based on feelings and intuition that is not correct. (I’m reminded of an old boss who, every time he saw the Blue Scree of Death on his PC would scream “BILL GATES!”) But people don’t get retrained when they learn the Internet wrong. Nope, we just deal and hope they catch up.
Jakob Nielsen points out many of the points of technical confusion in his blog post on the same topic.
- Operating-system windows vs. browser windows
- A window vs. an application,
- Icons vs. applications,
- Browser commands vs. native commands in a Web-based app
- Local vs. remote info
- Different passwords and log-in options (users often log in to other websites as if they were logging in to their email)
In short, when it comes to computers, a lot of us are still getting our act together. What happens as people gain experience using the Internet is they gain an intuition for how websites are supposed to work. And that’s why there’s a great list of usability conventions to use when developing your website.
What does this mean for web sites and user testing?
Signal vs. Noise
All communications come down to the issue of signal vs. noise. Those of us old enough to remember when Sprint was a long distance company who had sound quality so clear you could hear a pin drop have experience with the problem. In any communication medium, whether it’s spoken and heard, or transmitted by radio, TV, print, telegram, fax machine, or Internet, the message must be transmitted and it must be received. The message is encoded in a language and in a medium, and in a context. In order for it to be decoded correctly, the person receiving the message must understand the language, have access to the medium, and know the context in which the message is meant.
Think of a physical long-distance phone line. The reason Sprint was so happy about their clear signal is because it’s hard to eliminate all the noise. Static, buzzing, clicks, pops – all could (and did) effect the ability to be heard.
When you think about a telephone line, the communication can be broken down into two parts:
- The mechanical workings of transmitting a person’s voice from one place to another and,
- The content of the communication
If you were to user-test the phone line, you could test for the same two attributes above:
- The quality of message transmission and,
- The content of the message
Unless you own a tin-foil hat, phone companies today don’t care about the content of your phone calls, texts, and messages. They are focused on delivering on the quality of message transmission. Also, because we all have such a complete mental model on how to make a phone call, it turns out that isn’t the part of the phone business that’s growing now. The user experience for making and receiving phone calls is essentially complete. The new wrinkle is welding the digital side of things to the phone. Now it’s all about experiencing the mobile Internet. And that, invariably, leads to more half-baked mental models for using the Internet.
The Medium vs. the Message
Like the double rainbow dude, we’re left to wonder, “What does it all mean!?”
Let’s cut to the chase.
When you add up the fact that there are so many ways to leave a page – from going to another site, clicking the back button, closing the window, getting up and walking away, clicking into a different window, getting distracted, plus the various mental models that people have for how the Internet should work, it’s safe to say that there’s a lot of noise in the channel.
In the physical long-distance phone line we discussed earlier, the medium was the phone line. In your website your critical path is the medium.
Now let’s think about that for a second. Remember how earlier we said that clicking a link is a statement of intent. It could be that the user is expecting to end up on a YouTube video or is expecting to see an Amazon page for digital cameras or is looking for an article on Wikipedia or a house on an MLS. The content is the message. The Internet is the medium. And getting to your content is part of the medium.
We’ve all dropped a call on our mobile phone. There’s a distinct difference between a phone call where you’ve said everything you wanted to and hung up and when the call drops. When the call drops – you can’t communicate any longer. In the same way, if your website can’t get the user to the content they’re looking for, then it can’t deliver the message. It’s a failure of the medium to deliver the message. The noise in your website has to be low enough so that people can find the signal they are searching for.
This signal, the message, is what the message says. It’s the literal content of your website. This is why developing a strong critical path is so crucial. The only way to be heard above the din of distraction is to have a focused message and to provide an obvious way to access that content.
What we’re talking about here is signal quality. And all that’s necessary to test a signal is somebody on the other end to describe what they’re getting.
If we were testing long-distance phone service we’d whip on some glasses, stomp through the woods and ask “Can you hear me now?“. But since we’re testing websites, all that’s required is to ask somebody else what they see on your website.
User Testing in 3 Questions
When you’re looking for quick insights about your website, you’re looking to know something about signal strength. It’s evident in the type of questions that are asked.
- What frustrated you most about this site?
- If you had a magic wand, how would you improve this site?
- What did you like about the site?
None of them concern the content on the website. All of them are testing either signal or noise.
Break down the expected answers:
- What frustrated you the most? 99% of the time this is going to be an answer that relates to not being able to find something. It could also have something to do with your messaging.
- How would you improve the site? Because you directed them to the site and not what your site sells or provides, this answer is going to relate somehow either to accessing information or clarifying options.
- What did you like about the site? This will help with positive feedback and will help you confirm areas of the website that perform well or stand out to the user.
Nothing in there explicitly dealt with the quality of the material. What’s important is strengthening the signal.
I should state that these are not the only questions you could use in a 3-question test. You can test the critical path, a secondary path, or test for comprehension and cognitive load with your site structure without knowing anything about the user.
And we’re going to prove it. Next week, with a little luck, we’ll bring you a video of our own 3 question drive-by user test.