We’ve been compiling a list of the tools that we want to have in our User Testing Toolbox. After an initial look and then a more in-depth look, we really like Inspectlet and believe it’s worth using in your user testing process.
We reached out to Inspectlet’s founder, Rachit Gupta, with a few questions and he was nice enough to reply to us via email.
What’s your story? How did you get into User Testing?
I’ve always been fascinated by how important design is to the experience of using a product. I think there’s a fundamental difference in perspective between the designer and a visitor that makes it difficult to intuitively understand a visitor’s experience. I wanted to improve the process of understanding your visitor’s perspective so you can iterate more effectively. There are tools out there that do this but I wanted to create a full user experience suite that’s affordable to everyone, because I think everyone should care about user experience the same way they care about traffic numbers.
What’s the elevator pitch?
Inspectlet helps you gain a deeper understanding what your visitors are thinking by observing their actions naturally.
How did it start up?
About 8 months ago, I started building a team and we planned a roadmap for the product taking into account what was lacking from existing tools out there. We launched in June as a fully bootstrapped startup and it’s been an amazing ride ever since.
What’s up coming?
Some big changes are coming up! We’ve been working closely with users to create a stronger product. We’re looking into new ways of visualizing the mounds of data we gather, and possibly introducing a free plan as well to encourage people to understand their visitors better.
Do you have any partner companies? What’s your ideal tool set?
We’ve been in talks with some companies but there’s nothing to announce yet. 🙂 We like to use Google Analytics for traffic analysis, Inspectlet for understanding our visitors, Optimizely for split testing iterations, and GetSatisfaction for gathering feedback.
Do you remember what a mystery sex used to be? (Don’t worry Mom, it still is, I swear. ) You’d talk about it endlessly with your friends just telling and halfway believing each others lies. And because you didn’t know what you were talking about, you boned up on your reading. [Boned up? good grief – ed] All of the sudden, Cosmo – either your sister’s, your Mom’s with their “100 ways to spice up your sex life” was interesting reading.
You mean they just make lists of this stuff and print it for you to read?
At some point, the reading up becomes boring. The boundaries are well known and what used to be exciting is now just another list of stuff that other people are doing right.
That’s when you know it’s time to get down to business.
That’s basically where Ben and I have been with this user testing stuff for the past few weeks. You can hear it in the podcasts. Enough with the reading! Let’s make with the fu…n user tests!
And so we did.
On Wednesday, we showed you the video of how it all went down. If you missed that post, we’ll save you the trouble. Here ya go.
Could you tell how vulnerable I was in that clip? You just wait Kate, let’s do it again in a few years. I’ll be way better.
Coming out of that experience, we learned a lot. Here’s a list of the Top 5 takeaways.
5. Be prepared
I suppose that we were preparing the whole 4 months leading up to this first user test. We had practiced on ourselves and friends, but this was the first ‘real’ test with a stranger in a strange place. The rehearsals were key in getting ready the mechanics and hardware. But there is no way to prepare completely. There’s no way around sucking the first time and being un-smooth.
Smoothness didn’t seem to matter, luckily. Perhaps authenticity trumps smoothness. If you’re genuine and honest, people may overlook a herky-jerky delivery.
4. Getting permission is the hardest part
Getting the permission to record and use the coffee shop was tough. We had to explain who we are and what we are doing. If you’re not used to doing that, it can seem really awkward. But you have to do it.
It’s true that you really only need to get permission one per place and that it’s not really necessary to do a user test. I mean, you can do one at home or in your office just the same. But doing it this way, it was much more of a challenge than actually conducting the test.
This is an example of activation energy. The hard part is starting and now it’s much easier to keep it going.
3. It Gets a Reaction
While we were at the coffee shop we ran into four folks who work at a hotel on the island. It was a manager and her sales team. The minute they found out that we were doing website user testing, their body language became more guarded and the manager slowly made her way towards the door. We had nothing to sell and weren’t even interested in getting them to sit down for a test but just the words were enough to cause them discomfort!
Equally interesting, after their initial reaction, they warmed up to it. They talked about how they user tested the items that go into each hotel room. The manager told a story about having to get new alarm clocks for the rooms and that she wanted to test it before they used them. In her words, “if I can’t work it in two minutes then it’s not going in the hotel rooms”. She found a way to apply user testing to her own field.
We’ve speculated that the strong reaction is because small business owners have their personal identity wrapped up in their business. It’s hard to willingly submit to judgment. But, once they realize that it’s a way to improve their bottom line and that people are ALREADY judging them, they tend to get on board.
2. Good Things Come Available in Bite Size
We wanted to see if it was true that you could get value out of asking a stranger what they think of a website. Is it valuable or is it just a stunt? It turns out that we got some good data that we can give to the website owner. The moral of the story is to get out there and ask people their opinion about your website. You will learn something. Simple as that.
1. People Are Cool
Meeting people is always a highlight. Humans are social animals. We need to communicate and interact. That’s a big part of what this blog is about. The folks we met at the Daily Grind helped us dive deeper into what User Testing is all about. We are thankful for that. It didn’t have to go that way – we could have been chased out of there. Instead, it went really well.
It’s important to say that too. If you read most media, it’s mostly one group of people complaining about another group of people. You’d think that we are all angry people but that’s just not the case. People are cool.
Bonus: What we learned about the website in the user test
- Better and more headlines – People skim websites. It’s an issue of cognitive load. Successful user testing, as a rule, reduces the cognitive load a website imposes on its viewers. Better headlines would help the user quickly scan the page and drill down to the relevant content and would reduce cognitive load. At 3:10 our user-tester, Kate, responded to a question about services with, “I probably gotta read about it first, right?”. She was under a higher cognitive load because of the camera and the test, but that helped to reveal that one couldn’t skim the headlines for meaning. They weren’t there.
- Include pricing info (5:50) – As a professional we all get used to not seeing prices on a website that sell a service. “If they want to know, they’ll call” is a common thought. Be that as it may be, it’s also true that most people would prefer to at least know the general pricing structure. If you choose to not have prices on the site, you should at least address it. “Prices range from”, “Free Consultation” or, as our user suggested “10% off” are examples. You could also explain how your prices are structured – by the hour, by site, do you have packages of services, etc.
- Improve some of the photos (1:20 and 4:10) – The compass image could be replaced. Kate mentioned the color balance being ‘off’. We feel it draws attention without leading to the critical path of the site. It’s not imediately clear why the compass is there. Suggest: Either create a new photography or create a headline that explains the compass.
Implementing these suggestions would significant improve the user experience of your site.
Enjoy the holidays and be on the lookout for day-after-Christmas podcast!
How do you follow up a week full of looking back? By pushing forwards!
This week we finally put down the books, stopped all the chitter-chatter and went out and did a user test.
I know. We were excited too.
If you’re anything like us, you read a lot about user testing online. Five reasons you should do this or 10 ways to improve that. It’s not that those are bad articles, it’s just that they’re not touching what happens in a real test.
Last Friday, Newman and I went down to the local coffee shop in Wrightsville Beach, NC. It’s called The Daily Grind. We walked in to an empty house, except for Kate, the lovely barista on staff. We chatted with her and her boss, Tony, for a few minutes to explain what we were up to and before you know it Kate had volunteered to take a user test.
Our goal was to do what we’re calling a “drive-by” user test. Basically, it means you can just roll up on somebody – ask them a question about a website – and see what you get back. It’s not meant to be scientific. Just more of a survey. It’s our belief that even this quick method will yield valuable insights.
We could tell you all about what we did but we shot a video of it instead.
In the video you’ll see the entire user test (roughly 6 minutes) and Newman’s thoughts afterward. All-in-all, I have to say, it was a pretty good first attempt.
Thanks again to Kate – your pictures rawk! – for being our user tester and for Tony and the gang allowing us to bother their shop. As you’ll hear in my voice on Monday’s podcast where we did a whole before and after thing, their coffee will put a motor in you. No doubt.
Join us on Friday when we talk about what we learned from the user test – both about user testing and about the website we tested – and then on Monday – the day after Christmas – for your aural pleasure, we have a podcast lined up and ready to go about the whole event.
When Ben and I started learning about UX back in August, we committed to a period of learning that ended on December 15th. That doesn’t mean we’re going away, but it does mean that we’re at the end of the time we set aside to get our feet wet in the discipline. As a result, we’ve been doing some reflecting on what we’ve learned and what the most valuable take-aways are from the past 4 months.
On Wednesday, Ben covered the first 5 key UX concepts and today we polish off the list with numbers 5 through 1.
When you don’t know what you’re doing, watching experts is like watching magic happen. How in the world does somebody know how to do that? I mean, how did he know how to do that!?
For visual design and page layout, the saving grace for me was CARP.
We talk most about CARP during the Feng-GUI vs… posts. In those posts we would evaluate a page using the principles of CARP and then run it through the Feng-GUI tool. This gave us a way to reflect on the visual elements on the page and a way for active testing those reflections. CARP is important for UX’ers because it allows for a way to communicate design decisions to stakeholders who don’t have a visual eye. You can tell them “This is not ‘Nam. There are rules!” to quote the Big Lewbowski. It gave form to the void – a method to the magic. When looking at a design, it gave me a way of looking at it to know not just if it was any good but why it was the way it was. It gave me a system for thinking about design.
4. Prove vs. Improve
In the beginning of learning about user testing, we thought user testing was essentially a scientific endeavor. And by that I mean that we thought that you had to have strict controls and that it really mattered how the tests were conducted. We felt a strong need to make sure that no bad data tainted the testing process.
But thanks to the Wisdom of Steve Krug and the idea that user testing doesn’t have to be about proving an idea, it’s about improving the website. And because all websites can be improved, in a sense it’s fish in a barrel.
There’s more to that concept: we are all experts at filtering and grouping, it’s about getting points of view not about error bars, and that a majority of the improvements on the website are going to be able to be discovered by out-of-context users.
This means that quick, down-and-dirty testing can be effective because to a certain extent, it’s all about just measuring stimulus response. It’s a freeing concept.
3. Signal vs. Noise / Information Theory / Entropy
It’s important to remember that websites are at heart, a communication medium. The computer is a communication device. The Internet connects us. But your web presence is responsible for communicating the message.
The ability to effectively communicate online is essentially what it’s all about.
There are two parts to communication: the communication channel and the message itself.
Information theory is concerned with the communication channel. The message is for all intents is irrelevant. The reason the message doesn’t matter is because it’s generic. Web content is text, video, and audio. The specifics of those content types don’t have any bearing on how the Internet works.
The channel itself has a maximum capacity (or maximum bandwidth). In that channel, there are three possible states:
- the amount of the channel used by the signal
- the amount of the channel used by noise
- the remaining unused channel capacity
If you assume that channel capacity is 100%, all you’re left with is signal and noise. Or, more abstractly, order and disorder.
That’s why we dragged the concept of entropy into the discussion in the first place (Ben’s awesome entropy in UX podcast). At a real level, defining a website’s critical path is the same thing as creating order from disorder.
We know that entropy is the tendency of things to go from order to disorder. But it’s also a measure of the system’s inability to do work.
A web site’s job is to do work. “Work” in this case is defined as “the process of accomplishing the website’s goals”. If you lay out your critical path, you should be able to generate a number for each page in the process that is probability that somebody will leave that page and not continue any farther on the critical path.
Lower that number and you will have reduced the entropy on the website. You will also have strengthened the website’s signal and lowered its noise. You will have strengthened your critical path and will have additional leads or revenue to show for it.
2. Quick Tests Can Be Valuable
As we talked a bit #7 Prove vs. Improve, the real shocker for user testing is that it can be done really quickly and with all kinds of leading questions. Essentially, think of a professional yet scientifically invalid test and it’s probably good enough for a basic user test.
The take away is another Krug-ism: “Test early and test often”.
1. There’s a Great UX Community
When we started this site it was a selfish endeavor to learn about web usability. I only use the word ‘selfish’ to mean that we did all of this for our own gratification. Yes, we hoped that what we put together would resonate with others. Yes, we hoped we’d find a cool existing community of people who are into UX which we could become part of and could learn from and contribute to. And yes, we hoped to reach out to a web ux company here and there.
What we found exceeded our best expectations. Universally, the UX community seems to be made up of good folks who love to learn, are striving to get better, and who generally have a groovy outlook on things.
We’ve been lucky enough to talk to Rafael from Feng-GUI, Paul from Usabilla, and Rachit from Inspectlet. They were very generous with their time and their wisdom and we appreciate them taking the time to talk with us. Check out our Blogroll and Twitter … followees to find out our community.
In the coming weeks and months we hope to talk to loads more. We want to talk to people passionate about building a better user experience. Learning how to build sites that give users a better experience is what motivates us to do this website, to write these posts and to do the podcasts that we do. We’re fighting entropy and we’ve learned that we’re not alone.
There’s a whole wonderful community of developers, designers, interface experts and ux tool makers who are all fighting the good fight. We’re honored to be a part of it.
We started this website in late August, 18 weeks ago. It’s a learning experiment. We were two buddies, both web designers, who wanted to know more about web usability and user testing. We gave ourselves the goal of learning what we could by December 15th.
By my calendar, that’s tomorrow.
It’s also nearing the end of the year. And the end of the year is always a time for reflection and for taking stock.
Since we’re at the end of our original learning period, Newman and I felt it was only right to look back on what we’ve learned to see what the key takeaways have been.
We created a list of our Top 10 Key Concepts we feel we learned about building better websites.
This is a decent chunk of material to get through so we’ll be covering this topic for the next three posts. Today, numbers 10-6. On Friday, numbers 5-1. And on Monday, a podcast where we hash it all out. It’ll be fun!
10. Proactive vs. Reactive
The natural state of things is to be reactive. Reacting will save your skin if you are trying to run from a giant tiger leaping out from the jungle. In our modern times, troubling as they may be, there are no tigers. Death isn’t constantly at the door (and if it is, you’re doing it wrong).
Reacting has a down side though: the success of the reactor depends on the actor.
For a business owner, or really, any goal oriented individual, it’s a bad place to be.
Oh, it’s a comfortable place to be, but it’s not really how you want to be. Nobody has ever gotten their way because they were reactive. You don’t lose weight by not worrying about it. You don’t grow your business by only taking the business that comes through the door. It’s not a long-term strategy.
The advantage of being proactive is that you get to actually have a strategy. It’s chess, not checkers.
We talked about this as it relates to the small business owner back in September where we described an ideal initial meeting and how to create valuable monthly reports. This is important for UX designers because testing is inherently a proactive thing. It doesn’t happen in a reactive system. This is why oftentimes a cold wind starts to blow when the topic of user testing comes up. And it’s also the next key concept we ran into:
9. UX Testing Resistance
When told clients and business owners we know about UX testing, they generally respond negatively. “Watch people use our site and ask them stuff? Nah, I think the site is fine.” they would say. While we didn’t expect them to jump for joy, we did expect a little more… interest.
Why did they react this way? It’s because oftentimes it’s being introduced into a reactive environment and it rubs the wrong way.
I grew up playing the drums and I once got a piece of advice about how to play. The idea is that a drummer typically plays a little behind the beat or a little ahead of the beat. And either one is okay to do. The one thing you can’t do as a drummer is to shift from playing behind the beat to ahead of the beat mid-song.
That’s what introducing a proactive element into a reactive system is like. And its what accounts for a large amount of a business owner’s gut resistance to it. It doesn’t fit their work model.
And for the ones that it does, it’s still a matter of getting it incorporated into their system. So it’s a bit of an uphill battle, at least initially, unless you have forward-thinking clients.
Like most everything, the proof is in the pudding, and most businesses come around once they realize that user testing is a key component in raising their web site’s revenue.
8. The Power of Process
User testing fits best inside a larger structure of improvement and goal setting and seeking. It practically begs for it. The reason somebody would do a user test is because they plan on iterating their website. And once a website has been iterated, the change – good or bad – can be measured.
And that is essentially all the process you need. Once it becomes evident that sales/leads/revenue increases when improvements are made to the website and that the improvements are discovered through user testing – goal setting and allocating a proper budget for growing the web business will follow. The process will emerge. It has to. It’s how we make the website a better producer.
7. The Difference Between User Experience and Usability
It seems every sub-culture or community has weird semantic issues. When it comes to naming things, people aren’t very good. We like to borrow words that aren’t clear and then use them indiscriminately. As we jumped into the world of UX – which is short-hand for User Experience – we discovered a whole new vocabulary. One of the keys was understanding that User Experience was the umbrella, the super-system which includes Usability.
Usability is the technical side of UX. It is quantifiable and measured with tools like google analytics and A/B tests. User experience includes the qualitative and holistic experience. It’s harder to measure and includes emotions and feelings of the users. It is mysterious and an inexact science, but doing the ux techniques can reveal mistakes in a web design and improve sites.
6. The Critical Path
We first talked about the critical path back in late September in the post “Cut the Chute and Get on the Critical Path (to Profit)”. The concept is an easy one: design your website so that all of the energy is about moving a user from the beginning of the process to the end of the process. Define the pages in the critical process and boom, there’s your critical path.
But more important for me was the ability to finally know what’s supposed to go on a website.
That sounds like a shameful thing to admit: a web designer doesn’t know what’s supposed to go on a web site. But as a designer, we get wrapped up in the technical aspects of what we do. When we nerd out on our thing, we argue about technical arcana. For nerds, it’s not about solving business problems. That’s why there are business guys.
But business guys don’t know either. I’ve been building websites for a long time and it’s rare that a business owner takes ownership over the content and ordering of that content on the website.
They just leave it up to the nerds, and at least in the local market, nobody seems to specialize in knowing what goes where.
So the critical path is a big f-ing deal. Now we have a way to lay everything out: Do it in the way that makes the most sense to achieve the objective of the website.
Join us on Friday for #5-#1 on the list!
Essential Question: Can User Experience and Usability needs fit into a Maslow’s hierarchy of needs type pyramid?
Ben and I got to discussion a post on UXMovement.com titled “Are You Meeting the User Experience Hierarchy of Needs?” by Anthony. It was interesting and had a bunch of comments. I decided to research it more and share it with you. here:
Everyone ‘sort of’ understands Maslow’s pyramid. They may not understand it fully. It may be like Darwin’s theory of evolution – everyone can give you the basic tenants, but it is probably misunderstood. Here’s my understanding of Maslow’s hierarchy of needs. Food and water and shelter are basic needs that take precedence over things like morality and respect of others. You must satisfy the basic, base levels of the pyramid BEFORE you can / want to satisfy the higher levels. Maslow’s hierarchy of needs is often cited when explaining why a typically law-abiding person will break the law (higher order) in order to get food or sex (lower order). It’s a theory I have problems with.
Here are my questions: In order to be a whole person do you need to satisfy all needs? Is it harder to meet higher, mid, or lower needs? Does the pyramid establish a sequential order for achieving needs? And, to transition to interface design, must a “quality” website satisfy the all needs? Is it harder to meet the higher needs than the lower needs? Must the design process start with satisfying lower basic needs and move on to higher needs?
This article and chart puts functionality as the base of the pyramid. Together with information, this forms the first half of basic needs. The top half are higher needs with aesthetics below and usability on top.
Usability breaks out of the pyramid top suggesting that it is not inside the pyramid – or this could have been done to promote readability as the font would be very small if made to fit in the tip of the pyramid.
He outlines a model that says you must begin interface design with functionality and move on to content and aesthetics in order to achieve usability. Anthony defines usability as “the ease-of-use of an interface that increases user productivity.” There was some debate on this article and several commenters wanted to put aesthetics at the top of the pyramid – which makes me scratch my head.
Even though Anthony did not like this pyramid Chart, I like it quite a bit. Anthony called it pretty bullshit (which I thought was slightly rude). Just like a cow patty, this chart can teach a lesson. Working from the outside in, the chart establishes two contexts. At the top we have subjective/qualitative and ‘focused on experiences( people, activities, context)’ with an arrow pointing downward. The second context is at the bottom it states: objective/quantifiable ( products, features) and ‘focused on tasks’ with an arrow pointing upward.
Within the context of tasks (going up), the graphic seems to relate that the priority sequence is: functional( useful), reliable, … and peaking at meaningful.
Within the context of experience (going down), the graphic seems to relate that the priority of sequence is reversed – that meaningful is the first priority and functional useful is the last priority.
This chart tries to make the distinction between subjective experiences and objective tasks. That’s what I like about it. It seems state that if a subjective experience is meaningful and pleasurable that it does not have to be reliable or functional. Or that a subjective experience that is meaningful and pleasurable need not be reliable or functional. And this DOES seem to be descriptive of my personal experience.
If you have a meaningful and pleasurable relationship with a beautiful girlfriend, you don’t really care if it’s reliable or functional [Radio Edit]. But I don’t think he’s talking about relationships. If you just love your iPhone , you can subjectively overlook usability [Siri, why can’t I just drag and drop my files from my computer to my iPhone. Siri?] – Think supercars in the 80s – Toyota’s were WAY more usable. If you were focused on tasks, then you just care if it’s useful and reliable (Toyota)-you don’t really care that it’s not meaningful or pleasurable (Ferrari).
The fact that convenient lies in the center of hypercatalecta‘s diagram is important. That says that convenience is of equal value for objective users and subjective users. Both those focused on experiences and those focused on tasks care about convenience.
But here’s my problem: why is it a pyramid?. I have a feeling they just did that to make it look pretty. Perhaps this is why Anthony called it pretty bullshit (He called another guy pretentious ) But, because it defines the subjective / objective vectors, it gives me valuable insights. When working with some clients – subjectively oriented clients – they really only care about aesthetics… but here is where I think folks are missing the point – the reason they feel that way, is because that’s their experience. That’s how they experience the site. It’s their user experience.
This is why I concur with Paul Veugen – and others – that user experience is a super-system of usability. And, user experience is a complex interaction of many facets. Why? That little word USER. People are complex – they are different shapes and sizes, speak different languages, and use websites in different ways and for different reasons. Simply put – WE are the definition of complexity on this planet …currently
Alternatively, usability is technical. Usability can be measured… and can be measured by any asshole. Can you hear me now?
Moving forward –
Ugh… let’s look to Smashing Magazine, surely they are shed some light here.
This article,’Designing For A Hierarchy Of Needs‘ starts off with:
Based on Maslow’s hierarchy of needs, the idea of a design hierarchy of needs rests on the assumption that in order to be successful, a design must meet basic needs before it can satisfy higher-level needs. Before a design can “Wow” us, it must work as intended. It must meet some minimal need or nothing else will really matter.
Is this true? Or could a design that’s hard to use still succeed because it makes users more proficient or meets certain creative needs? Do you have to get all of the low-level needs exactly right before considering higher-level needs? To answer these questions, let’s start by looking at Maslow’s hierarchy.
Notice the author (Steven Bradley) is talking about general design. Ugh! Sure enough, we have some semantic issues here. Damn you language and our plastic language brains.
I feel much of the confusion and debate is sparked by imperfect language and vocabulary. Design, user experience, and usability: These terms are sometimes used inter-changeably. Yet, they can be very different.
Can you use these pyramids to build sites and interfaces? Sure. But, here’s my insight, you CAN test this way – Regardless of if testing Design, User Experience, or usability. First test for the base needs and then test for the higher needs. Why do this? Well, why not. Think of it like a map and a way to segment the complex task into manageable chunks. And, it can help prioritize your testing and help you make decisions and communicate problems to the stakeholders.
For your viewing Pleasure, I’ve collected all the Pyramids on my own private Giza.
Let’s face it – People love charts and pretty bullshit. Pyramids are old and busted – honeycomb is the new hotness. Users are complex.
So! There’s my insight. There’s my learning to share. I hope you enjoyed it. Now, I’m off to share a beer!