We’ve been compiling a list of the tools that we want to have in our User Testing Toolbox. After an initial look and then a more in-depth look, we really like Inspectlet and believe it’s worth using in your user testing process.
We reached out to Inspectlet’s founder, Rachit Gupta, with a few questions and he was nice enough to reply to us via email.
What’s your story? How did you get into User Testing?
I’ve always been fascinated by how important design is to the experience of using a product. I think there’s a fundamental difference in perspective between the designer and a visitor that makes it difficult to intuitively understand a visitor’s experience. I wanted to improve the process of understanding your visitor’s perspective so you can iterate more effectively. There are tools out there that do this but I wanted to create a full user experience suite that’s affordable to everyone, because I think everyone should care about user experience the same way they care about traffic numbers.
What’s the elevator pitch?
Inspectlet helps you gain a deeper understanding what your visitors are thinking by observing their actions naturally.
How did it start up?
About 8 months ago, I started building a team and we planned a roadmap for the product taking into account what was lacking from existing tools out there. We launched in June as a fully bootstrapped startup and it’s been an amazing ride ever since.
What’s up coming?
Some big changes are coming up! We’ve been working closely with users to create a stronger product. We’re looking into new ways of visualizing the mounds of data we gather, and possibly introducing a free plan as well to encourage people to understand their visitors better.
Do you have any partner companies? What’s your ideal tool set?
We’ve been in talks with some companies but there’s nothing to announce yet. 🙂 We like to use Google Analytics for traffic analysis, Inspectlet for understanding our visitors, Optimizely for split testing iterations, and GetSatisfaction for gathering feedback.
Do you remember what a mystery sex used to be? (Don’t worry Mom, it still is, I swear. ) You’d talk about it endlessly with your friends just telling and halfway believing each others lies. And because you didn’t know what you were talking about, you boned up on your reading. [Boned up? good grief – ed] All of the sudden, Cosmo – either your sister’s, your Mom’s with their “100 ways to spice up your sex life” was interesting reading.
You mean they just make lists of this stuff and print it for you to read?
At some point, the reading up becomes boring. The boundaries are well known and what used to be exciting is now just another list of stuff that other people are doing right.
That’s when you know it’s time to get down to business.
That’s basically where Ben and I have been with this user testing stuff for the past few weeks. You can hear it in the podcasts. Enough with the reading! Let’s make with the fu…n user tests!
And so we did.
On Wednesday, we showed you the video of how it all went down. If you missed that post, we’ll save you the trouble. Here ya go.
Could you tell how vulnerable I was in that clip? You just wait Kate, let’s do it again in a few years. I’ll be way better.
Coming out of that experience, we learned a lot. Here’s a list of the Top 5 takeaways.
5. Be prepared
I suppose that we were preparing the whole 4 months leading up to this first user test. We had practiced on ourselves and friends, but this was the first ‘real’ test with a stranger in a strange place. The rehearsals were key in getting ready the mechanics and hardware. But there is no way to prepare completely. There’s no way around sucking the first time and being un-smooth.
Smoothness didn’t seem to matter, luckily. Perhaps authenticity trumps smoothness. If you’re genuine and honest, people may overlook a herky-jerky delivery.
4. Getting permission is the hardest part
Getting the permission to record and use the coffee shop was tough. We had to explain who we are and what we are doing. If you’re not used to doing that, it can seem really awkward. But you have to do it.
It’s true that you really only need to get permission one per place and that it’s not really necessary to do a user test. I mean, you can do one at home or in your office just the same. But doing it this way, it was much more of a challenge than actually conducting the test.
This is an example of activation energy. The hard part is starting and now it’s much easier to keep it going.
3. It Gets a Reaction
While we were at the coffee shop we ran into four folks who work at a hotel on the island. It was a manager and her sales team. The minute they found out that we were doing website user testing, their body language became more guarded and the manager slowly made her way towards the door. We had nothing to sell and weren’t even interested in getting them to sit down for a test but just the words were enough to cause them discomfort!
Equally interesting, after their initial reaction, they warmed up to it. They talked about how they user tested the items that go into each hotel room. The manager told a story about having to get new alarm clocks for the rooms and that she wanted to test it before they used them. In her words, “if I can’t work it in two minutes then it’s not going in the hotel rooms”. She found a way to apply user testing to her own field.
We’ve speculated that the strong reaction is because small business owners have their personal identity wrapped up in their business. It’s hard to willingly submit to judgment. But, once they realize that it’s a way to improve their bottom line and that people are ALREADY judging them, they tend to get on board.
2. Good Things Come Available in Bite Size
We wanted to see if it was true that you could get value out of asking a stranger what they think of a website. Is it valuable or is it just a stunt? It turns out that we got some good data that we can give to the website owner. The moral of the story is to get out there and ask people their opinion about your website. You will learn something. Simple as that.
1. People Are Cool
Meeting people is always a highlight. Humans are social animals. We need to communicate and interact. That’s a big part of what this blog is about. The folks we met at the Daily Grind helped us dive deeper into what User Testing is all about. We are thankful for that. It didn’t have to go that way – we could have been chased out of there. Instead, it went really well.
It’s important to say that too. If you read most media, it’s mostly one group of people complaining about another group of people. You’d think that we are all angry people but that’s just not the case. People are cool.
Bonus: What we learned about the website in the user test
- Better and more headlines – People skim websites. It’s an issue of cognitive load. Successful user testing, as a rule, reduces the cognitive load a website imposes on its viewers. Better headlines would help the user quickly scan the page and drill down to the relevant content and would reduce cognitive load. At 3:10 our user-tester, Kate, responded to a question about services with, “I probably gotta read about it first, right?”. She was under a higher cognitive load because of the camera and the test, but that helped to reveal that one couldn’t skim the headlines for meaning. They weren’t there.
- Include pricing info (5:50) – As a professional we all get used to not seeing prices on a website that sell a service. “If they want to know, they’ll call” is a common thought. Be that as it may be, it’s also true that most people would prefer to at least know the general pricing structure. If you choose to not have prices on the site, you should at least address it. “Prices range from”, “Free Consultation” or, as our user suggested “10% off” are examples. You could also explain how your prices are structured – by the hour, by site, do you have packages of services, etc.
- Improve some of the photos (1:20 and 4:10) – The compass image could be replaced. Kate mentioned the color balance being ‘off’. We feel it draws attention without leading to the critical path of the site. It’s not imediately clear why the compass is there. Suggest: Either create a new photography or create a headline that explains the compass.
Implementing these suggestions would significant improve the user experience of your site.
Enjoy the holidays and be on the lookout for day-after-Christmas podcast!
Essential Question: Can User Experience and Usability needs fit into a Maslow’s hierarchy of needs type pyramid?
Ben and I got to discussion a post on UXMovement.com titled “Are You Meeting the User Experience Hierarchy of Needs?” by Anthony. It was interesting and had a bunch of comments. I decided to research it more and share it with you. here:
Everyone ‘sort of’ understands Maslow’s pyramid. They may not understand it fully. It may be like Darwin’s theory of evolution – everyone can give you the basic tenants, but it is probably misunderstood. Here’s my understanding of Maslow’s hierarchy of needs. Food and water and shelter are basic needs that take precedence over things like morality and respect of others. You must satisfy the basic, base levels of the pyramid BEFORE you can / want to satisfy the higher levels. Maslow’s hierarchy of needs is often cited when explaining why a typically law-abiding person will break the law (higher order) in order to get food or sex (lower order). It’s a theory I have problems with.
Here are my questions: In order to be a whole person do you need to satisfy all needs? Is it harder to meet higher, mid, or lower needs? Does the pyramid establish a sequential order for achieving needs? And, to transition to interface design, must a “quality” website satisfy the all needs? Is it harder to meet the higher needs than the lower needs? Must the design process start with satisfying lower basic needs and move on to higher needs?
This article and chart puts functionality as the base of the pyramid. Together with information, this forms the first half of basic needs. The top half are higher needs with aesthetics below and usability on top.
Usability breaks out of the pyramid top suggesting that it is not inside the pyramid – or this could have been done to promote readability as the font would be very small if made to fit in the tip of the pyramid.
He outlines a model that says you must begin interface design with functionality and move on to content and aesthetics in order to achieve usability. Anthony defines usability as “the ease-of-use of an interface that increases user productivity.” There was some debate on this article and several commenters wanted to put aesthetics at the top of the pyramid – which makes me scratch my head.
Even though Anthony did not like this pyramid Chart, I like it quite a bit. Anthony called it pretty bullshit (which I thought was slightly rude). Just like a cow patty, this chart can teach a lesson. Working from the outside in, the chart establishes two contexts. At the top we have subjective/qualitative and ‘focused on experiences( people, activities, context)’ with an arrow pointing downward. The second context is at the bottom it states: objective/quantifiable ( products, features) and ‘focused on tasks’ with an arrow pointing upward.
Within the context of tasks (going up), the graphic seems to relate that the priority sequence is: functional( useful), reliable, … and peaking at meaningful.
Within the context of experience (going down), the graphic seems to relate that the priority of sequence is reversed – that meaningful is the first priority and functional useful is the last priority.
This chart tries to make the distinction between subjective experiences and objective tasks. That’s what I like about it. It seems state that if a subjective experience is meaningful and pleasurable that it does not have to be reliable or functional. Or that a subjective experience that is meaningful and pleasurable need not be reliable or functional. And this DOES seem to be descriptive of my personal experience.
If you have a meaningful and pleasurable relationship with a beautiful girlfriend, you don’t really care if it’s reliable or functional [Radio Edit]. But I don’t think he’s talking about relationships. If you just love your iPhone , you can subjectively overlook usability [Siri, why can’t I just drag and drop my files from my computer to my iPhone. Siri?] – Think supercars in the 80s – Toyota’s were WAY more usable. If you were focused on tasks, then you just care if it’s useful and reliable (Toyota)-you don’t really care that it’s not meaningful or pleasurable (Ferrari).
The fact that convenient lies in the center of hypercatalecta‘s diagram is important. That says that convenience is of equal value for objective users and subjective users. Both those focused on experiences and those focused on tasks care about convenience.
But here’s my problem: why is it a pyramid?. I have a feeling they just did that to make it look pretty. Perhaps this is why Anthony called it pretty bullshit (He called another guy pretentious ) But, because it defines the subjective / objective vectors, it gives me valuable insights. When working with some clients – subjectively oriented clients – they really only care about aesthetics… but here is where I think folks are missing the point – the reason they feel that way, is because that’s their experience. That’s how they experience the site. It’s their user experience.
This is why I concur with Paul Veugen – and others – that user experience is a super-system of usability. And, user experience is a complex interaction of many facets. Why? That little word USER. People are complex – they are different shapes and sizes, speak different languages, and use websites in different ways and for different reasons. Simply put – WE are the definition of complexity on this planet …currently
Alternatively, usability is technical. Usability can be measured… and can be measured by any asshole. Can you hear me now?
Moving forward –
Ugh… let’s look to Smashing Magazine, surely they are shed some light here.
This article,’Designing For A Hierarchy Of Needs‘ starts off with:
Based on Maslow’s hierarchy of needs, the idea of a design hierarchy of needs rests on the assumption that in order to be successful, a design must meet basic needs before it can satisfy higher-level needs. Before a design can “Wow” us, it must work as intended. It must meet some minimal need or nothing else will really matter.
Is this true? Or could a design that’s hard to use still succeed because it makes users more proficient or meets certain creative needs? Do you have to get all of the low-level needs exactly right before considering higher-level needs? To answer these questions, let’s start by looking at Maslow’s hierarchy.
Notice the author (Steven Bradley) is talking about general design. Ugh! Sure enough, we have some semantic issues here. Damn you language and our plastic language brains.
I feel much of the confusion and debate is sparked by imperfect language and vocabulary. Design, user experience, and usability: These terms are sometimes used inter-changeably. Yet, they can be very different.
Can you use these pyramids to build sites and interfaces? Sure. But, here’s my insight, you CAN test this way – Regardless of if testing Design, User Experience, or usability. First test for the base needs and then test for the higher needs. Why do this? Well, why not. Think of it like a map and a way to segment the complex task into manageable chunks. And, it can help prioritize your testing and help you make decisions and communicate problems to the stakeholders.
For your viewing Pleasure, I’ve collected all the Pyramids on my own private Giza.
Let’s face it – People love charts and pretty bullshit. Pyramids are old and busted – honeycomb is the new hotness. Users are complex.
So! There’s my insight. There’s my learning to share. I hope you enjoyed it. Now, I’m off to share a beer!
Remember you can subscribe to our podcast on iTunes – new episodes each week!
Follow us on Twitter: @BUXofficial
Friend us on Facebook: facebook.com/BetterUserExperience
View us on Youtube: youtube.com/user/BetterUserExperience
Posts we reference in the podcast:
Steve Krug’s “Rocket Surgery made Easy” makes you want to run out and user test some websites. And, that’s what we did.
Here are two videos of Ben and Me testing some random websites. We did it to test our screen recording software. And, we did it to get the feel for facilitating a test. And, we did it to test out our New YOUTUBE Channel!
Of course, we had some insights to share – but it’s too dang late for typing this out. We’ll talk about it on Monday’s Better User Experience Podcast!
For your viewing pleasure!
and the Friday night Double Feature!
Are you ready for some PODCAST?! While the Giants and the Saints battle on the gridiron, Ben and I delve into the finer points of website Bounce Rates and the magic of Inspectlet user interaction videos – Yes, I just made that up, “User interaction videos”. If you watch them, you’ll feel inspired as well.
Remember you can subscribe to our podcast on iTunes – new episodes each week!
Posts we reference in the podcast:
SmashingMagazine.com – Easier is Better than Better
First Impression with Inspectlet
We place Inspectlet in the category of ‘Heatmaps / Mouse Tracking Tools’ and ‘Screen Capture Tools’. In the past, I had used Userfly, another tool in this category and had been underwhelmed. The code snippet slowed down the page and I eventually took it off. I don’t want stuff to load from another site on my page.
So, I was alittle leery Inspectlet, but Ben was favorable in his survey of UX Tools and we decided on this review. So keeping an open mind, I started to use it.
Here it is: Inspectlet let’s you observe people using your website – Actual users on the actual site. Like Coca-Cola, it’s the real thing. Paste a code on your site, and <magically> a recording shows up on the Inspectlet dashboard with a video along with key metrics – time on site, browser information (screensize and type) and number of pages viewed.
The numbers are good. It’s good to know that a visitor using IE found the site thru a google search for this spent 2 mins on this page.
The video is GREAT. It’s invaluable to know where they scrolled and clicked on the site. It’s the thing you can’t get unless you discover a cloak of invisibly and a teleportation device (Hey, I could build a kick-ass business in England if I could hook up with the Chosen One)
What is the experience of using Inspectlet?
Inspectlet is surprising simple to use, considering it’s magical abilities. It’s straightforward process and nothing seemed counter-intuitive to me. I was expecting to sign up, get the code and paste it into my site, and watch some videos. Real-Time Analytics and Heatmaps are two other features. These were not so valuable to me because of the low traffic the testing sites. Analytics would be more valuable if the sites got more traffic. And, the heatmaps would be more valuable if the sites had more interaction / clicks. I suppose over time the patterns would become clear, but in the few days I tested not enough data was collected to make Real-time Analytics and Heatmaps valuable.
The videos are ‘The show’ – the real value for me. You get a table of all the captures for the site which includes – IP address, starting page, capture length, browser (type), screen size, and referer and date. Click, and you get a video display page which is very well thought out and easy to use. Each page viewed by the user gets a new ‘chapter’. You can pause the video or speed it up (up to 5x speed). Unfortunately, you can’t scrub back and forth. There is a nice feature where the video automatically skips parts with no interaction AND the sections with the most interaction (clicks and mouse movement and scrolling) are highlighted in red. All of this can be viewed in their demo -that’s what sold me on it.
The Process of setting up the test
- Sign up / login / get to the dashboard
- Add a new site
- give it a name
- RealTime Twitter Query (Twitter should be capitalized in the form – new tool!)
- skip some stuff that’s not documented (Exclude IP, screen capture method, and screen capture frequency)
- Get and Install the code on each page you want to test
- Watch the videos – scribble notes frantically, and look quizzically
- Analyze the results – Come up with ideas for testing questions and changes to be made
Doing this process is fairly straightforward, but I do wish for more documentation. Getting comfortable with stuff you don’t understand is key for using alot of stuff on the webs – you can’t be an expert in everything. Still I wish for more documentation. Yet, the tool just works.
I did have a problem – no data was coming in – and it was quickly resolved by tech support via an email – ON A SUNDAY. Once again, support is good. The problem was that I had the ‘Staggered Captures’ set up incorrectly. (some more documentation would be good here)
How to get the most value from ‘Usability Observation’
Here’s my thing: Using Inspectlet will benefit your usability plan. I think a tool like this should be in every UX toolbox and here’s why.
Like I’ve said before, user-testing is about observing users with the intent of improvement – to make changes. Inspectlet gets directly to the observation. You are like a fly on the wall (Great name Userfly!) You don’t disturb the user and they are having an authentic experience with your site. This by-passes many of my issues with user-testing. This is really Usability Observation.
We aren’t taking them out of the flow – they don’t know they are being watched. It’s like security cameras in a retail store. But we aren’t watching them for shoplifting. We’re watching to see where they go, how they got there, where they click.
My big issue with testing – the thing I can’t get my head around. Is what to test? What questions to ask? Observation is the answer. Observation and testing go hand in hand. Observation leads to exact and specific test questions. Those test questions lead to more observations.
Here is the pitfall: You can’t observe and test at the same time. We’ve talked about this many times, but now I’m having an ‘Inward Singing’ moment. Avoid this pitfall: by observing first. We listen first, correct? So, listen to your users. If you listen, they will tell you what to ask next. Observe first. Test second. repeat. Hmmmm. Or Observe. Change. Test. repeat.
My thoughts get unclear here: But bare with me a sec. Maybe these are the three fundamentals and they each relate and rely on the other. Observation is a form of Measurement. Inspectlet is both Qualitative and Quantitative measurement. The point is that it’s dangerous to mix the elements or try to perform them together.
We tried to remove the observation from the testing in our testing script – by starting with the participant simply using the site. Ben even suggested leaving the room while they complete the tasks. Like: “Here do these tasks and I’ll come back and we can talk about it”. That’s good. But, Inspectlet is better. They don’t know they are being watched. They are thinking about their goals and needs – not being a test subject or providing insight to you, the builder.
Natural users are better than un-natural test subjects.
So, how do you avoid messing up at writing a test question? Start with observation or a measurement. Then specifically ask / test about that measurement. Bounce Rate is a common metric we want to lower. If it’s high, people are leaving your site within 10 secs and are not going deeper into the site. You see it in Google Analytics and you SEE it in the Inspectlet video screencaptures. That’s an observation. Inspectlet would show you this – and more, you can see if the user did anything during those ten seconds. Now, make a change to lower bounce rate – put key content above the fold, make a clear call to action, make text bigger and bolder. And, the final step, make a question and test with a Usabilla type tool – “What are you most interested in on this page?”; “Where would you click if you wanted to do [insert Critical Path step one]?; “Which text would you likely read first?”. These questions test if your change made a difference. Well, you could measure again over time to see if the change made a difference. Or, you could create a Usabilla test to ask about first impressions of the site – or 5 sec test.com or Feng-GUI it.
Okay: Enough rambling. Point that started that digression is good, I feel. Here are my findings:
- Observation is different than testing.
- Both are important and relate to each other.
- Inspectlet is an observation tool – it will provoke questions to test and changes to make.
- Three general types activities in usability or design are: Observe->Change-> Test and can be followed in that order.
A few final thoughts:
Using Inspectlet, I found myself wishing for an intercom button. “Excuse me, website visitor. Why are you scrolling up and down like a madman? ” I realize now, that I want to switch from observer to tester. And, of course, I wanted to make changes. The big insight I had – do something for smaller screen sizes. Could I have seen that in Google Analytics?; yes. Did I know we have 10% ‘small’ screen use?; yes. Did it have a big impact seeing those numbers?; no, not until I SAW it with my own eyes.
We talked about finding users to test in previous posts. Inspectlet [because it is an observation tool] doesn’t have this problem. The users are right there on the site now – right now.
I bet site owners get addicted to watching the videos, just like some are addicted to watching visitor counts. There is data there – actionable data that will bring in more money. Because of that, I think Inspectlet is a great value at 8 bucks a month.
By way of explanation of that digression into the process of design and testing or website revision. I’ve just finished David Zull book on the brain Learning Cycle and I think that’s where those ideas came from. He basically says the brain learns in four stages: Gathering, Reflecting, Creating and testing. Inspectlet is a gathering tool – a sensory tool. Usabilla is a testing tool – an active probing tool. Reflecting might be Analytics – where you integrate the data and decipher patterns. Creating is where you make changes to your design and plan.
Thanks for reading and see you next time!