- A Paper and Pencil
- Concept Feedback
- Crazy Egg
- Five Second Test
- Feedback Army
- Mechanical Turk
- Open Hallway
- Simple Mouse Tracking
- Google Analytics
- Google Website Optimizer
- Website Grader
1. A Paper and Pencil
Paper and pencil you say? The most powerful and dirt cheap of usability testing tools says I! The reality is using a paper and pencil to draw interfaces, wireframes, cards for card sorts and a host of other usability mechanisms is an extremely fast, extremely effective way to conduct usability testing.
Paper and pencil are amazingly simple to use, communicate quite effectively, are so low cost you probably have them all over the office and home, and are just about as cheap as dirt.
You can’t go wrong using paper and pencil to help conduct early prototype usability testing, it’s a great way to get quick, fast and meaningful results at a rock-bottom price.
Pros – Cheap, fast and extremely effective
Cons – Early design stage testing only, not for use in testing interaction
Concept Feedback was and is designed as a way to gather input and feedback from experts about new designs for marketing or advertising purposes. However, this tool can be used by web site designers and usability researchers to gather information about potential new web site designs, or interfaces.
It works quite simply, you submit your concept to the expert community, and reviewers provide their suggestions, recommendations and input about your design. You then judge the quality of their responses by taking into consideration each reviewer’s quality score, higher scores mean more people consider this reviewer an expert, which means their advice might be worth more.
This community of experts is available free of charge, and because each reviewer can be graded by others it offers a means to determine the quality of each opinion you receive.
ProsGet free expert advice in a very quick manner. In addition, you can follow-up (and/or network) with the reviewers.
Cons – From a usability testing perspective the reviews are not conducting actual tasks (they’re viewing an image), which means interaction feedback is not possible. In addition, there’s no guarantee the reviewer’s opinions reflect the actual user experience once the site is live.
3. Chalkmark by Optimal Workshop
To quote the site: Do people know where to click? Quickly run a test on your UI prototypes to answer any nagging questions about usability.
Chalkmark provides a means of sharing an image with a user to gather feedback on where the user would click to perform a task. From a usability testing perspective, this is the same concept as a reverse card sort, which means the terminology and navigation is tested to ensure users know where to go to accomplish a task.
The method for sharing images is easy, the test image is uploaded to Chalkmark. Next, a survey URL is produced by Chalkmark which usability researchers can share with testers via an email, or on a web site. A researcher provides a brief description of the task that needs to be accomplished, and the tester clicks on the image where they think that task would be. The clicks are recorded by Chalkmark, and a realtime display of a heatmap showing the location of the clicks is provided to the usability researcher. The time each click takes is also recorded. Reports can then be downloaded by the usability researcher as a PDF file.
Pros – Records clicks and displays realtime data. A major advantage is the ability for usability researchers to provide Chalkmark invitations to actual users via email or on a website.
Cons – Unfortunately, researchers are unable to ask participants the ‘why’ of where they clicked, which is a critical component of reverse card sorts.
Pricing – Free account can be used as long as you like, and creates surveys with up to 3 tasks. 30 Day Plan is $109 for unlimited numbers of surveys & tasks. Annual Plan is $559 for unlimited surveys and tasks.
4. ClickHeat by LabsMedia
Another in the usability tools that track where users click, ClickHeat provides an interesting display method of results via heatmaps. But unlike other usability tools, ClickHeat is a free OpenSource tool that can be deployed on your web server. This provides a major advantage, which is actual users will be providing the data that drives the click maps.
For WordPress users, there is a very interesting WordPress ClickHeat Plugin that enables ClickHeat to be integrated with your WordPress website. This provides anyone with the ability to gather real data from actual users about where they click on a website.
Pros – First, ClickHeat is free, so the price is right. Second, ClickHeat tracks real users trying to conduct real tasks on your website, this kind of information is priceless.
Cons – There are rather specific server requirements and a few other restrictions that must be understood. As with other click map tools, there is no way to ask users why they clicked where they did, or what they expected to find by clicking. For large sites with massive amounts of data, there is not currently a way to download database formatted data for analysis.
Another in the click map type recorders, ClickTale offers a very interesting twist. ClickTale is a paid hosted service that tracks user keystrokes, mouse clicks and moves and the time it takes for users to move around a web page.
Single user sessions are saved as a movie with a large round circle around the user’s cursor so it’s easier to see. A nice feature is the ability to show aggregated data in the form of heat maps or as reports. The heat maps display red hot zones where most users spend longer periods, and blue or cold areas where your users spend the least amount of time.
A very nice feature is the Form Analytics tool which displays aggregate form field information. This information includes time of field completion, the number of entries and clicks as well as which form fields have the highest abandonments, or take the longest to complete, or have the most back-tracks due to errors or confusion.
A final nice feature is the ability ClickTale provides to assist I.T. teams with finding and repairing hard to find form bugs.
Pros – All in all ClickTale is a nice recording tool for capturing and analyzing your real user data as your site visitors complete tasks and enter form information. The ability to analyze either heat maps or aggregate data provides additional methods for evaluating the usability of a site or form. The form analytics tools is a very helpful feature, especially for eCommere web sites.
Pricing – Can be somewhat confusing with monthly, 6 month or annual pricing among Free, Blogger, Bronze ($99/mo), Silver ($290/mo), Gold ($790/mo) and Enterprise packages. Each comes with specific support options, number of pageviews, domains tracked, and recording history time. The Free plan is very bare-bones and does not, for example, allow playback of all of the pageviews a user visits during a session (only the first 2).
Clixpy will not track form fields with input type-password but Clixpy will track everything else. It is possible to manually add attributes for Clixpy to not log credit card numbers, but as with all such recording services caution should be used when ensuring privacy and security of sensitive user data.
Pros – The pros of Clixpy include the incredible low cost and the ease of use of the tool and Clixpy web site.
Cons – The cons include the lack of larger scale aggregate reporting that will be required of more frequently visited web sites and the inability to explore with users the “why” of what they did.
CrazyEggs reporting features capture the location of the click on the page, referrer information including search terms (handy for SEO folks too), operating system and related information. Reports can be shared via a read-only link. In addition, you can download the data and use it in Excel or databases.
Pros – Pretty cheap, and has a nice array of data and reports to help you visualize your click traffic.
Cons – As with the other click tracking software, CrazyEgg demonstrates the location of the click, but not why the users decided (or didn’t decide) to click. Clicks are tracked on a single page only, so gathering session click data on multiple pages or multiple hours or days of a single user set of visits isn’t included.
Pricing – CrazyEgg has a 30 day money-back guarantee, is priced on a monthly basis: starting at $9 per month for the basic plan to $19 per month for their Standard plan, and $49 (Plus plan) to $99 (Pro plan). As you move up in plans, you increase the number of trackable visits, and you can track more pages.
Ethnio is not a data usability testing tool, instead it’s a tool usability folks can use to find and recruit real web site users for a live remote test.
Pros – Ethnio solves the problem of trying to find your real web site users to participate in usability testing.
Cons – Enthnio only works if your researchers are actually ready and able to conduct the test. Separate screen sharing services are required to actually run the test.
Feng-GUI is a very interesting usability testing tool in that it does not use real users. Instead, it uses algorithms to simulate a real user, in this case to generate eye tracking studies. The principle of Feng-GUI resolves around their belief that human eye movement can be replicated based on variable on the page, along with typical actions the average human makes.
To use Feng-GUI you upload an image you would like to have eye-tracking conducted on. Feng-GUI then generates eye tracking heat maps using their artificial intelligence algorithms that predict what a human eye would when presented with the image. Overlay heatmaps, hotspots and Gaze Saccades (the path your eye draws as it moves around the page) are presented to the usability researcher.
I like to think of Feng-GUI as being a low cost alternative to more expensive human-based eye tracking studies, but with the knowledge that you’re dealing with algorithms, not people.
Pros – Great way to have a cheap alternative to human eye tracking studies.
Cons – Feng-GUI is predictive only in the extent that the algorithm enables it – meaning you are not dealing with data from your actual users. Images have to be uploaded, so html pages must be screen-captured and there is no interaction with web site functions.
10. Five Second Test by Angry Monkeys Pty Ltd.
Five Second Test is a tool that enables researchers to upload images and have people provide feedback about the image – but the people seeing the image are only provide 5 seconds to take in the image.
The five second test, because it only lasts 5 seconds, is supposed to mimic the process the typical web site visitor uses to determine what the site is about, and whether he or she will stay there, or move on.
Two versions of the five second test are offered, the first is a visual demonstration of the just the page, the other version is a click test where the researcher can ask the user to list 5 things about why they clicked. With five second test a researcher can choose to display his or her image to the random reviewers that visit Five Second Test, or if the image is more sensitive in nature the researcher can invite select viewers to take the test.
It should be noted that Five Second Test does not include the ability to rate a reviewer’s feedback, such as is possible with Concept Feedback, also, reviews have no data about the people (if they selected Random viewers) such as their age, expertise with the web, etc.
Pros – Five Second Test is free, and it utilizes the concept of displaying an image in only 5 seconds to determine what, if anything, viewers will see and or react to.
Cons – For random viewers, there is no ability to judge each reviewer by the quality of their typical answers – nor is there information about the type of person completing the five second study.
11. Feedback Army by Dashnine Media
Another in the reviewer usability testing tools, Feedback Army enables researchers to share URLs with reviewers, who then make comments or answer questions about the web site or design.Â Because Feedback Army enables a URL to be shared, the reviewer can explore the web site, an advantage over Concept Feedback, which only displays an image.
The researcher can post questions about the web site or image, and then capture the responses back. There are no guarantees however about the quality or quantity of responses. To its credit, Feedback Army provides the ability to replace a bad comment with a new comment, and offers refunds if the researcher feels the comments were particularly unhelpful or off-topic.
Researchers can order 10, 25 or 50 responses per study.
Pros – Easy and fast way to gather reviews about web sites or images. Pricing is very economical.
Cons – As with other services, the review quantity and quality can vary, and the researcher does not have the ability to follow-up with reviewers regarding their comments as they are exploring the site.
12. Loop 11
Loop11 is a unique usability testing tool in that it allows unmoderated remote usability testing using actual users. A researcher provides a simple task to a user, for example, finding a particular type of gift book for a relative on a book site, then tracking user interaction. The data is presented via reports of task completion rate, time on task, common fail pages, paths and a nice detailed path analysis for each users.
Loop11 does not require software to be loaded on a web site. As is mentioned on the Loop11 web site, this means remote unmoderated usability testing can be done on your competitor’s web site! Because real users are being tested, Loop11’s results will be accurate, or at least as accurate as the real users are.
With Loop11 a researcher can have up to 1,000 participants per test project. Each test project costs a $350.
Pros – Simple to use with no code required, provides real data from real users, can be used on any web site.
Cons – Unmoderated means not being able to ask the users questions about their decisions, the flat $350 fee might be high for researchers only needing to test 5-10 users.
13. Mechanical Turk
Mechanical Turk is Amazon’s human intelligence tasks service (HITs) expert review group. Reviewers join Mechanical Turk and volunteer to complete tasks. The reviewers are paid for each completed task.
For usability testing, a researcher could post a task of trying to find a book as a gift for a relative on a book site (as an example) and then receive the feedback from the reviewers as to how easy or difficult the task was to complete. As with the other reviewer services, there is no ability to ask the reviewer follow-up questions as they go through the tasks.
Other services mentioned for usability testing use the services of Mechanical Turk, but as mentioned these services do much of the leg-work for the researcher, meaning using Mechanical Turk directly is cheaper, but requires a bit more effort on the part of the researcher.
Pros – Provides fast and low cost reviewer network to gather usability feedback.
Cons – The researcher will be required to put a bit more effort into this service vs. others mentioned, requires an Amazon.com account for processing and payment. Quality will vary depending on the reviewer that chooses to complete the task.
14. Morae by Techsmith
Morae is a complete suite of usability testing tools for the Windows OS (a Mac version of Morae is not available), including a Recorder module which captures all user interaction (including voice, video, keystrokes, mouse movements, screen action, etc.), an Observer module for displaying sessions to remove observers, and the Manager module which is used to analyze, edit and package the resulting usability test.
Morae is sophisticated, and can do an amazing amount of research activities, which means there is a bit of a learning curve to actually using it. It’s not that it’s particular hard, it’s just that it takes time, and without reading the instructions researchers may not be aware of the full capabilities of Morae.
Morae is a software package that is priced at $1,495 for the Bundle of all modules, or $1,295 for the Manager module. The Recorder and Observer modules can be purchased separately at $195 each. Webcams, microphones and high-end video cards are necessary to record users and edit the resulting videos.
Pros – Morae is an entire usability testing lab in a box. Morae has sophisticated capture and analysis tools that make it extremely useful for usability researchers.
Cons – Morae is expensive, although discounts for students or educational institutions are offered. Morae is Windows only, Mac OS is not supported.
15. Open Hallway
Open Hallway is a browser-based screen and audio capture device that enables usability researchers to conduct usability testing sessions remotely, then play them back later for analysis or demonstration.
Because Open Hallway is browser-based, there is no code for the researcher or the user to download.
Open Hallway is a very easy to use. Test recordings are located on OpenHallway’s servers, and can be downloaded for editing and further analysis. Only Premium account customers can download the videos (mp4 format only) for editing purposes.
Videos can be up to 10 minutes long, which might be a problem for some usability testing which might go longer. The makers of OpenHallway suggest that tasks be limited in scope so that the 10 minute video recording length won’t cause a problem.
The number of videos you can record and store depend on the plan you purchase. The videos are stored on OpenHallway servers, so if your account is closed all videos will be deleted.
There are a few minor issues of note with OpenHallway, for example the fairly common set up of using multiple monitors is not supported (the suggestion is to do testing on the primary monitor, which will be recorded). Likewise, high resolutions monitors might cause visibility issues when played back, and audio won’t be captured of the user does not have a microphone. IE 6.0 is not supported.
Pros – A very easy to use and easy to set up browser-based tool for capturing usability sessions remotely using screen capture and audio.
Cons – Only Premium customers can download video, maximum 10 minute recording time, not all monitor set-ups or browsers supported.
Pricing – $49 per month Basic (3 hours of recording storage), $99 per month Plus (10 hours storage) and $199 per month Premium (30 hours storage).
16. Silverback by Clearleft
Silverback is a Mac-based software tool that enables the recording, analysis, editing and playback of usability testing sessions.
Silverback is $49.95, and a certain percentage of profits go to save the Gorillas.
Because of the low cost yet easy ability to record usability testing sessions, Silverback is a good usability testing tool for the office and the road, and completely replaces a more traditional and far more expensive usability testing lab.
Pros – Easy to use, very nice price usability recording software for Macs.
Cons – Sadly, Silverback is Mac only.
17. Simple Mouse Tracking by Luis Leiva
Simple Mouse Tracking is free click tracking software that can be used to capture and analyze the clicks users make while visiting a web page. Simple Mouse Tracking includes the ability to replay a real-time session, and tracks all mouse movements on an html page. Data is captured in a MySQL database for access.
Simple Mouse Tracking, as with the other click tracking usability testing tools, does not allow the researcher to ask questions of the user as he or she navigates the web site. However the ability to track hundreds or thousands of click streams will help a researcher better understand the existing navigation paths on the site.
Pros – It is a free tool with support of most browsers. Easy to use with nice real-time and database analysis of mouse movements and clicks.
Cons – Like the other click-tracking tools, Simple Mouse Tracking will not enable a research to ask the user why they are clicking, or not clicking, on page elements.
18. Usabilla by Usabilla B.V.
Usabilla is a free (beta) unmoderated remote usability tool that enables researchers to collect feedback about a web page or image from users. In addition, Usabilla tracks where users click, which enables both click-tracking information to be captured, as well as qualitative data (such as; “please click on the part of the web page you like best”). Not only is click information provided, but researches can enable notes so that users can share information about their opinions such as why they clicked on a certain button, or what they expect to find next.
Usabilla then provides the researcher data results as visual reports of heatmaps or scatterplots of clicks. Data is also available for download as PDF, CSV, TXT or XML files for further analysis. In this regard, Usabilla can be used for three usability purposes; as a reverse card sort tool, as a click-stream recorder and as a qualitative tool for gathering user opinions and feedback.
Pros – Enables easy to set up and administer remote un-moderated usability testing. Can capture both quantitative and qualitative data. Currently in Beta it is free for usage for up to 500 participants.
Cons – As with other unmoderated usability testing tools does not enable the researcher to ask follow-up questions while the user is conducting the test.
Recordings are stored on the UserFly servers, and are stored for a maximum of 30, 60 or 90 days. Various monthly pricing plans are available which vary depending on number of captures per month and storage time of the captures. The more expensive plans offer a greater number of captures per month, and longer storage time for each capture. As with other unmoderated remote usability tests UserFly does not offer the ability interact with the user as they are going through the web page.
There are a few notes researchers should be mindful of, including the fact that this tool does not work with AJAX calls. However, on a positive note, coupled with a webcam and/or call-in number this can easily become part of a useful moderated usability testing tool.
Pros – Easy to use and inexpensive (or free) screen capture device.
Cons – Captures cannot be permanently stored and aggregate behavioral data is not available. No ability to interact with users as they are being recorded.
Pricing – $200 per month for the Enterprise Plan (10,000 captures per month and 90 day storage, https supported), $50 per month for the Business plan (1,000 captures per month and 60 day storage, https supported), $25 per month Pro plan (1,000 captures per month, 60 day storage), $10 per month Basic plan (100 captures per month, 30 day storage) and Free plan (10 captures per month, 30 day storage).
UserTesting.com is a panel-based usability testing service that provides feedback from Testers regarding questions you ask them about your web site. Included with each test is a Flash video of the tester talking about their interaction with the site as they browse it. In addition to the video, a written summary by the tester explains what they liked or did not like about the site. Tests are normally done within one hour, and thus are quite fast.
Each test costs $29, however there is a pre-pay quantity discount available.
To conduct a test, the researcher enters basic information about the typical user that might use the web site, including gender, age, income and computer expertise. Optional technology requirements (such as requiring only Mac OS systems be tested) can also be added. UserTesting.com then provides at tester that most closely matchs your typical user profile.
A nice feature is the ability to rate each tester, which helps provide the researcher with some quality information about the feedback received from the tester. UserTesting.com does offer the ability for researchers to use their own testers, instead of the panelists, the cost is the same ($29 for each test, users who participate are paid $10 by UserTesting.com).
Pros – Fast and affordable panel-based usability review. Includes video of tester walking through the site, plus written summary.
Cons – No ability to ask follow-up questions while the tester is conducting the test. Panelists may not be an appropriate fit for niche audiences.
21. UserVue by TechSmith
UserVue, by TechSmith (makers of Morae usability testing software) is a web-based remote moderated usability testing tool that enables researchers to connect, interact with and record users remotely. UserVue’s VoIP feature is available in the United States and Canada only.
Unlike other remote usability testing tools, UserVue enables the all-important one-on-one interaction between a usability moderator and a test participant via phone (or chat), and includes the ability for remote observers to watch the test as it happens.
An additional nice feature is UserVue recordings can be analyzed and edited using Morea’s Manager tool, if you have already purchased it. UserVue is a subscription-based model that costs $149 per month for access by a single moderator.
Pros – Easy to use and cheap remote moderated usability testing tool that enables and records one-on-one sessions, including the ability for observers to watch the tests.
Cons – UserVue is not available for participants with dial-up internet connections. VoIP phone connections are available for the United States and Canada only.
Pricing – A single seat subscription is $149 per month.
Honorable Mention – Usability testing tools
Although these technically might not be classified as usability tools by some, each of these three tools provides a wealth of data about a web site that can and should be analyzed by a usability researcher.
22. Google Analytics
Google Analytics is an easy to use, free, and very handy usability testing tool that provides a comprehensive set of web site data tracking and analysis tools. Usability researchers seeking to understand usage information for a web site can use the reports to identify behavior, trends, and any red-flag issues. Data includes user sessions, visits, page views and much more.
Because it’s free and provides a good set of web data, Google Analytics should be used by any web site owner that does not already have a web analytics tool. Usability researchers who want to understand existing user behaviors can use this tool to gather a great amount of usage statistics and reports. A nice feature for eCommerce web sites is the ability to identify conversion events and pages, which will provide an easy to use yet powerful set of data around eRetail lead flow, order-flow and sales statistics.
However, as with all other web analysis tools Google Analytics will not identify the “why” of user behavior (that’s left to follow-up 1-on-1 usability testing sessions).
Pros – Free comprehensive web metrics reporting tool.
Cons – Data does not provide the “why” of user behavior.
Google Website Optimizer is a free A/B and Multivariate testing tool that enables usability testing of various “what if” scenarios. For example, using Google Website Optimizer a usability researcher can test two versions of an online order form, to determine if the usability improvements made to the form really did improve conversion rates.
The way Google Website Optimizer works is simple and easy to set up. The usability researcher can set up experiments, as they are called, in which two different versions of a web site page can run at the same time. HTML code is provided which will be added to your existing web page code. Then, with the test turned on the traffic to that page will be split, 50% of traffic going to page “A,” and 50% to page “B.” After enough visits have been recorded the performance of the two pages can be analyzed and a winner selected using the simple reports that are part of the package.
Because it’s free, easy to use and has real users conducting real tasks Google Website Optimizer is a very handy tool for conducting testing on web sites. As with all web site analysis tools however the “why” of the user behavior may not be known, and further exploration via on-on-one usability testing sessions may be required.
Pros – Free easy to set up and use A/B or Multivariate testing tool.
Cons – Won’t identify the “why” of user behavior.
24. Website Grader by Hubspot
Website Grader is a free and handy SEO usability tool that can help researchers identify issues in a web site that might need to be addressed. Website Grader analyzes 22 key indicators of SEO success, including on-page and off-page factors.
The reason Website Grader is helpful for usability researchers is because usability changes to the structure or content of a web site will impact SEO components. Thus, when making usability changes to a site it’s a good idea to check those changes with Website Grader, or related tools that evaluate SEO health of a web site.
Pros – Free and fast SEO website grader tool.
Cons – SEO factors only, does not provide user behavioral data.
Pricing – Free