User Experience (UX) is a broad concept that encompasses many aspects of a user’s interaction with your product and company; namely “how they feel about you.” Usability is more the practical side of things, the main question being answered is “ease of use.” The combination of the two leads to a better understanding of emotional experience and interaction with your brand and hopefully adds to a memorable, positive experience. But, how do you test for both?
Every website, app, or tangible product should go through some degree of UX & usability testing. Since just about everything can fall under this category, we are going to focus on web-based interactions and highlight some main considerations when embarking on advanced UX & usability testing with biometrics. The primary elements of advanced UX & usability will focus on how to approach different modalities, experiments, and considerations for web-based testing.
- How to use Eye Tracking in usability web testing
- How to use Facial Expression Analysis (FEA) in UX
- How to use GSR or Electrodermal Activity (EDA) in UX
- Usability testing with Electromyography and Facial- Electromyography (EMG & fEMG)
At iMotions we of course highly encourage the use of biosensor tools for UX testing, but it should not be the first consideration when developing the scope of your testing. The most important first step to take is to frame your business question as clearly as possible, and then apply the appropriate biometric tool. In short, the research question should always dictate the tool.
Understanding the ‘business impact’ of your business question can help you get a better understanding of how your experimental design can help you reach your goal. Using advanced biometric research to gain insights into your different website designs can help you answer questions such as conversion rates and interaction rates with elements on your website.
When deciding what kind of study will give you the most conclusive answer to your business question, it is important to choose the right hardware and biometric tool that will fit your study best. Below we outline the most popular modalities for web-based UX & usability testing as well as a couple of additional tools to complement your biometric findings. We will go into more detail further below after a brief introduction of the hardware modalities that are generally used for UX & usability testing.
Since web-based products are exclusively screen-based, eye tracking is an almost unavoidable tool for this kind of test. Below are some valuable metrics for eye tracking that can give you some useful information on how users use your website.
Heat Maps: Eye tracking heat maps are color-coded visualizations of where the audience is looking at a given point in time. Visual attention is aggregated across the entire audience and displayed as “heat” overlaid on a stimulus. This heat can help identify what areas of the content draw the most visual attention, and what areas or elements may go missed entirely.
Areas of interest (AOI): AOI’s can be identified to determine how quickly specific elements are noticed/which go missed and how long people spend on different elements.
Navigation paths: Navigation paths outline, on average, the visual navigation of the audience through the stimulus.
Whereas eye tracking can tell you where and for how long a respondent is looking at a screen, facial expression analysis can tell how they likely feel about it.
Facial expressions are often discrete, fleeting events. They are also heavily context-dependent – you might not smile at a website (unless the content is funny), but you might be more likely to furrow your brow if a site is frustrating or confusing.
Facial expressions are reported as a percentage time during a piece of content or a segment within that content when facial expressions are detected above a certain threshold.
Greater time expressing brow furrow may suggest that people are confused by the content, concentrating, or struggling with the assigned task.
An EDA “peak” is a biological indicator that something relevant happened at that moment – an emotional connection was made. EDA Peaks per Minute indicate, on a rate basis, how many emotional events occurred every 60 seconds. The higher the number, the more of an emotional response the users were having while interacting with the content. Since websites can be fairly neutral stimuli in and of themselves, there are certainly going to be studies where EDA might not yield much information, so refer back to your research question to determine whether this tool can be of use for you.
For example, there will probably be strong EDA peaks when evaluating a user’s response to inflammatory content on social media, or for assessing the magnitude of frustration with a design when given a specific task to accomplish (paired with FEA data). However, if you are employing free-form navigation while participants are casually surfing two different designs, you might not see a difference.
While this modality is primarily used in device testing, ergonomics, and human factors, it has its uses in web-based testing. EMG measures the electrical activity of muscles in respondents. If attached to facial muscles (fEMG) it can act as another way of measuring valence (positive or negative reactions) in your respondents and is sometimes preferred over webcam-based FEA due to its sensitivity (at the cost, however, of being more invasive). EMG in more product, device, or environment-based usability testing can be used to measure grip, stress, gait, posture, etc., which can be valuable measuring points, given the aim of your experiment.
To make sure that you are getting as accurate feedback as possible, several non-biometric self-reporting options are available to use to strengthen your findings.
Open-ended question survey: Ask respondents to elaborate on their user experiences. This can also help you conclude more concisely on the respondent’s biometric data. Questions such as; “How did the task I set you make you feel as you were performing it?” will often lead your respondent to give an answer that should be following with the biometric results that the person gave.
Scaling questions: Ask your respondents to rate a specific task from 1-10 (or 1-100). Questions such as; “on a scale from one to ten, how difficult was it to navigate to/submit/log onto X or Y?” these types of questions will give you a good insight into how their physical and psychophysical responses correlate with their answers.
Behavioral Measures: Set up other measurement points to concretize the results gathered from your respondents is always beneficial. You can establish objective annotations such as live-markers, “clicks used” that you mark as a researcher while a participant is performing a task.
Now that you have a brief overview of the possibilities of getting data that gives insights into emotional and practical UX & usability tools, setting up your research to gather a holistic view of the user’s experience requires some well thought-out processes.
- Step 1: Setting up your study respondent pool
- Step 2:Designing your Study
- Step 3:Refining your hypothesis
- Step 4: Aligning your hypothesis with your deliverables
You have your research question and the appropriate modalities you want to use to answer your business question, so now it is time to find your respondents.
Clarifying the respondent group characteristics for your experiment is central to an approved study design. This is important because it is necessary to exclude side effects that could alter the outcomes of your data collection process. When designing your respondent pool, you have to make sure that there is coherence between age, gender, education level, income, etc. Aspects such as life experience and health issues must also be included in your decision process. Simply put, a website aimed at catering to a specific demographic must be UX tested by that same demographic.
There is a consensus that 5-7 participants are the perfect amount for website UX testing when not using biometric equipment. 5-7 participants will find between 75-95% of usability problems present on a website design, so adding more participants will usually just mean you will be wasting your time. It is better to conduct a succession of smaller experiments, with groups of 5, aimed to target the entire website UX flow.
- The first group finds the problems.
- The second group redesigns the issues.
- The third group tests the redesign again.
That way, you have only used 15 (to 21) participants, but you have covered your entire design process.
This consensus does not stand when using biometric measuring tools. The recommended participant pool when conducting eye tracking studies is 30 and above. This is because the versatility of biosensor studies allows for a wider breadth of study objectives to be explored beyond simply finding sticking points on a website design, and given the granularity of the data from biosensor sources, you will need more participants to derive any usable insights from what your testing group is doing.
It is important to remember that not all respondents will give usable feedback, which is why it is a good idea to add 25% more participants than you think you would need. This is also true for the other modalities mentioned above – more is generally better when it comes to biometric research.
Potential Business Use Cases for Website UX Testing
Depending on your business question, there are several ways to practically go about answering it. These are some of the popular methodologies for UX testing:
– Site Navigability: Evaluate how website users navigate the different pages both naturally and with specific tasks.
– Website Comparisons (A/B test): Compare users’ experience across the website designs. Re-exposure to forced stimuli can be used to provide more granular insights.
– Biometric Informed In-Depth Interviews: Allow users to naturally surf while interviewers assess live biometric feed to inform more personal questioning during in-depth interviews.
When setting up your study there are several things to keep in mind. Keeping your study design simple, with few differences to the individual website designs is generally the better way to go. This allows for A/B testing, which is the best way to make sure that priming will not happen.
Good study design example:
As mentioned earlier in this article, having a good and clearly defined study question is very important. An example of a good question could be: “Which website layout makes the path to purchase easier?” This question is good because it lays the foundation for a very concise and simple study design. A design that could look like this:
2 versions of the website, with 2-3 specific differences in layout or architecture
A/B test design means that respondents see either one prototype or the other
Participants are given the same task, to follow through on a shopping journey and purchasing decision
So, why is this a good study design? First of all, having a between-subjects (A/B test) design prevents any priming effects from participants seeing both designs. Controlling for a task, especially with a specific outcome or goal in mind, allows the researcher to directly compare the user journey between website versions. This keeps the number of variables small and it allows the researcher to isolate specific ways to optimize the website.
Narrowing down the stimuli (or what you are exposing your user to) for A/B testing, based on psychophysical reactions, helps you create a positive experience with your product or website among other behavioral observations that can be recorded for quantifiable data.
Less Good Study Design example:
To reiterate the difference between good and less good study design, here is an example of one that does not set the foundation for a concise and simple study.
Again, any study design starts with a question. This time we could ask: “How do users respond to each website layout?” At first glance, this seems like a fine study question to ask, but it comes with significant flaws when followed up like this:
2 different pages with many differing variables across layout and architecture
Respondents see Prototype A, followed by Prototype B, in that order.
Why does a study design like this need improvement? Firstly, having many differences between the two prototypes makes it difficult to isolate what elements are good or bad for the user experience. Secondly, having participants see the same order of websites can introduce priming and other effects on the data.
When you have decided on and confirmed the study design, you can start solidifying more specific hypotheses.
Using the good study design question and your study design framework from earlier “Which website layout makes the path-to-purchase easier?” you can start to formulate some hypotheses that you would like to have confirmed or disproven. In this case, some hypotheses could look like this:
A site design that makes relevant links most visible will make the purchase process easier.
Users will show prolonged engagement with a more effective website layout – as people get bored or disengaged with an inefficient design, engagement will decrease over time.
An inefficient site design will produce more pain points or points of frustration for the user.
The longer the customer journey and the more interactions the user has with the page, the most frustrating the path-to-purchase.
It’s important not only to identify the modality, but also specific metrics you intend to use to answer your business questions.
Using the modalities described at the beginning of this article, we can apply them to the hypotheses posited in the above section.
Eye-Tracking: A site design that makes relevant links most visible will make the purchase process easier. Method/Metrics Solution: 1-3 Areas Of Interest drawn on features relevant to moving the customer journey forward on both websites. Heatmaps to help identify alternate paths.
|Facial Expression Analysis: An inefficient site design will produce more pain points or points of frustration for the user. Method/Metrics Solution: Amount of time spent in Brow Furrow, for websites A versus B as an indicator of negative valence.|
Setups for website UX testing can take several forms. Depending on the scope of your research, this can be more or less hardware heavy, and it can also impact how your study will be set up using iMotions. If you are curious about what possibilities for UX and usability testing setups we can offer at iMotions, please reach out to us and we will help you get the best solution for your research.
The latest feature from iMotions is our Online Data Collection tool. It offers the ability for you to collect data online as well as in the lab, allowing users to interact with your website from home, no matter where they might be in the world. This allows you to organize participant testing without needing people to physically show up in the lab while simultaneously opening up the entire planet in your search for participants.
Contributing Writer: Morten Pedersen
Marketing Specialist at iMotions.