How to prepare for a technical UX Research interview
I recently had an interview at a company in the U.S. One of the researchers, a Ph.D. in psychology and experimental design, quizzed me on all sorts of technical aspects of UX research.
While I stumbled through answers, I also told her that this was “one of the most interesting interviews I’d had” and I meant it, as the types of questions were challenging me in a way that a “tell me about yourself” question does not.
Here are some of the questions, and how you might correctly answer them.
1. Explain the difference between ordinal, categorical, interval, and ratio data.
This is something I’m sure I learned in Statistics 101 as a freshman in college, or even as early as Stats in high school. However, it’s been over a decade. I certainly didn’t remember. But good fortune had it, I had just started taking the Machine Learning course on Udemy.com, and it happened that one or two days prior, the instructor Frank Kane went into this. Unfortunately, I couldn’t remember the exact answers for which type of data was which, and I stumbled through the question.
What to do instead:
I think the best way to answer the question is to explain at a high-level what those terms are. So for example:
Those are all terms to describe different types of data.
And then explain each one using an example.
Categorical data is data represented by words that have no relational meeting to one another. For example, sections in a grocery store such as: fruits, vegetables, dairy, seafood. You can assign values to the categories, but the values don’t have any mathematical meaning. One number is not better than another.
Ordinal data are words used to represent a direction, or ordering (hence ordinal). For example, they might be used in a survey when you’re asking people “strongly disagree, disagree, agree, strongly agree”. These are words but there is an ordering to the words, and they can be represented by numbers. So a 1 strongly disagree is worse than a 2 or 3 or a 4 strongly agree.
Interval data is when ranges are used to describe a set of data. A common one is income. Please select your annual household income, Below $40,000, $40,000 — $80,000, $80,000 — $120,000, above $120,000. The other important thing to note is that the intervals should be evenly spaced, otherwise, it would be ordinal data.
Ratio data is similar to interval data, but all values must be positive. A good example is height. It is not possible to have a negative height, and you can also multiply and divide ratio data by other ratio data in order to make assertions such as something is “twice as tall”, or “half as tall”.
COIR is the acronym to remember these all. I’m not sure that helps but better to have an acronym than stumble through an answer.
2. How do you ensure a study is “valid”?
A study is valid when the data you collect represents the concept you’re trying to measure. For example, let’s say you are trying to measure the affordability of a product. Your questions should be related to the concept of affordability. If your study is valid, then the questions and data you collected should be able to express a view on affordability. Valid questions are relevant to the topic at hand.
3. How do you ensure a study is “reliable”?
A study is reliable, when you can ensure that it can be repeated and the results should be the same. This is why it’s very important to have “unbiased” questions in a study, in order to ensure that someone else could repeat the study (in theory) and obtain the same results.
4. What types of biases are there?
I knew on Wikipedia there is a list of over 100 different types of biases (who knew there were so many) and I mentioned that in my answer. But I struggled to name and properly define them and give examples relevant to UX research.
In order to answer this question effectively, you want to be able to memorize and give an example of a few types of biases. So you might say:
Anchoring bias — the tendency to rely heavily on usually the first piece of information when making a decision. This can be relevant when forming research questions to participants if you wrote, “So what did you find good or bad about using this mobile app today?” Because you use the word ‘good’ first, you may be biasing them to think of only good things about the app, and their answers may be more positive than if you asked the question in a different way. A more “unbiased” way of asking that question might be, “So what struck you about using the mobile app today?” A question like that doesn’t imply positive or negative thoughts, and leaves it up to the user to determine what he or she felt strongly about in either direction.
Groupthink — this bias is somewhat self-explanatory. Groups desire conformity and may reach a decision without critical evaluation of alternative views. This is a useful bias to be aware of in doing stakeholder interviews, because you may be ignoring individual wants and needs. It’s why I like to deploy a short survey after the group interview sent individually and privately to stakeholders. As researchers, it gives us the individual voices for what could be improved, as well as written answers to key questions, which we can refer to later while we choose methods and tasks.
Loss aversion is another bias. Losing $5 is way worse in someone’s mind than gaining $5. While I’m not sure it’s directly relevant to user research, one of my other interests is in finance. The effects of this are investors might under-optimize risk in their portfolios (being too risk averse) because the fear of losing is far more painful than not gaining. I recommend young investors to invest heavily in diversified small-cap equities, which despite high volatility, generate higher returns than any other major asset class over the long-term.
If you chose the first two biases, anchoring and groupthink, you could come up with a third based on the Wikipedia list that you may be able to connect to one of your interests outside UX.
5. What is the ISO definition for usability?
I knew the ISO definition existed, but didn’t remember it from memory.
This is the ISO 9241 definition.
“The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.”
The key takeaways here is a product can be used by “specified users” with “specified goals” with 1. Effectiveness, 2. Efficiency, 3. Satisfaction, in a specified context or use.
6. How might you quantitatively measure a usability study post-tasks, besides using SUS?
Thinking about the ISO definition above, you can quantitatively measure this by asking users in a written post-study questionnaire:
“How effective was the mobile app in allowing you to complete your tasks?”
[1 not effective at all, 2 slightly effective, 3 somewhat effective, 4 mostly effective, very effective]
“How efficiently did the mobile app allow you to complete your tasks?”
[1 not efficient at all, 2 slightly efficient, 3 somewhat efficient, 4 mostly efficient, 5 very efficient]
“How satisfied were you with the mobile app allowing you to complete your tasks?”
[1 not satisfied at all, 2 slightly satisfied, 3 somewhat satisfied, 4 mostly satisfied, 5 very satisfied]
Now depending on the client, I might use the word “trust” (for financial services company) or “clarity” in place of the words above, to quantitatively measure other more relevant aspects of the experience.
“Did you feel the mobile app was trustworthy?”
“How clear did you feel the mobile app was in allowing you to complete your tasks?”
Other quantitative measures:
Now if they ask how else you might quantitatively measure things in a study — you can refer to things like the SEQ (Single Ease Questionnaire), NASA TLX (Task Load Index), SUPR-Q (Standardized User Experience Percentile Rank Questionnaire). I won’t go into these, but here’s a pretty good article covering them and others.
That’s all that I could recall from the interview. Anyway, some of these questions and answers might be obvious. But for others, it helps to ensure you’ve refreshed your knowledge on these things before you go into a UX Research interview.
