Don't people act differently because they know someone is watching them?
To an extent, yes, but in a way we can compensate for.
- At the beginning of the interview, we point out the video equipment and tell the respondent about other observers (e.g., in a focus group facility observation room). Inevitably, after the first few minutes they forget about the video and observers.
- We also give respondents a card labeled "I'm Outta Here" or "I Quit" that they can point to or hand to the moderator if they've reached the point where they would leave the website/give up on the product. Having the card visible and available helps remind users that quitting is an option.
- As a rule, users will put somewhat more effort into a task in a usability test than they would "for real." Knowing this, if you see a user attempt a task four times, you can be sure that at home they would have quit after the first or second try. Clearly, the task they are trying to accomplish is a problem.