We talk dealing with people who are derisive toward research participants and a bit how to handle when participants … let’s just say, aren’t their best selves.
You can watch the episode or listen to it.
Matthew: I was... See dammit. I was trying not to start off by saying so, and I almost said it. Matt: It's a good lead, and I know it's hard to perforate that habit. Matthew: It's the I'm about to talk now. Matt: Right. Matthew: But somebody who I know who's here in Portland posted something about dealing with when you're presenting research whether you're showing videos, or audio, or just talking about what people did. How do you deal with team members if you're internal or I guess client representatives if you're a consultant. How do you deal with them being derisive toward the participants? Matt: I had that issue. Interesting. Matthew: Yeah and I've had that issue a couple of times, but one of the things she notes is they were in disbelief about things that they, the participants, said and did which was illustrated by shocked facial expressions, laughs, jokes, and questioning the participants' honesty. And I'll say that I've had times too where I've seen a participant do something and go... Where I sort of can't help myself raising my eyebrows because it was a very surprising moment. Yeah so I was wondering if I read that and thought if you'd ever had issues with presenting research where the people you're presenting to either acted derisively toward the results or toward the participants ever if you've ever taken clients out with you. Matt: Not derisive. I mean we've it's hard not to Matthew: Maybe derisive is too strong a word but... Matt: I've done interviews where the participant was perhaps kind of awkward or maybe even inappropriate. Matthew: Where the participant is? Matt: Yes. Matthew: Oh okay. Matt: And then jokes perhaps were made after the session like wow can you believe they said X, Y, Z. Matthew: When the participant was actually being rude or something like that? Matt: Right or say something let's say inappropriate or questionable or made somebody uncomfortable. Put it that way or once we leave the participants the one I'm thinking of is an office. Like a business person and once we leave we're in a car driving away then it's wow can you believe they said X, Y, Z? That's pretty surprising. Matthew: Right. Matt: Not so much in a demeaning way I'd like to think but more just in a... I don't know. Matthew: It stood out enough that the team needed to talk about it to some extent kind of thing. Matt: Not like that guy's a big jerk and we're not gonna use this data or whatever. It was just that was kind of odd awkward. I will say it's funny you mentioned this cause I had a conversation with a colleague but a networking person somebody in our field maybe a month or so ago where I don't even know how we got on this subject but they were telling me the story of exactly what you're talking about where they were doing in-lab tests these facility tests. They happen to not be facilitating at this time. Someone else was but the client was so irate with what the participant was doing. They actually barged into the room and was talking to them like how can you not... I forget the exact services but how can you not understand how this works. And they said it was the most humiliating ever aspect of user research they'd ever experienced. They're just humiliated that that happened. Matthew: Yeah. Matt: That's never happened to me personally. Matthew: So I thought of three examples. One was sort of a... In order of magnitude kind of thing. The first one I thought of was we had a participant say someday I hope to make it to all 52 states. And I definitely remember my eyebrows going way up on that. She wasn't looking at me at the time, but I had that moment of how many? And we talked about it afterwards because I had some of the client people there with me. And we were kind of chuckling about 52, but my comment was it definitely is an odd statement but it's not really relevant to... And then is just let go. Not a big deal. Odd little moment but not relevant. And then I've had in showing some video clips of people trying to use the concept prototype that had been put together. One of the people in the client room or on the client side in the room said what an effing idiot and I said to them that person pays your salary and it shut him up in the moment but it made problems later. Matt: Oh is that right? Matthew: Because he was internal so he can make little waves when I'm not there and it made other things problematic in the project later because basically I embarrassed him in front of the group. And then the other one was having someone on-site at a clients house where they got kind of mad at what the client was saying about the product. And I had to ask them out and say leave and don't come back. So I basically fired them from the project or at least that part of the project. So it really got me thinking about like at the beginning of my career I don't think I... Everything I just explained to you is like the past 10 years happened. The previous 10 years I don't really have a little... I would say super fuzzy memory except for one moment where there's a developer pounding on the one-way window. It's not a two-way mirror, it's a one-way window. Anyway. Pounding on the window yelling at the participant saying it's the green button, it's the green button. I remember that clearly, but I didn't say anything, and I realized the longer I go in this career the less patience I have for that crap. I'm totally fine with those moments of 52 states where it's like that was odd but to be mean to the participants is like no. Cause I feel like part of my job is to protect them which is why in reports we don't use their names. Matt: Right. Matthew: You use participant one, participant two especially with the client. They don't get the email addresses or contact information of the participants. I've had that asked for and I'm like no. I don't know. It's a weird... I know that it is not a normal thing, but I think especially for people who are doing the kinda work that we do knowing that it happens and deciding what you're going to do about it, generally speaking, cause you never know exactly how it's gonna manifest itself but deciding what you're going to do about it before it happens I think is something to think about. Matt: Yeah. I think being mentally prepared for those types of situations it's not something you normally think about cause you just don't expect it. Even myself, I can't think of... I can't put my finger on any situation that was that bad like you described. You probably catched me off guard to be honest. I don't have to think like... And to your point assess a situation like how do I want to handle this. Do I want to call this guy out in a meeting or do I want to pull him aside privately afterwards and say that's not appropriate. We want to be more supportive and understanding. Matthew: Yeah and that's one of the things that I noted in my response to the person who @ me is... The biggest thing that I've feel I've learned in my career is if you really wanna change people, you can't do it in a group setting. You have to do it one-on-one. Matt: Absolutely true. Matthew: And it's... Takes more work. You can't use the emotion of the moment to be like zing! Matt: And a jerk reaction to just shut em up. Matthew: I know there are classes available. I remember taking a class when I started my first job in difficult conversations. I don't know if it's that kind of training or if it's... Just falls under planning for research because we often will plan what to do if things go wrong in the field or maybe not to the exact minute detail but certainly a general understanding of: Hey we don't feel comfortable in this setting. Let's wrap up and leave. And we all know beforehand that that's what we're going to do. Matt: Cause I feel like... I know what I've done in the past, especially when I got a new client, is I have a sheet of do's and dont's for observation. Matthew: Yeah, yeah. Matt: And I can see it tying into that. Here's... The general how to ask questions and who's leading and stuff like that and there can also be, I don't have it in there now, but I could see adding something like: Here's expected behaviors or attitudes towards the participants. Like I said if it's an uncomfortable situation, we follow my lead too and early. Matthew: I have an observer etiquette document that I've-- Matt: Which everyone should have. If you're leading a research with clients then you need to have that. Matthew: Yeah, but now I'm thinking based off of Amy, her question on LinkedIn, and our conversation about @ing. It's not just observer tips when you're in the field or in the lab or whatever. When you're viewing it live, but it's also treating the results of the research with respect as well. Matt: I think that's a good idea. I think I'm going to do that too. Matthew: Alright. Matt: Alright! Matthew: We'll joint publish something and it'll be a white paper and people will download it and change the world. Cause that's what white papers Matt: What font are we using? Matthew: Well, I don't know. Times New Roman? And I'm thinking of like three spelling mistakes? Matt: Anything but-- Matthew: See people can find them and feel like they're helping. They're contributing. Matt: Right, right. Matthew: That is often when I am doing report walk-throughs. I used to do this internally is I would leave, purposely, one or two mistakes in there so that people could say oh hey that's spelled wrong and I'd be like oh thank you for pointing that out. They feel like they said something so they don't have to come up with something to say. Matt: Interesting psychology. Okay. Matthew: Because I feel like if you don't give people something to contribute in those meetings, sometimes they're just gonna find some thread and just start pulling at it so that they can be seen as contributing and smart. When they should just sit back and--