We chat about what to do when a client (or internal stakeholder) asks for, demands, or mandates a particular method—and you know they’re wrong.
Matt: I have headphones pinching on my ears, so I'm going alfresco, or whatever the word is. Matthew: Al dente? Matt: Hey. Matthew: Hey. Matt: We're back. Boy. Matthew: Boy, that was quite a break. Matt: A fall break. Matthew: Fall apple season something... Matt: Yeah, so how did apple picking. Matthew: When are we planning for this to go out? Let's pretend, 2020, yay! Matt: 2030! Matthew: Happy 2030. Matt: We made it, we've been doing this show for 11 years. Matthew: And the asteroid is only six years away now. Matt: Right, oh man I can't wait. Matthew: Boy, it's gonna be good. Got half of the bunker built Matt: Plenty of time to finish. So, we've got a show today, our first show in the new season. Matthew: Right. Matt: And a topic that's been at the top of our list. Maybe it sounds harsh the way I'm gonna word it, but it's when stakeholders dictate research methods. And dictates, a little bit of a loaded word, I realize that, we can address that. Matthew: We could say mandate, we could say-- Matt: I like to say try to influence? Matthew: Influence or really, what is is often, is they come to it having experienced one way of doing something, and say that's the way we're going to do it. Matt: Here's a question for you as we get into this, when you've experience this, what are the methods that you are being presented with? That they're trying to use? Matthew: By and large, the thing that comes the most is focus groups. Matt: Right, followed by? Matthew: Oh, the second one is basically what it's requested is sort of usability testing, but not really. It's basically, hey could you come and sit in the employee lounge, and if anybody's got time, just ask them what they think. Matt: Interesting. Matthew: So it's kind of this, and there'll be some artifacts, so it's sort of this weird mix of usability testing and intercepts on a population that is never going to use the thing that's being built. Matt: Gotcha, yeah. Matthew: And it's like, well we've got all these people wasting time eating lunch, let's get their feedback. Matt: Let's interrupt their lunch and get feedback. Matthew: So those are the two for me. What did you think I was gonna say for my number two? Matt: So I hear-- Matthew: Number one, focus groups obviously. Matt: Focus group for sure, followed by for me, surveys. Matthew: Oh surveys. Matt: Let's do a survey. Not that surveys are always bandwagon, we've talked about that previously. I feel to your initial point is what are people most familiar with? Matthew: Yeah. Matt: Non-researchers usually. Matthew: Surveys, because they've taken surveys or-- Matt: Or they've come from a marketing background. I'm running into that, not that it's an issue, but on two current projects that was the initial approach, hey so we want to run some surveys. Matthew: Right. Matt: And again, not mandating but-- Matthew: Right, this is just as opposed to the ask of we would like to talk to our customers in some fashion about this thing we're thinking about doing or that we've done, please guide us. They come and they say, hey, we would like to buy some focus groups, or we would like to buy a survey. Matt: Right. Matthew: For me the survey doesn't come up quite as often. Matt: Okay. Matthew: And I'm really glad of that, because surveys are really difficult to get anything good out of. Not that people don't do it well, like if you're trying to elicit information that's beyond how do you feel about our brand? Even if you've got that reasonably robust, it still doesn't take into consideration if the person reads something else into your question that you didn't intend. And if you're in person, you could potentially be like, oh they didn't quite get where I was going, or maybe what they took away from what I asked is better than what I was trying to ask. Matt: Lead down a whole survey episode which Matthew: Yeah, sorry, sorry. Matt: I agree with you. I was gonna share a quote that a very close friend of mine says about surveys, surveys are a really easy way to get bad data. Surveys are hard, but again, people often come and they say, hey, so we wanna get a survey, we want to get some data. Or in your case they want to do some intercepts, or they wanna run focus groups. So let's talk about techniques or tactics for redirecting. Matthew: Potentially, you can say, well no we don't do that here, goodbye. Matt: Goodbye and good day sir. Matthew: Good day sir. Matt: Or ma'am. Matthew: No I don't want work, goodbye. I just mimed hanging up a phone, like a corded phone, like who-- Matt: The kids don't know what that is. Matthew: Millennials killing corded phones and rotary phones. The thing that works best for me, and I will say it does not work all the time, is really saying like, what are you hoping to get out of your, and I don't necessarily say it like this but, your quest for information. What are you trying to achieve? We can do x, y, z, there are a lot of different methods we could apply to get information, not all of them are appropriate. And that works okay because then they talk themselves, it is usually the first time that they're voicing what they're trying to achieve and why they chose focus groups. And I kinda let themselves talk themselves out of it. Matt: Right. Matthew: Doesn't always work, but. Matt: It's a good point. I feel like a lot of times I kinda use the same tactic. Where I just ask them, which I do regardless of whether they're kind of trying to guide me to a method or not, but start with the basic questions, and what are you trying to learn, like you were saying. And who do you wanna learn it from and what are you gonna do with that information. All those kind of baseline questions and a lot of times they don't even know the answers to those questions. Which is a good way to also kind of reset the conversation, like okay, let's kind of start over, let's take a step back and really think about the proper way to do this. Fortunately I've been lucky with some of my clients where they might come into it with an approach, but they recognize that they're hiring an outside firm and they're not necessarily the experts, and so they do defer and wanna have that conversation. I feel like, and I've heard from people who work in-house where they don't have that luxury of pushing back as much, or the people that they work with are a little more forceful. They don't feel like they have the authority-- Matthew: Right. Matt: To push back on those types of conversations, which is unfortunate. I think the same tactics can be true, I mean this comes back to communication and negotiation, understanding the other person's background and what are their goals and motivations, and you have yours, and having that conversation. And just kind of making your point, and sometimes you don't win, sometimes you don't get your point across. Matthew: It's weird to me, weird and also not weird that, says the outside consultant coming in, and them as the internal researcher or designer who is capable of doing research, who may have the same level of expertise as us, but the power dynamic with the stakeholder that's asking the question, or making the request, is so different when you're internal. Even though it's like, for us we're like, well, I have to either come to an agreement to do something else, or I have to do what they want in order to a job, or get a project that gets me money. We, being brought in as the expert, regardless of what triggered bringing us in, we have a lot more power to say, hey, let's not do that idea, let's do this idea for x, y, and z, and often get, oh, okay, we'll try that. Matt: And yet, I don't mean in any way to downgrade the abilities of in-house-- Matthew: No, no, no. Matt: They're perfectly, or more capable than we are. Because they know the domain and they know the background in a lot of cases. Yeah, there's like you said, there's a power dynamic which is unfortunate, where they're kind of going up against maybe a product manager or director whose been there for longer than they have or whatever, they get paid more money, they have a bigger office. They feel like they know everything. And yeah, there's not a good way, they can't just say no to a project like we could as outside consultants. Like if we get that vibe from a client, okay, this might not be a good match. But when you're internal you don't have that kind of power unless you're just gonna quit, which-- Matthew: Right. Matt: Isn't likely. Matthew: Or, you know, this comes back down to the whole politics issue, your ability to negotiate. We start with a more equal power dynamic, I think, being external, but I think as an internal person you still have, it depends on the company you're at. You still have the opportunity to have a conversation with someone and say, what outcomes are you trying to achieve? Look to other projects that have gathered information, say hey, look we did this in three weeks or two weeks. We did it in a sprint or whatever, and we got this good data, and we did this other project that's like what you're requesting and we got, eh, it was okay data, but we weren't really confident about whatever the purpose was. Matt: In a healthy company culture, regardless of whether the researcher and the product person or whoever, they're brand new, they're young, they're older, they've got experience or they're inexperienced, there should be a mutual respect from each side as recognizing the experience that the other side is bringing into that conversation. And I feel like that's the missing ingredient in a lot of those cases. Not to harp on culture and all that kind of stuff, and kinda veered off topic. Matthew: Oh, we can harp on culture. Matt: Yeah, I know, but I think that's part of it. I'll also throw in that, they're always done, not always, but, with the best intentions. Like I don't think anyone's trying to sabotage research obviously and get bad data, they're just uniformed or less informed than maybe they should be. Again, deferring to the researcher who should be well informed to make the appropriate recommendations. Matthew: You tell me what you want and I'll tell you that skywriting a survey and having people use Nerf darts to choose their-- Matt: That's a great idea. Matthew: Classic method, nobody uses that anymore. I mean, it's expensive and it doesn't work. Matt: But it's a lot of fun. Matthew: But it's a lot of fun. Matt: Might get you on the news. Matthew: To me, one of the potential decision points of, especially if you're internal, like I don't think this works so much as an external unless you really think you can establish a long term relationship with the client. Usually for internal people, it may be an opportunity to say okay, let's do that survey. I'm gonna bring you along on this journey with me, and I'm going to run this project as transparently for you as possible, so that you see what goes into it, what we come up with and what the outcome is. And then we can have a retrospective at the end, together. I as the researcher could say, well, you know, we got some good data out of this, but we've got a ton of rows in the spreadsheet that we have no idea what to do with because we just couldn't get that level of insight from what we were asking. Matt: Right. Matthew: We could have done that if we were talking to people directly. Matt: And that, I think part of that conversation upfront is here's my preferred method or my recommended method, to do it this other way there are risks. And that's usually a word that gets peoples attention, and I like to use it, maybe, a little bit of a scare tactic, but not really, 'cause they're true. There are risks inherent with this method that you're recommending and here's what they are. Matthew: You know, the method I'm going to recommend instead of the one you want, here are the risk with that, too. It's not like, hey your idea's dumb, and we shouldn't do it, it's, that's not the optimal way to get the information you want. Matt: Always a good conversation to have, to call it out, even if you're recommending something and it might not be perfect to explain why it might not work. Matthew: Yeah, so almost to have a rubric of method. Matt: That word again, Matthew. Matthew: Well I mean, it's because I'm talking with someone this week about coming up with not literally a chart or a rubric, but basically a hey I need to do x and I have these constraints, what are the best methods to go forward to help a team better understand what choices they should be making, or negotiating for, or planning for ahead of time to have the resources just to help the team make better decisions about how to approach research. So we're gonna go through and talk about the current state and all that kind of stuff, but one of the things we're going to talk about, and even before we decided to talk about this topic, I wrote politics. What are the political constraints going to be? Who are the people who are intransigent about decision making and they're like, no, I'm dictating, back to the top of the show where we said dictating, 'cause there's sometimes there are some people who are like, this is what you're going to do. Matt: Right. Boo. Matthew: Boo them. Matt: Yeah, jerks. Matthew: Yeah, you walk into a doctor's office, this is what you're going to do. Matt: Right. Matthew: Oh, okay. Matt: Take out my spleen dammit. Matthew: Not that we're as cool as doctors, it's going to the expert and saying, hey expert, let me tell you what you're going to do. Matt: Respecting each others roles and professions. Matthew: One of the things I wanted to note, and I think we touched on it a little bit, is it's not just the stakeholders that you're talking to who are making these requests that you then have to maneuver to a better outcome. It's also building up those advocates internally. Matt: Absolutely, finding internal support system. Matthew: Yep. Matt: To help you get what you want, basically, but that understands what you're trying to get done, is super helpful. Whether it's your boss, whether it's like you said, someone else just in another department that gets it. Matthew: For me this feels more like an internals only kind of thing, but do the survey. And then maybe also in your own time, if you've got time, do some interviews as well. Run parallel methods for the same topic and the same user groups. Matt: Yeah, I hate to advocate that, but I had the same thought as sometimes you just have to bite the bullet and just try it. It might not work. Matthew: Right. Matt: And then you can explain, well, we talked about the risks and this is what happened, but we can get maybe some data out of this, but we can get more, better data if we just do another round or parallel round or some other method to get better data. Matthew: Right, or if the opportunity is there, do a survey and in that survey you're collecting, hey, would you be willing to talk with us one on one about blah blah blah, and then reaching out to those people to have a more in depth conversation about, hey when you answered B... Matt: Yeah that's become a standard question, all the surveys that I do now is just to collect that. Even if I don't plan on doing follow up research, it helps my clients build that repository, so to speak, of willing- Matthew: Yeah, people are interested, yeah. Matt: Yeah. Matthew: The big thing to do, especially if you don't want to be completely aggravated throughout your entire career, is if you end up having to do the survey, or having to do the focus group. Or having to sit in the employee lunchroom and ask questions that they don't care about, you're going to have to do more work. But it's in pursuit of teaching the people you're working with, there are better ways to gather the information for better outcomes. Matt: Thanks for watching everybody, we'll catch you next time and thanks for watching. Matthew: You can't say thanks for watching twice. Matt: Did I say it twice? Matthew: Yes. Matt: All right, good show people, good show.