We talk about what we don’t like about Focus Groups (everything) and what we do like about running group sessions. Are they the same? No. Is this just semantics? Also no.
Matt: I don't even call them focus, I call them like group feedback sessions or something like that to make it-- Mathew: So it sounds like what you're saying, Matt, is you don't like to use the F word. Matt: I don't, that's right. It starts with F and ends in C-K. It's feedback. Matt: All right. Matthew: Hey and now you're back in the... Matt: Now I'm back in focus. fantastic, two birds. Matthew: We did the whole show just like this. Matt: Oh, that's what we should do. Oh, I forgot my hand was up that whole time. Matthew: Well, I had a question? Matt: Right. So hey, not so, just hey. Matthew: Just hey. Matt: But we're just talking about group sessions earlier, or focus groups. Matthew: Yeah, right, the F word, yeah, yeah. I'm sorry. Mat: Yeah, I think it'd be a great topic for today's show. Matthew: I think so too because I just did one recently, and you just did one recently. Matt: That's right, and not the same one for a change. Matthew: Although I did wish you were there. Matt: Likewise, it would've been a lot more fun. Matthew: When it was initially proposed to me, I had the reaction of uh, focus group? Mm, no, I don't think that's a good idea, and it turned out it went okay because really it wasn't what most people think of as a classic focus group. Matt: And it was funny because I had a similar experience, and I've talked about this the day it happened where I had a client, prospective new client, contacted me through a referral and we talked, and the first thing they said is, "Yeah, we want some focus groups." Like, "Really, tell me more," and as we got to talking-- Matthew: The price is going up and up. Matt: Right, right, and as I spoke with them, I got an understanding what they were looking for, we morphed it to sort of a focus group, and more like what I call feedback session or group reviews because I know focus groups get a negative reaction from a lot of people. They have a bad vibe around them, and I've been guilty of that without valid reason other than that I know why they're bad. Matthew: Right, which I think is a valid reason. Matt: I guess it is a valid reason-- Matthew: Like a carte blanche valid reason for all group sessions, and that's why the title of this show is group sessions and not focus groups, right? Matt: It's still ingrained in my head that focus group's bad, focus group's bad, but over the past few years especially, I've done more of these group sessions around concepts and gathering feedback, and I've learned techniques to make them not suck and actually be useful within the parameters of what they are and understanding what the limitations are. Anyway, right as I was in the throes of doing my focus groups, or group sessions if you will. I was at a networking event and was talking to someone, and just talking about what we're up to and I said, "Oh, I'm trying these group sessions, focus groups type of thing," and she recoiled, and put this really angry look on her face like, "Oh, you're terrible, why would you do that?" Made me feel slightly bad, but I explained to her why this was a case where I thought they would be good and some of the techniques I was doing, and she kind of said, "Oh, actually, that kind of makes sense. I get what you're doing," but I think there's this natural, yeah, negative connotation. Let's talk about why we think they're bad, or why people think they're bad, more on the practitioner's side versus the client's side, because clients love them. Matthew: Clients do love them. Matt: We hear that over and over. Clients like to say, "Hey, let's just do some focus groups." Matthew: Because the idea is hey, let's talk to 20 people in a day, and then we get feedback from 20 people about anything that we need to know about. Matt: Which is a noble goal, that they want to get feedback from a bunch of people. Matthew: Yeah. Matt: How can you manage that in a productive and efficient way? Matthew: A lot of the times when we are presented with, we'll say opportunities, which is that, and where we get that feeling of, oh, focus groups, they suck, is because my first thought is group thing. Matt: Right, right. Matthew: You get six, 12 people in a room, two people are gonna do all the talking, eight people are gonna be sitting there going... Matt: I think that's valid. I know it's valid, and I think, when I talk to other practitioners, that's what they always come back with, of why focus groups are bad. Oh, you're gonna get all these people, and one person's gonna dominate the conversation, force everyone into their own silo of thinking, so to speak, their own view, but beyond that, there are definitely some other things. You're limited because you have six, eight people sitting in a room. As far as how deep you can get into these topics, into the conversation, because you don't have the time usually or the flexibility to really dive deep into each person's thinking. Matthew: In that way that you could do if it was a one-on-one interview. You've got an hour with a person, single person, versus an hour and-a-half or two hours with eight people, you can't have that level of depth. Matt: Yeah, because you're not gonna spend 15 minutes talking to just one of the participants while the others are sitting there watching. You definitely lose that opportunity. So one of the struggles beyond the group thing and the other is we've mentioned is coordination of facilitation. It's just logistically more difficult to get a group of six people in a certain market segment, which is usually what you would go after, to be available at the same place, day, time, so I think the coordination of the recruiting can cause some people some concern, some facilitators some want to get into this because it does take a little bit of time and a little more effort. Hopefully if you're using an outside recruiting company, which is what I try to do, they make that a lot easier, but if you're doing it yourself, I don't normally like doing recruiting myself anyway. That would be a challenge. Matthew: And overall it's just harder to get six or eight people into a room on your schedule versus when you would do one-on-one interviews, you're working on that person's schedule. It's like, "Oh, I can meet at 6:30 at night on a Thursday," and you say, "Okay, because that's when you can meet, and I want to talk to you," whereas your group session is we need everybody there at 5:30. Matt: Yep, exactly. Mathew: You're gonna be limited by who can show up for that. Matt: Now what do you think about remote group sessions? Matthew: I love them. No, no. I mean we've talked a little bit about that previously where we almost ended up doing a remote group session of many people and then decided to, we ended up with, the most on those calls were four people. They were more of the validating side, right? Matt: Right. Matthew: Rather than the generative side that these focus groups or group sessions are often used to generate ideas rather than get people to react specifically to something that's been created beyond a prototype, like something that's gonna be a real deliverable. I'm hesitant about them as always. Matt: Yes, I would agree. I would say they're definitely not... Matthew: But it went okay, it went okay. Matt: I think the context was different, and in both of our cases, what we're talking about here today is, like you said, more generative, early-phase stuff. I would not do usability tests with a group anyway, but definitely not remote. It'd be a mess. One of the topics we talked about is whether the clients stay anonymous or not, and my feeling is, because I like to be transparent with the participants, that I always encourage, or maybe not encourage, but I always give the client the option, but always would prefer them to not be anonymous. I just think it makes everything more out in the open. I've never had a participant, or at least visibly, say something inappropriate or that you wouldn't want the client to hear anyway. Matthew: Yeah, exactly, and I think that that's, that sometimes the worry that I hear from clients when we do these group sessions, or even when we're doing one-on-one sessions of, "Oh, don't tell them who we are, because then they might not want to tell us the bad things about the stuff we do," and I'm like that's never a problem. Matt: Part of the opening introduction to the sessions, I tell people like, "Hey, here's the client. They're sponsoring this. They're listening in," if they are. Matthew: Or they'll hear about this eventually. Matt: Don't be afraid, don't sugarcoat it. They want to hear the truth, and you know, like any other session, so people generally do not have a problem being negative in any session. Matthew: Right, ultimately if the client's like, "Hey, don't tell 'em who we are," I'm like all right, fine. Matt: In fact, what's happened most recently to me is the client has asked me whether I think they should stay anonymous or not, defer to me, and I just tell them I don't think you need to. Matthew: See, I find that it generates a bit of, "Huh," out of the participants when you say the client wishes to remain anonymous, or we're not gonna name who it is. They're like... To me, it's like it may not be insurmountable, but there's that moment of lack of trust seated there of... Matt: I'll tell you another anecdote around that is I've done research with group sessions with existing clients, or existing customers, excuse me, and they actually appreciate it, knowing who the sponsor is because it's like, "Oh, they actually care," and everyone's like, "Oh yeah, that's really cool that they're actually doing this and they're talking to people," so it can also generate goodwill from your existing customer base, which is not unique to group sessions, but in my examples with my client, I'm like, "You know, you might wanna share this information, because it might make you look good." Matthew: We hinted about this, about you can't deep dive, but it's really limited data that you're getting. Matt: Right. You have to set that expectation with your client as far as what their goals are and really be careful about it when you build your protocol and your discussion guide, like what are the things that you can get reliably and what things are not gonna translate, and it gets you really into more high-level, surface-level stuff. What do people think? What are their feelings? But you're not gonna get down into necessarily the why or, definitely not any behavioral stuff about what they would have done because they can't predict that, so it's really that gut instinct initial feedback, thoughts on flow, those types of things. Matthew: Yeah, and you almost have to, because of the type of feedback you're getting and the type of environment you're in, when you get comments like, "Oh yeah, I would definitely use this," you almost have to be like, "Okay, thanks for saying that. That is not data we can use," or at least that's not data we can use solely on its own. Matt: Yeah, unless that's what they're looking for, is just an initial stick my finger in the air and see what lands. I don't even know what that reference was. Matthew: Would a survey be faster or more appropriate? matt: Almost as useless as a survey. I mean it's someone saying, "I would use this," versus someone saying, "I wouldn't use this." What's that worth to you? Probably not a whole lot, but it might be better than nothing depending on where you are in your product cycle and what you're gonna do with this data. One thing that's great for these sessions, just like surveys, is you kind of understand where else you need to do research, so it's that jumping-off point, like, "Okay, we did this, we got some initial feedback. This seemed to maybe be causing some issues. Let's do another round and dig a little deeper." Matthew: Ideally, again ideally. It should never be the only method you use to . I, also ideally, from a topical perspective, you likely shouldn't do just one and rely on that one group session to drive all your decision-making from that point forward. Matt: And, you know, of course going back to the group thing, we wanna make sure that everyone in the room is speaking, so that's something that, as a facilitator, you're tasked with managing, just going back to more of the logistics or the facilitation steps that you can do is to set those expectations up front and let everyone know what we're doing. Everyone here is gonna have a voice. Everyone's voice is equal, and if you disagree with someone, speak up about it. Matthew: So one of the things that I don't like about it, or I think that a lot of people don't like about it, it can be a lot less useful if you've just got people in the room and they're just talking about something. You know, hey, here's the topic. Let's have a two-hour discussion where we just sit around the table and talk at each other. We want people to talk, obviously, but I think things run much more smoothly when you give them something to do, something to react to, something to build together. The most recent one that I did, there was a part of the group sessions where people were given a question or a prompt, and they spend a little bit of time, three, four minutes, writing down their reaction to it on a piece of paper that was made just for the session to capture that, and then we had a discussion about it, so that... They got some time to really think about it and put it in their context, and then they had a discussion about it, so it was more of an informed cohesive discussion that way. Matt: I did something similar where everyone had time between each area that we were focused on in the group sessions to have their own moment to go ahead and make notes and review individually, so they're not influencing each other. Matthew: Right, right, right. That's a really good point. It's an opportunity to capture that data without having that influence, but then the group, when you bring them back together to talk about it, then it's like you have those ah-ha moments of someone's like, "Oh, I also run into that. I hadn't thought about that, that's a good point," or whatever. Matt: And it can also make it go a little faster, so in some cases where I do that, and I'll pick one person at random and I'll say, "Just go ahead and share what you thought, what you wrote down," and they'll share, and if I see a lot of heads nodding, I'll think okay, so everyone's kind of in agreement. I can put out there to the group, "Did anyone disagree with this? Anyone have different thoughts?" so it kind of condenses the amount of feedback that you're collecting because if everyone agrees, generally speaking, we can capture that and move on, as opposed to spending everyone saying the same thing over and over again. So when do we want to use group sessions? We talked about kind of what their issues are, what their limitations are, but I think you and I both feel that they are useful in some cases, right? Matthew: Definitely, definitely. We talked about efficiency. Mat: Right, and I think that-- Matthew: We don't have a lot of time. Matt: It's valid, you get groups of six people, you can do 30 people over a few days, assuming you do multiple sessions in a day. Mattew: Right. Matt: There's something else we could talk about, but yes, you get a lot of people in a short amount of time. From a cost perspective, they tend to go a little cheaper. Obviously it's less time facilitating, so if you're billing by the hour, that's gonna be cheaper, and you still have to pay a recruiter. You still have to pay incentives. You sometimes have to rent a venue, but it only really requires a conference room. You don't need a fancy usability lab or anything like that. It can be fairly bare bones. I always tell my client like, "You just need a room and a table for six people." Matthew: Right, but it's a cost that will sometimes come up more often in a group thing than it would on the one-on-one. If your one-on-one, if you're going to visit someone at their house or their office or you're meeting someone at a coffee shop or something like that, you don't necessarily have to pay for those places, but if you need to rent a venue or a conference room or something, the one that I just did in Seattle, it was like $400 for two hours, and it's not overly expensive. It's not nothing, depending on the budget consciousness of the client, though it's something to think about, and I also feel very strongly that if you have people in a room for more than an hour and-a-half, you need to be feeding them because you want them to have energy to make it through, and also there's food for you, so you can have energy to make it through. Matt: I agree, good tip. Matthew: And that's a cost, you know. It costs to feed people, and depending on the group size, we're talking around that six to eight people, which I think is something that a single researcher can handle, and to me like six or seven is really as much as you want to have a single person dealing with. The one that I just did in Seattle, we had 12 people, and so it was two researchers. Matt: You're almost running it as two separate sessions, right? Matthew: Yeah. Matt: Just in one room. Matthew: Right, but it was a, even though the cost was higher because there were two of us, we got through 12 people in one evening. Matt: So some of the other benefits of the group sessions is, we mentioned this earlier, is you kind of can scratch the surface and see where else you wanna dig as far as research goes, but you can also look at it as an opportunity to vet your participant pool, so if you bring people in, obviously you're screening them, but when you get them in the room, you might find that people are a little more passionate, vocal, what have you about certain areas of the application or just knowledgeable in certain domains, and so you might decide you want to bring them back later for a deeper dive, so you can use it almost as a further screening process or pre-interview process to say, "Okay, you're really good. We'd like to learn more about you and your experiences." And so another good thing about having multiple people in the room versus a one-on-one is people can build on each other's ideas, so one person might be saying something about what they saw in a prototype or in a screen, and it'll trigger a thought that someone else might have had but maybe they didn't write it down or forgot about, or they just have a different perspective on it, so you do get different opinions, different ideas, and again as long as you make people feel comfortable, which is the facilitator's job, you kind of can foster some really good conversation, not so much debate, though sometimes it can be more like a debate, but a really good building of conversations and thoughts that can come out just really beneficial. Matthew: Yeah, you don't usually get that with the one-on-one sessions. You don't have that, if it's just one other person, you don't have that prompt of, "Oh yeah, yeah, you mentioned that, and that made me think of," because there's no one else there, so... Matt: Right, because I mean realistically, we know fatigue sets in with participants, and if you go for too long, they just, their brain starts to get tired and they shut down, but letting someone else take the load for a little bit and speak and then someone else that maybe was getting tired, then they can speak up again as ideas are sparking other ideas. Matthew: Again, just to reiterate, because I feel like this is where the group sessions can really shine, is it helps you understand areas that you can further research into by the data you collect in the group session. To me, that's like one of the biggest benefits of doing a group session is it really helps you outline where you need to be looking. Matt: So why don't we talk about some tips? Matthew: Tips? Matt: On how on how to facilitate and run these group sessions. We already talked some as far as size and logistics. You wanna make sure, if you have multiple segments that you're researching, you try to get them all similar to each session, so each session should have the same types of users, if possible. It'll just make all of their perspectives the same, so you might have expert users and novice users, and it'd be a little bit, I think, distracting if you have them in one session because they have different levels and abilities, and you don't want the novice users to feel inadequate or lesser than a pro user or power user. In my case, at least in some of my sessions, I want to keep all the very experienced people in one session because they're talking in different lingos and they're-- Matthew: Yeah, because otherwise you won't get those ah-ha moments of, "Oh, I have that too," or-- Matt: Right. Matthew: You want people with relatively, not exact backgrounds, but similar enough backgrounds to the topic that you're looking at and experience with it, that you can have those deeper, as much as you can, those deeper discussions, rather than someone, to your point, a novice who's saying, "I can't relate to that." Matt: We talked about having the participants review materials alone and document their thoughts individually before sharing with the group, which is I, one of the biggest things that differentiates what we're talking about versus a traditional focus group, we are giving people time and space to really collect their own thoughts and form their own opinions before sharing it and potentially influencing other people, so that's pretty huge. We talk about timing and a number of participants. I like to keep it under six. I think you said seven or eight. I think that's valid too. I wouldn't ever go over eight per researcher in any one session. Matthew: Yeah, , I had seven, and it was kind of weird that it was just one more than I was expecting, but I remember saying to the group, "Okay, everybody, we are four minutes behind schedule at this point," and we ended at four minutes after the hour, and I was like, "Is everyone okay with that?" But yeah, I mean if it was only six people, we probably would've ended it a little bit early because of-- Matt: Right, yeah, that one guy. Like any session, you wanna record. I like to use a transcription service, so I record audio. I don't have to worry about taking notes. I just send it off to my service. They get it back to me in a day or two, makes it super-easy. One thing I do do particularly in the group sessions that I think helps the transcription services, if you're using one, is I do introductions before it starts, make sure everyone else say their name. That way whoever's listening to transcribe it, they can at least put a name of the voice, and then as I'm facilitating, especially because I have people listening in a lot of the times, I'll say, "Oh, Matthew, what do you think about that?" so I'll try to prompt with their name so, again, people listening or transcribing, make it easier on them to keep track of who's saying what. Matthew: And of course it helps the people around the table, if people are introducing themselves, rather than just wearing a name tag. Matt: And I do use name tags, mostly for me because I am terrible at names. Matthew: I know I've asked you four times now, but the fifth time's a charm. What's your name again? Matt: Well, you know, it's funny. I was doing one, the most recent one I did, and I had name tags, and this one woman had really long hair that kept covering her name tag, and I just could not remember her name, and once early on I said, "Oh, I'm sorry, I can't see your name tag," and she moved her hair, and we had a laugh about it, but then her hair fell back. Matthew: And you're like, "Damn it." Matt: And I'm not gonna keep asking. I'm just gonna... You have to use eye contact. Matthew: And what do you think? Matt: Right. So name tags are good, but not foolproof. Again, managing the conversation, so setting expectations upfront that everyone is gonna speak and as facilitator, that's your job really, to make sure you're keeping track for each prompt who has spoken and who hasn't, and not always starting with the same person. Matthew: Right, right. Mat: When I do it, I vary at different ends. I have in my head a little map of the table, and so I have a little how I go through the people and the participants, and you haven't spoken, you haven't spoken, so I can keep track, so it's not completely random. Like if it's a long table, I'll go down the left side, and then back up the right side, just to make sure I'm getting everyone. Matthew: Yeah, the most recent one I did, I really tried to break things up a bit, so it wasn't always the same order every time. So the first time I went around the table to my left, and then I went around the table to the right, and then I was very consciously saying, "All right, we're gonna mix it up, and I'm gonna go every other person, and now I'm going to do every other person this way." Matt: Right, yeah, I just wouldn't tell them what I was doing. I would sometimes. I knew what I was doing, but I didn't . They didn't know what pattern I was-- Matthew: I was doing it because it was an evening and it was just a small way to sort of entertain people, just to get that little like, "Oh, is he gonna be able to keep track?" you know, and it broke down when I tried to mentally reverse what we had done in the zig-zaggy order, and I was like, "Nope, I can't do it," but we got everybody, and it made them laugh for a moment, so that was good. Matt: Awesome, yeah, stuff like that is good. And encourage people to have different opinions, have their own opinions. Again, I set that expectation up front that everyone's voice is the same, is equal, and really encourage, I say encourage disagreement, but not that I want people to just pick a fight with someone else, but if you have a different thought, I want them to express it, and I really try to make the whole environment safe and I've had a lot of success with that. I'm sometimes surprised when people come in, you don't know what you're gonna get, but generally people are, again, people are people that will speak their mind. As a facilitator, you make that environment safe for them to do that and you encourage it because it gets you better information, better data. Matthew: Right, we talked about having a group of similar people in the room, but within that group, you still want to hear all the different perspectives that might be there. You don't want to just have the one perspective and have nodding or nothing, you know. Matt: And with this many people, one of the issues that is maybe exacerbated by the group sessions is you are gonna run out of time. Like you mentioned in your recent session having four minutes and being over a little bit. It's tough to rein in one person, let alone six or seven people. Sometimes tangents happen. Sometimes conversations go on. You want to let it go for a certain amount. You want to get those nuggets that people might share, but you do have a track to stay with, and as long as you know your research goals and everything's prioritized, you should be able to be flexible enough to make sure you're hitting your priorities in research, and that's another reason to do multiple sessions, because a lot of times you can't get to it all in the first one, and maybe you flip things around, you shuffle the deck on the subsequent ones so you make sure you're getting full coverage. Matthew: Yeah, you've got a lot more leeway with tangents on a one-on-one session to let someone talk themselves out or though an idea. Even if it's not totally related to what's going on, you allow them to speak. In a group session, you have to allow for that a little bit, but the impact of letting it go on for too long really just ripples through the whole session. You don't necessarily want to cut people off too soon because then you might be training them to not speak, right? They're like, "Well, he doesn't want to hear from me, so I won't," when really you really do wanna hear from them, but you also don't want to hear three minutes on something that's totally unrelated, but it was just an idea that sparked their mind. More recently because I've been able to set aside that focus group label and think of it in terms of this is a group session where we're gonna have group activities, and think of it more like what would happen in a workshop, right? Matt: Yep, exactly what I was gonna say. Matthew: I think that mental shift is I'm very much less immediately against them, and it's more about is this appropriate given the time, the budget, what we're trying to get out of it, where we're at in the lifecycle of the product or service, and if the answer is yes to all those, then yeah, let's do it. Matt: I agree, and it goes back to the ask. Might be a focus group from a client or your stakeholders at your company. That doesn't mean you have to do a traditional focus group. You can switch things up to get better data for what you need. Matthew: Right, all right, hey group session. Matt: Everything I know about group sessions, everyone. Matthew: And now not only are we, but you, gentle viewer, love them as well. Let's call it. Perfect. Matt: All right, good show people, good show.