Episode 056: Grant Packard
What difference does it make to say “You” instead of “I”?
In this episode of Podcast or Perish, I talk with Dr. Grant Packard, Associate Professor of Marketing at the Schulich School of Business. He talks about his research on the way saying “I” or “You” when speaking to customers changes how they feel about the interaction. Do they feel acknowledged and heard, or did you make it all about you? Do they feel like you took responsibility for the situation, or did you put the blame on them? Pronouns are short words with long consequences.
Transcript
Cameron: Grant, welcome to the podcast.
Grant: Thanks very much, Cam. Glad to be here.
Cameron: Can I get you to introduce yourself?
Grant: Sure. I'm Grant Packard. I'm an Associate Professor of Marketing here at the Schulich School of Business, and also the Director of our Master of Marketing program.
Cameron: You are studying marketing from a very particular perspective, which is language, how language influences people.
Grant: Language is fundamentally the thing, or one of the things, that are really novel about humans, in that we can share ideas, really complex ideas, by using words.
And to me it's fascinating to understand that complexity. We've got a rich history of people that looked at language kind of qualitatively, but what I do is, I bring kind of new methods and new approaches to that.
Cameron: Okay, we'll get into those methods in a second. I just want to go on the record that I also study language, right?
Grant: Yes. I know.
Cameron: I study accounting as a language and I'm looking at linguistic theory. I'm looking at the social construction of reality through the way that we represent reality and language. One of those things is numeric and financial representation. So accounting is a way for describing reality, and when we describe reality, we help create reality because our experience of, of the world is mediated through a language. That's the approach I take. I'm not sure that that directly aligns with your approach though, so you're looking at the way people talk.
Grant: Yeah, I am looking at how the words people choose to use either kind of represent their own thoughts or they influence others in terms of their perception of the speaker or even their attitudes towards what that person is talking about.
I like to think about... I mean, linguistics and the psychology of language is a huge field with many different kind of approaches to study. But I think of about what you do as kind of the macroeconomics of language, and I kind of do the microeconomics of language, looking at smaller features.
Cameron: So those little interactions between people and word choices and how they influence things.
Grant: Exactly. I mean, looking at the English language, we have these decisions about what verb do we use? We have prepositions and conjunctions that we connect thoughts through. We have pronouns that describe a subject of a sentence and an object (sometimes) of a sentence. And so to me, those little moving parts that we make a lot of decisions about every time we construct a thought to share with somebody and beg the question, do those decisions matter, in terms of how others perceive the speaker or perceive what the speaker's talking about.
Cameron: When you say “matter,” there's a bunch of alarm bells go off here. Like, how do you tell if something matters? What does it mean to matter? Now you're taking, would it be fair to say, a psychology approach?
Grant: Yes.
Cameron: I don't want to put words in your mouth, but …
Grant: Yeah. The social psychology of language is often how I describe the work that I do.
Cameron: Well, that's, to me, two very different things. Social is how people interact. Psychology, when I think psychology, I think what's going on inside somebody's head.
Grant: Social psychology is fundamentally the part of psychology that's about how do we perceive others and how do we want to be perceived. So it's about interpersonal aspects of anything from our emotions to attitudes, to behavioral phenomena between people. And everything that I do when it comes to language is between people.
My very early research is people trying to prove themselves to others through their language: to show that I'm knowledgeable about something, I'm confident, that kind of thing.
And then I look at the other side of that phenomenon. How do people perceive somebody who's trying to prove themselves as confident, or as expert, through their language?
And so both of those dynamics, both sides of a conversation, if you will, are fascinating to me.
Cameron: It's very kind of an interactive understanding of language.
Grant: Very much so.
Cameron: If I'm concerned about how I'm going to be perceived, it... The way that you're describing your research, there has to be a fundamental empathy there. You have to understand that there's a person at the other end of how you're perceived.
Grant: Right, right. The way I started working in this space was language that appears online, in online reviews and social media.
Cameron: Which can be pretty harsh!
Grant: Yeah. And all we have to go on, to judge the source of that language, is the words. We don't have another signal of their age or their job. Or do I have a long relationship with them?
Cameron: There's no affect.
Grant: All we usually have is the words. And so that to me was the perfect context to say, do words matter? And so that was how I started to get into this.
Cameron: You've moved beyond online research?
Grant: Yes.
Cameron: Do you look at personal interactions, face-to-face?
Grant: I have looked to personal, like telephone conversations I've had in real time. First person conversations, movie scripts, legal transcripts, academic articles, customer service in all of its forms: email, chat, in person, phone.
Cameron: Tell me about the movie transcripts.
Grant: In movie transcripts, we were looking at things like the pace of dialogue, how from language you can try to tease apart how fast a story is moving, its circuitousness, and the kind of the ground it covers, if you will. The space, the linguistic space. And so we can essentially kind of quantify that now.
So think about a movie that's really circuitous. For example, what's his name? Uh, Nolan. Christopher Nolan, the director, who tends to take you back and forth in time, where you don't quite know where you're going and you ultimately land someplace.
To another kind of a more traditional, let's say a Steven Spielberg kind of movie, that's a very linear kind of storytelling that tends to play with one single moment of truth or surprise that transforms the experience.
Increasingly, as our tools to look at things like movie scripts or conversations have gotten more sophisticated, we're looking at the dynamics of conversations or of scripts or of songs. How things move over time, and not only how what we says matter, but when we say it.
Or when it's spoken.
Cameron: My sense, culturally, as a person who's reasonably old...
Grant: Yeah. Me too.
Cameron: … certainly compared to our students… is that the pace of media has picked up incredibly. Like everything's chop, chop, chop, chop. Hundreds of cuts in the time that an older movie director would just have the camera on one person's face.
So that fast-paced visual imbrication, overlaying of visual metaphors and glimpses, to get you, I don't know, riled up or something. Or interested, invested in the story.
Grant: My most frequent coauthor, Jonah Berger, studies that a lot, which is called cognitive arousal. This sense that we need a bit of anxiety, anger, agitation, and that emotion is what the new media environment really plays on and really hooks people in. He studies that in the context of virality: why things get shared is just one outcome that we look at, of the new media environment, of these fast paced, fast cut, really short, compelling ideas … if they're successful.
Cameron: I was just looking at a clip from My So-called Life, back in the 1990s. Claire Danes.
Grant: Yeah.
Cameron: And the podcast I was listening to suggested that this was a really interesting way of telling a story. They've got this interaction between these two teenagers in the hallway, but the camera's capturing the reactions and the disappointments and the excitement of other people who are observing these two people connecting.
Grant: Fascinating. Fascinating.
Cameron: You've got that way of telling stories by letting people see into other people's perspectives. Multiple perspectives in the story.
Now, shifting then to your interactive research, looking at the interactions between people, is it normally just two people that you're talking about?
Grant: We call that a dyad, and that's kind of the most common form of conversational analysis. Things get quite complex when you've got more than two people involved. A lot of psychology research looks at, like you mentioned with My So-called Life, the third-party observer versus the second-party observer. The participant, right?
And so both of them can perceive the same person, but you actually have quite different psychology when you're outside the interaction than when you're right in it, in terms of what you look for in the speaker or the actor. What you hear and what you process is quite different. And I always focus on the two participants.
Cameron: I'm going to take us into one of your papers, which is about the use of second-person pronouns in these kinds of interactions. But, as we're arriving at that paper, I just want to talk about your co-author, Jonah.
Grant: Okay.
Cameron: How did you meet, how did your background, his background, bring you together?
Grant: We met at an academic conference when I was still a student. My psych theory is what's known as self presentation or impression management. How people manage impressions with others. He was doing work on...
Cameron: Not relevant to business schools at all!
Grant: Not at all. And he was working on an adjacent kind of context. He calls it identity signaling, which has a very similar theory base.
And so I pitched him this idea that I had, about how people described their experience as a consumer and the fact that you could describe the same thing from my perspective or from somebody else's perspective, and that there might be something about identity signaling or self presentation linked to how you describe.
Saying "I like that restaurant" or saying "You'll like that restaurant" are two very different forms of word of mouth that the same person could say. And I had some ideas about who would tend to say each of those two things. And that's what I pitched him at – I remember it vividly – at a lunch table at a conference, and he said, "Hey, that's super interesting. Let's, let's get to work."
Cameron: So your background and his background are not quite the same. Yours is marketing and psychology?
Grant: We're both in marketing faculty.
Cameron: Oh, okay.
Grant: He's at Wharton and I'm here at Schulich. I would say identity signaling and cognitive arousal are his two main spaces. And he had a burgeoning interest in language and I kind of brought him more and more into language, to where I would say after probably 10 years of working together, he's as expert in that space as I am.
He wrote a book on the subject. He gives me a nice thanks in the acknowledgements. So that's good.
Cameron: Cool. So this paper, the title is lovely: "Thinking of You: How Second Person Pronouns Shape Cultural Success."
There's a lot about pronouns in culture right now. I am very careful in my classes to make sure. I want to use pronouns correctly with students. I invite them to self-identify as they wish, and I will try to fit in with that. That's a big challenge for instructors, to manage the way that you interact with students properly, to respect them as students.
This is not quite the same pronoun issue though.
Grant: No. So those are third-person pronouns, which I don't study, the gendered pronouns in English. I study second-person pronouns and first-person pronouns, which are very important in terms of identifying attribution of responsibility, to blame – well, that's kind of another attribution phenomena – to who's being acted upon (in grammar terms, the object) versus who's the actor.
We make decisions about those things in almost every sentence. And especially in dyadic contexts, or even in general contexts. We make decisions about how we point at, in the second person context, "you." You know, "you" can be the listener, "you" can be the person I'm speaking to right now. We often use second person pronouns to point at who we're speaking to.
Or we use them to, to describe norms, right. We call it the universal or generic "you." You know? "You don't eat lunch before breakfast." And that's not about you in particular. That's about all of us.
And so the "Thinking of you" paper was about the idea that there was a third way second-person pronouns would be used. They're not just to point at who you're speaking to. They don't work just because of their general universality, description of norms. They can just reflect somebody in your own life that you're kind of imagining.
So in song lyrics, we studied this. The idea, under the traditional literature, is if you hear Whitney Houston sing "I Will Always Love You," or Freddie Mercury sing "We Will Rock You," you think that Whitney or Freddie are singing to you.
Cameron: Of course they are!
Grant: Right. You're ...
Cameron: No, not you, me!
Grant: Right, right, right. They're pointing at you, the audience, because that's how "you" is used.
We said that's not how it works. The idea was, when Whitney sings that, I imagine a “you” in my own life. I don't think Whitney loves me, right? And Freddie Mercury – and this would be true in any fiction kind of context – Freddie Mercury, I don't think he wants to rock me. I think he wants me to imagine a "you" that I would rock, right? Like the other sports team, or somebody else, that we might chant in the stands at a hockey game.
And so we took that idea into music first, to say, does the presence of second-person pronouns matter in music? Because we thought, well, singing in the shower, right? That's a way you consume songs. And when you sing it out loud, you're singing it to somebody in your life, right? And you're probably singing it in the shower. Either you don't have a great voice or you're kind of expressing your own emotions in songs. So we found this positive effect of the presence of second-person pronouns in music, in the Billboard charts.
So the more "you" pronouns, more second-person pronouns are in there, the better the song performs in the charts. And then we had to turn... we kind of pulled the language apart to understand, in the field data, which means the songs that we had, thousands and thousands of songs in the Billboard charts, to look at how they used the "you." Was “you” the object or was “you” the subject?
And so if you used the subject, sorry, the object... I specialize in this and I always mix those two things up... that allows you to imagine you're singing to somebody else in your eyes. Right?
Cameron: So I will always love you.
Grant: You. Yeah, if "you" is the subject, it's not as easy to do that. So the "you as subject" songs didn't perform as well as the "you as object" songs.
Cameron: Interesting.
Grant: Yeah.
Cameron: Please tell me that you've managed to find a way to do some shower-based research.
Grant: Of course, singing in the shower. I do it. I imagine a lot of people do it.
Cameron: Yes. But how would you go about collecting data?
Grant: Oh, right. Oh. Yeah, we have some people in the field that do that kind of thing. If we get to talk about replicability, we'll bring that up.
Cameron: Yes. I want to turn to that later. Alright. So this is about how second-person pronouns, "you," are used in… you're talking about cultural success, so is this specifically a marketing focus or what? How would you define success?
Grant: So I think about that project, where second-person pronouns and song lyrics, as words in a product. I think of music and movies and literature as cultural products.
Cameron: I see.
Grant: Essentially. So it might be how you title a movie. It might be the product description on packaging. It might be email marketing, how a service rep writes an email to a customer. You have to think about…
Cameron: So you can measure this with sales, you can measure this with downloads of songs or videos.
Grant: For example, the second person. Using "you" in customer service is negative. It's essentially a blame game. We found, in another project I did, with Sarah Moore and Brett McFaren on these same pronouns, first- and second-person pronouns in customer service, you want to focus on first-person singular pronouns. "I am happy to help you." Like, "Yes I can, I can do that for you.”
You don't want to say, you know, "You've got a problem." "You need to return that."
So basically, what happens is with "I" pronouns, you're taking agency and you're showing empathy because you're personally involved in the situation, the question, the problem. If you're using "you" pronouns, second-person pronouns, you're shifting that attribution to the customer.
So these things are complex, right? You can't just say, universally, "you" is good. It depends on the interaction context.
Cameron: Hmm. I'm trying to figure out ways in which this interaction can go wrong. Did you study how these things go off the rails, or are you just measuring how well things do?
Grant: In, customer service we're typically looking for positive effects. So in that, that example I gave you, the first person was positive, the second person was negative. And in the first person, particularly the singular version of that pronoun is what matters. Because if you say we, the plural first person, you become this, this kind of robot, part of the larger firm entity. You say, you know, "We might have that in back." Right.
Cameron: Right. You're becoming the corporate "we."
Grant: Exactly.
Cameron: Hiding behind that.
Grant: It's the same thing that uniforms often do. They make the employee kind of an impersonal member or agent of the firm, rather than a personal agent of the customer.
Cameron: Interesting. Yeah. Okay.
Grant: So we do, we do look at the positive and negative and it the key is context, right? Right. Context does matter.
Cameron: So this paper, "Thinking of you: How second person pronouns shape cultural success." How do you translate the results of a paper like that into some sort of an impact, either in the business community or perhaps in the classroom?
Grant: What I really focus on is teaching students the importance of their words, the words they choose. That's true whether you're an executive, whether you're in advertising, producing kind of copy in ads, whether you work in customer service, or manufacturing where you have to prove yourself to others.
Because the words you use like are, are quite powerful in all of those contexts.
So I usually, I try to give example simple examples from a variety of contexts about how verb tense can shape whether you seem to be confident about what you're saying or not. Merely because using the past tense seems like something that may no longer be true.
Right. These subtle ideas we have as humans that the past tense means, "Oh, maybe that's not true today," even if, if it's impossible for the reality to change. I give them examples like, you know, have you ever gone into a store and a service employee said, "Is there anything I can do for you today?" 90% of students will raise their hands.
That's a really bad thing for an employee to say because essentially what they're doing is going through the motions that they've been trained to do a hundred times and paying no individual attention to the customer.
And then I show them examples of, what if the employee noticed you were in the jean section, and they say, "Can I help you find some jeans today?" Right away, you know that they're paying attention to you. Right?
So imagine you are giving a speech, you're speaking to your team, in the organization, in whatever kind of capacity. Being concrete with your language can show you're paying attention to their concerns.
So by, by using those specific nouns rather than those broad, abstract, generic nouns. And so I, I tend to give slice-of-life examples and I show some evidence.
I often share my insights with organizations and they try these things out and I hear their failures and their successes, and I bring those back into the classroom, as well, especially with our executive and MBA students who, who know about the importance of experimentation in their organizations and say, you know, you're going to have to maybe try to do these a few different ways to see what works because we can't just tell people talk different. We have to find ways that are kind of intuitive or natural that might change their language, kind of more obliquely.
Cameron: Mm-hmm.
Grant: Again, I can give you an example of that if...
Cameron: Sure.
Grant: I find interesting, when we found this, the first person, singular pronouns, importance in customer service, we ran an experiment with an actual organization. They trained their employees to refer to themselves as "I," right. They, they did a great job for about a week and we, they had a, a neck microphone where they were actually recorded, and after about a week, they stopped doing it.
Because you can't tell people what personal pronouns to use. We don't think that way, right? So what we tried the second time, and what stuck was said, "Imagine instead of working for the company, you don't work for us, you work for the customer. It's just you and them. That's our service philosophy now." That changed their language and it stuck for four weeks because instead of of saying speak different, it was think different about your role, right?
And I think that's more natural for us to do.
Cameron: So it's reconfiguring that relationship with the customer.
Grant: Exactly. Where they're an individual helping the customer rather than a, a person representing we the firm. We the company.
Cameron: Now I can imagine that that kind of example would work pretty well for an undergrad class because they've probably got some experience working retail recently, because those are entry level jobs.
Grant: For sure.
Cameron: It's not going to connect so much to your MBA students or your Executive MBA students, I would think.
Grant: They, especially when they think about how they present themselves to management, when they've got to pitch something to the C-suite or they've got a board or a client that they're presenting things to, I find them really interested, really engaged with these topics.
They can see how these things matter, where there are often things you just don't think about. You know, who thinks about the tense with which they describe the past. When we can often describe the past in the present tense. If we just think about it as something that's true today, I might use the present tense instead of the past.
Cameron: So in that context, then making a pitch to the C-suite, the CEO, the CFO, it's the same mental map that you've got of the relationship that you are, you're trying to understand what is the particular. Issue that this person is trying to resolve?
Grant: Right.
Cameron: Not just, can I help you, but can I help you find some jeans, right?
Yes. You've noticed some specific thing.
Grant: Yeah.
Cameron: So you have to do the same thing with your boss.
Grant: Absolutely. Absolutely. So, and there's, there's quite a bit of work on first person pronoun use and hierarchy in organizations.
Cameron: Ooh, tell me about that.
Grant: That's not my work. That's essentially, so CEOs need to use more second person pronouns and more, more first person plural pronouns.
Cameron: When they're talking.
Grant: They take too much credit. For the organization's success...
Cameron: Really!
Grant: ...Or failure.
Cameron: CEOs take too much credit? I am shocked!
Grant: So there's, there's work in that space. And so I share, you know, my own work and other people's work when I talk about these things, obviously, because they need a kind of a rich view of the contextual factors.
Cameron: Right. Yeah, yeah. Tell me which courses you're actually teaching when you're introducing these concepts.
Grant: I currently teach something called Marketing Dynamics to our Master of Marketing students, and that's a marketing strategy course. So a lot of that is in, how do you frame the competition or government or your own organization conceptually to make decisions about how you're going to respond to changes in the market?
And so my language research comes in a little bit in there. Not a ton. What I've taught to our Kellogg-Schulich Executive MBA students, I had a customer experience design course where the language research was quite central to that, in terms of how you do the, the frontline kind of interaction with consumers, to how you think about language in visual space, in retail environments or in client service environments.
The marketing management class that I teach is kind of the intro to marketing for executive students. I use this for a variety of contexts in terms of how large language models and AI are impacting marketing work and, and work throughout the organization, which is all the work I do in language modeling essentially.
So how they can, how these tools are working and helping them improve, you know, analysis of qualitative content. So customer feedback, client feedback, that sentences and paragraphs. You can now analyze tens of thousands and millions of these instantly and get really, quite rich insights about more than just the sentiment of customers, which is what we did 10 years ago, a really crude kind of thumbs up or thumbs down, to linking those attitudes towards objects in language almost automatically.
So when somebody, you know, says, " I really like this hotel," we just know that they were positive. But they could have talked about many different features, right?
And they, they could have had a negative attitude towards the location, but a positive attitude towards the front desk service and a bit ambivalent about the quality of the room. And we can now tease all those apart, and what's more, through kind of econometric models, determine for each person who spoke about them, just from the language which matters the most.
Right. Is it the thing that you are positive about? The thing that you are negative about? The thing that you are ambivalent about? We can tease that out from the language.
Cameron: Mm-hmm.
Grant: So that's a, a very exciting development for anyone who wants market insight, whether it's consumers, investors, you know, business partners, that kind of thing.
Cameron: Grant, I want to go back to the beginning of your academic career and how you got into it. Because understand this wasn't your first career.
Grant: No, I, I'm a, a proud Canadian now of about 25 years, but before that I was an American in Colorado. I wanted to be in advertising, so I, I got in a very large Chevy Suburban, drove into New York City, wished I did not have a Chevy Suburban in New York City, and struck out to work in, ad agencies.
Offered myself up for free. Had to do that for a couple months then started an illustrious agency career that was not as fancy and as exciting as the TV shows suggested, but it was a lot of fun. But I loved it.
Cameron: It was Mad Men.
Grant: It was,
it was kind of like Mad Men. I, I remember the day that the martini infused lunches disappeared.
It was, it was while I was involved. So...
Cameron: I hope you weren't involved in the crisis that led to that change!
Grant: Oh, yeah. At a small level, because I was junior. You have to be careful right, when you're still kind of building your reputation. But I, I spent a few years in New York and then I went to Montreal where my now wife was.
and I was an MBA student and I was working there part-time as a copywriter. So I was the kind of anglophone copywriter. You know, the, the team there would say, "Take this idea that's in French and make it work for our anglophone kind of consumer base."
And so I did that for a couple years and then came to Toronto where I worked in, in ad agencies and then went what we call in marketing, "client side," for Xcite which only the oldest of listeners would recall was one of the top three search engines in the early days of the Internet, along with Yahoo and Alta Vista.
And when that kind of went belly up in the 2001 crash, I went to Indigo who was quickly confronted by Amazon and the challenges of, of the online retailing market. Worked there.
And I all along kind of had this combination of, I, I was a writer often in my job and, and developed pitches for, for the team. And that was kind of a, a particular skill of mine and was also increasingly doing research in organizations because I loved taking focus group transcripts. I loved taking data, transactional data and figuring out what are people going to buy next.
And gradually made my job more and more about consumer insights until I got to the point where I was like, Hey I want to do this thing full time, and be able to kind of decide what I'm going to do with different kinds of data.
And that led me back to academia, to my PhD.
Cameron: I've got a similar timeframe. I did 20 years in information systems, but my reason for becoming an academic was I was so bored, I was so frustrated. I just wanted something else, something new to do. And everybody in my company was just saying, oh, you know, Cam, you're great at programming. You're great at information systems. Like, just keep doing that like forever.
And I desperately wanted to change. So I had to quit and go back to school in order to make that change.
Grant: Interesting.
Cameron: And find out what else was going on.
Grant: Huh?
Cameron: And, when I walked into my first MBA class, I just knew I was home. I thought, I love being at university, and I've never left.
Now your story is quite different though. You're actually building on some enthusiasm for research in your day job that you just want to translate into something deeper in your PhD.
Grant: Yeah. I mean, I actually knew I loved being in the classroom, so when I was an undergrad, I was lucky enough to kind of co-teach a course. We brought in CEOs.
I brought Shelly Lazarus, who was the CEO of Ogilvy, which is a big ad agency. And came in and basically introduced her and taught students about the ad industry. The professor of that course who kind of managed the student teacher said, you know what, being a professor is not just about teaching.
There's also this thing called research, that's going to be a lot of your job, so figure out if you want to do that first. And so I occasionally taught, I did kind of the part-time lecturing thing throughout my career in industry. And my industry career was, do I like research enough to do the full teaching and research gig that comes with tenure stream kind of research faculty work?
Cameron: Do you?
Grant: Oh, I love it. I love them both. I recently had a sabbatical that was too long and it was kind of during the pandemic, and it was more than the typical year, and I didn't realize when I came back into the classroom how much I missed it. I al... I literally shed a tear. Like, it was so nice.
It's so important having that dynamic of students and you engaging with ideas, right? And seeing the light bulb go on. Seeing people challenge each other with different perspectives, different experiences, is just great. Great part of the job.
Cameron: The feedback loop is shorter in the classroom, right?
Grant: Yes, for sure.
Cameron: Academic work, you could work on a qualitative paper like yours and mine for a couple of years before it is even ready to be submitted to a journal. It could take a year to get through the review process.
Grant: Yeah.
Cameron: And then, you know, does anybody read it? Does anybody start to cite it a few years down the road?
It can be five years before you know if anybody even liked your paper.
Grant: Yes. It's a real challenge. The kind of long lead times, the isolation potential of research. A lot of students don't necessarily realize what professors are doing when they're not in the classroom. It's an incredibly different kind of work, and it, it is indeed kind of tough.
Now, now that I'm involved in editorial teams at journals, I really am pushing towards faster, you know, let's get ideas out there and let's let other people challenge those ideas kind of in real time. Because What happens when somebody spends 2, 3, 5 years in, in an idea, they can get lost in it and they can make some bad decisions about that idea.
Cameron: Yeah. Well, getting ideas out there faster brings us to the next thing I wanted to talk to about. You have a paper from 2014 and you have a couple of papers around this topic. The one I've got is from 2014. The previous paper that we talked about was just you and Jonah.
This one, there are probably about 40 different authors listed here, and this is called, "Investigating variation in replicability."
Can you tell me about what's been characterized as the replicability crisis in scientific research?
Grant: Absolutely. There's this recognition – this particularly comes from kind of the psychology paradigm, and there are people in marketing that are consumer psychologists – that the incentive structure in academia can produce some bad decisions. So in order to succeed as an academic, there's often this sense that you need to publish an idea that's counterintuitive. That's really surprising that's really quite novel, and that's, that's, that's kind of the currency of success in academia, is publishing a certain number of papers in the best journals with ideas that are really surprising and get popular media attention.
Cameron: Innovation.
Grant: Yeah. Right. And so the, the challenge is, what we found is that a lot of that work, I mean, what we want in the scientific method are ideas that you know exactly how they tested this idea. So somebody else can go and test that exact same idea, maybe in the exact same context and the exact same way.
Maybe in a slightly different context, in a different manner. But the core idea should work again, right, if this is real understanding, new understanding of the world.
And what we found in psychology and consumer psychology is that that was not happening, the replicability of ideas, as much as it should.
We have these ideas about confidence intervals and the probability that something will replicate. And we talk about things like 95% confidence. So what we were doing, there were a couple really big theories that people were struggling to replicate and that there were almost entire subfields around things like embodied cognition, framing effects, that people just couldn't get these highly cited, well-known ideas to replicate. They couldn't reproduce them.
So what we set out to do – and there were 36, 36 of us led by a Rick Klein who was a postdoc, and Brian Nosek, and Kate Ratliff, those are probably the three leaders, the other 33 of us were very involved but they deserved the leadership credit – was say, let's have a very systematic... we did what's called a pre-registered kind of report. So we actually described...
Cameron: Explain that. Explain that one. Yeah. Because that's crucial.
Grant: Yeah. So, one of the problems with replicability, we have what are called researcher degrees of freedom. So there's some different things they can do with how they design a study in terms of what they say to, to participants, what the room is like, how they ask a question, who the participants are in terms of their demographic or cultural mix, and all those things can potentially shape your result. And so by pre-registering, what we do is we lay out exactly what we're going to do, exactly how we're going to analyze the data.
You know, another degree of freedom for researchers is how they analyze the data, because there are many different ways you can statistically test something. We lay all that out and say, this is how we're going to do it. And a journal typically says, Okay, we're going to accept that that's the right approach to do it. And they'll, the experts will push back and say, actually we think you should do it this way and, and we'll negotiate the right analysis approach.
Cameron: Do you register your hypothesis as well? You think we're going ... I think we're going to find “X.”
Grant: Exactly. Yes. And, sorry, I should have mentioned that here, it was actually less important because we already knew the hypotheses. So we were pre-registering a set of studies. I can't recall how many there were. There might have been 12 different kind of major studies that had received a lot of attention in the past. And we were going to structure the design of the studies exactly like they did, because they documented their studies well in terms of how they did them. Got the approval and then, and we ran those studies across 36 different labs. And so a typical experiment, you might run it with 50 participants per condition. And so when we say condition, it may be that one group saw a story, a newspaper story about greed. In the background, there was money, an image of money. And in the other condition, in an experiment, they saw the same story, but the background was just gray. There was no money in the background. And so that difference is an example of an experimental manipulation where we want to see just the presence of money behind that story change how people respond to the story.
Cameron: Hmm.
Grant: And that's an example of priming. So this literature where there can be subtle things in the environment that change how you respond to some cue. So that was one of, I think it was 12 different kind of things that we tested across 36 labs.
Another thing is the more participants you have, generally, the more you can deal with the fact that there's variations in how individual people respond to things.
And so we had a very large sample and so we could tease apart some things about gender or age or cultural, you know, the geographic location of the participant. So, ran these studies, all preregistered, ran the analysis and what we found was around a third to 40% of the studies don't replicate.
And these are major, you know, very commonly cited studies that a big literature relies on that people could be using in, in clinical work, in, in therapy, for example, they could be using in their jobs or personal life, that 36 labs could not replicate.
Cameron: At all? Or just like, not consistently?
Grant: Some of them, some of them were what we would call weak directional results, where you get some replications and, and some you don't. And we have tests to kind of determine the skew of that distribution. We did things like we measured the color in the lab, the temperature of the lab, how many other people were in the lab, all kinds of these what we call covariates, other things that could be shaping how you respond.
Cameron: Mm-hmm.
Grant: And what we found is none of those things drove whether a hypothesis could replicate across those 36 labs. It was just the effect itself. Because the common counter argument is, "Well, you know, the way I did it was slightly different than what you did."
Or, "I did it in the summer and you did it in the winter."
And so there are those kind of counter arguments. And so what this – this was called the Many Labs project – that paper you described was kind of the first of several, what we call large scale, many-lab replication attempts that consistently found somewhere in the 30 to 40% of published research is probably questionable in psychology.
Cameron: Questionable in the sense that if you tried to run that experiment again, you would probably not get any useful results out of it.
Grant: That's right. You would not what we call replicate. You wouldn't find the same thing. You would either find, you know, when I used that money priming example, you would either find no difference between the two conditions, whether the money was present in the background or not, or you could find the opposite effect in some cases. And what the field has since found is that it tends to be an individual, where there's a streak of their papers that have this problem.
So there are certain individuals we found, some of whom are no longer in the field, others are still fighting their way through. Some of whom have good arguments, you know, for why the work isn't replicating and test their own prior studies and report failed replications. So it's a big challenge in psychology. And more broadly in other fields, I'm sure as well.
Cameron: Oh, absolutely. Yeah.
Grant: Yeah.
Cameron: So one category of this is, those… you've heard of these situations where it turns out that the data was completely faked.
Grant: Yes.
Cameron: Right?
Grant: Yep.
Cameron: I listen to a podcast called Very Bad Wizards.
Grant: Hmm.
Cameron: And they look at some of these replicability issues in some of their episodes.
And in one case they said like, you know, they did the confidence intervals. And if you actually zoom in on the graphic, all they've done is they stick a letter T at the top and turn it upside down and a T at the bottom to create those markers on the range.
Grant: Yep.
Cameron: For the confidence interval. It's not even generated by statistical software. It's that bad!
Grant: Yeah.
Cameron: And in those kinds of cases, typically what happens is those... well, always what happens if they find that kind of a problem is those papers are formally withdrawn.
Grant: Yes.
Cameron: By the journal.
Grant: Yep.
Cameron: Right?
Yeah. With whatever consequences for the researcher, that's kind of secondary, but the journal will withdraw them completely and say that we will no longer continue to have that as part of our back catalog of research, because it was faulty. Now, in your situation, does anybody care that you couldn't replicate?
Grant: I mean, there are some cases in which the, this kind of work causes either the institution where the author is located or the journal it's published in to challenge that work.
They'll conduct formal investigations. I mean, increasingly today most journals now require some pre-registered study in a paper. So psychology, experiment-based papers, can have often three to eight different experiments, and at least some of those will need to be pre-registered. Your data from the studies needs to be published on these online repository so other people can go and run the analysis on your data.
Cameron: Right, yeah.
Grant: But the sophistication, so there's one kind of famous case at Harvard where they were manipulating the data, moving conditions around in the data. And it was only discovered because another group of people knew how to tap into the metadata of Excel and see when they moved the responses to the other condition by just changing the tags of which condition the participants were in. So there are many different potential ways that someone who I'll say is motivated, or desperate to get tenure or publish a really interesting idea, can kind of play with their data. And increasingly, it's that horse race of trying to come up with methods to make sure that as a field, if we want good science, that we don't incentivize that, and we have ways to deincentivize that kind of behavior.
Cameron: Yeah. Well, as you said the incentives are all around coming up with new ideas, new insights. Right. Yeah. New, new, new, new, new.
Grant: Yeah.
Cameron: Is there a way to succeed as an academic to get tenure, for instance, by just focusing on replication studies? Or is that still not enough?
Grant: It's probably not enough. So there are some journals that now have replication sections where you can publish replication work, that that's kind of really all you're attempting to do.
Cameron: I was going to ask if there's like a Journal of Replication Studies.
Grant: There are a couple that are just about replications and there are many journals that have a section. So you can submit an idea pre-registered. There are some things where even the journal agree to publish it ahead of time before you've even collected the data, because it's an important enough question and that gives, provides the incentive on a risky kind of question to go ahead and invest the effort in testing that.
Cameron: One of the undercurrents of academic research is that only certain kinds of papers see the light of day. Right?
So if I do hundreds of hours of interviews somewhere and just don't seem to find anything interesting in it, it's all just the same, then I can't really write a paper about it, right.
And there might have been something there, but I don't publish a paper saying, I did all these interviews and didn't learn anything.
Right? So that's not something that can get published.
In terms of the, the way that we frame these hypotheses, right. If it doesn't turn out the way that we thought would be interesting to people, we can either, as you've pointed out, it's possible to twist the data so you can make it say what you want.
But in most cases, no, the researcher just says, well, that wasn't an interesting result.
Grant: Yeah, yeah. The, the best case is you get an interesting result that's not quite what you hypothesize, and you can hopefully do further investigation to see if that could be generative of something kind of meaningful and interesting.
Cameron: But is the lack of a statistically significant result, is that ever a publishable paper?
Grant: Rarely. So there are some papers that we call null effect papers, that have been successful.
Cameron: How would that work?
Grant: So an example. It has to be a phenomena where the existence of a null, so the lack of effect, is interesting and is novel. Ah, so I can't recall… Leslie Johns at Harvard published something about Facebook "likes," and I think it was the notion that if you liked something, you would like it more, but the act of clicking the Like button would make you like it more. So if something is a lay belief that people are like, yeah, that's intuitive, and you can say no, that lay theory is not right, that's often when a null, a null finding, can get published.
Cameron: Okay. So when we think we've got a common sense explanation that everybody agrees to, and you're able to show that actually that isn't the way it works.
Grant: There's a fair amount of gender-difference null-effect research, right? Where if we can deal with stereotypes and essentially say there's no empirical support for that stereotype, or that's, or it's weak or it's very conditional, that kind of thing can get published.
So I guess you would say that the lack of an effect, the absence of something, a significant difference needs to be meaningful. It's the challenge, right?
Cameron: Well, it's all about finding meaning in the world.
Grant: Yeah. Yeah. It is, and that's the exciting thing about science, and it's the dangerous thing about science because the public trusts us.
I hope it isn't happening in medicine, in medical research to the extent that it, it was happening, and still happens, likely, in psychology based research.
Cameron: Yeah.
Grant: Yeah.
Cameron: I don't know. I've never had too much interaction on my podcast with. The medical community because my podcast is about academic research and why it matters, and I think everybody understands why medical research matters.
Like that's, that's like not interesting to me. Right? It's these other things that people do in the academic world that I want to find, like, why does that matter?
So if you could, you know, sum it up as we wrap up our discussion today, why does your research matter?
Grant: My research matters because it's, it's essentially about the human, like a fundamental human condition of having to articulate what's going in our brains to others, and then having to decode that. To me it's just a fascinating part of human behavior and something that we only just really got the tools to do around 2010, is when we were able to do that.
And so I got incredibly lucky to say, Hey, we can, we've got like a new decoder key, like you get in the bottom of a cereal box, to look at language in a really kind of new quantitative, deep way and just that kind of exploration, the ability to be on the frontier of trying to figure something like that out, of how language works in, in conversation and persuasion and self presentation.
To me it's really exciting to be able to test ideas and have things hopefully that will be around until after I'm gone. It feels like there's something I've contributed to the world that helps us understand the world a little bit more, even if it's about something as mundane as customer service or song lyrics.
To me it's neat stuff.
Cameron: Those things matter to people.
Grant: For sure. They matter to me. Yeah.
Producer Andrew Micak: Just two quick questions. You said 2010.
Why was 2010 the date everything changed?
Grant: So that was when social media and online reviews when, when kind of the consumer, the human production of language content online exploded.
Andrew: Okay.
Grant: And then that was also the year that a guy named Jamie Pennebaker at University of Texas created an approach to analyze text at scale.
Okay. Which only computer scientists were doing. They didn't really care about the language. Jamie Pennebaker's a psychologist and he said, can we actually, instead of just predicting what's going to happen, can we actually explain and understand psychology through words?
Andrew: Interesting. Another question, what was the moment where you decided you wanted to go to academia? What was the thing that sort of like sparked it? What made you decide, I'm going to go for a PhD?
Grant: It was the excitement of hearing customers, consumers, talk in focus groups more than anything and being kind of behind the glass and trying to understand the emotion of something as mundane as a can of soup or a bank account, and how those things could affect their lives in such, to me, profound ways.
Andrew: One last one. So part of the discussion you mentioned earlier was the I and we discussion in the customer service position, obviously. Do you find that that sort of changes when we get to these app-based companies where they don't technically have employees as their frontline?
Grant: Yeah.
Andrew: Where there, is there a different level of distance?
Cameron: You're talking about Uber?
Andrew: Yeah. Like dealing with DoorDash or Uber Customer service, for example.
Grant: Yeah, there's a big group of people in my field who are now looking at what happens when the agent of the firm is non-human, right?
And so looking at all kinds of context when that agent, we'll call it an AI agent, should try to be human, more human, or should not bother. And we've learned things like functional kind of arguments, functional content or functional products like say you need help with your laptop computer, how to fix something. That's when an AI agent should be more human. When you're dealing with anything that's kind of experiential or, or hedonic, anything that's about being human? So let's say something went horribly wrong at a hotel or at a restaurant, like they spilled something on you. You were upset about it. Or something at your bank. Anything that's more emotional, the AI should be a robot.
So you have what we call persuasion knowledge. The “schemer schema,” we call it in marketing, where when your ears are up for authenticity. So like if I've gone through something emotional, that's when I look to detect authenticity in the person or the thing that I'm speaking to.
And we don't get it from an AI in that context.
But if it's about my computer, if it's about something functional that doesn't really kind of emotionally affect me, great. It can be a human, because I'm not really thinking about whether it's a human or not.
Cameron: I would think that that actually has some, significance for, you know, being an instructor, right? If a student comes in, they're upset about the course, upset about the school, upset about their mark, right? To use “I” words, makes it all about you as the instructor. And you need to give space to let the person feel heard, right?
Grant: Yeah.
Cameron: And express themselves.
Grant: Yeah. It needs to be a bit of both in those contexts. So even in clinical psychology, the listening exercise that you do, right, which is true with a teacher and a student, or an employee and a customer, you need to show that you're personally there for them, but also that you hear them. And so how you use both "you" words and "I" words are very important in that context.
So you have to be careful not to blame with the "you."
Cameron: Absolutely.
Grant: "This is how you feel, but it's not reality." You have to be careful you're not doing that. And you have to use "I" words without saying "I understand exactly what you're saying." You have to be empathetic without kind of taking too much on yourself, you know, assuming that you know exactly what the student, the customer is feeling.
Cameron: Yeah. Cool. Thank you very much, Grant.
Grant: Thank you, Cam.
Cameron: It's been an interesting conversation.
Grant: It's a pleasure to be here.
Dr. Grant Packard (photo credit: Schulich School of Business)
Links
Grant Packard’s faculty webpage
Grant Packard on Google Scholar
Research
“Thinking of You: How Second-Person Pronouns Shape Cultural Success”
“Investigating Variation in Replicability” (Many Labs Project)
Credits
Host and producer: Cameron Graham
Photos: Schulich School of Business
Music: Musicbed
Tools: Squadcast, Descript
Recorded: February 2, 2026
Location: Toronto
