Tracy (00:44): Hey, experiment Nation, it's Tracy and I'm with, he's honestly a little bit of a blast from my past and I'm really excited to record a conversation with him. His name is Joe Fitzpatrick and he might also be known as the ad scientist in some circles. So, thanks for joining us, Joe.

​

Joe (01:03): Hey folks. Thanks for having me, Tracy.

​

Tracy (01:06): Of course. I am so excited to have you on here. This kind of comes all full circle because you are the first person to ever to tell me that sample size requirements are a thing in AB testing. I remember you telling me this and it was like my whole world fell apart. I thought I was doing everything right. I was like, I'm killing it. I'm rocking this, and it just changed everything. So thank you for ruining my life.

 

Joe (01:36): I'm glad that I going to be a part of your journey to this, what you're now and all. Yes, absolutely.

 

Tracy (01:45): Well, you've also had quite a bit of a journey since we last worked together, so you and I both worked together at a fun little startup called Willful, and I was more on the conversion optimization side. You are more on the performance marketing or paid acquisition side, both very complimentary disciplines. Yeah. You've kind of gone off on your own and really done a lot since then. So can you tell our listeners a bit about what you've been up to and why you started Ad Scientist?

 

Joe (02:20): Yeah, so I mean, I've been in the digital marketing/performance marketing space for well over a decade now. And every time I've joined a company and I've gone through the process and got quicker at going through it, and on the side, I've been consulting, freelancing here and there with different startups, and eventually it just got to the point where I wanted something that was a sort of more a way to meet more companies quicker. I feel like getting under the hood, seeing what's going on with different companies is the thing that I most, so rather than spending my time with one company, starting my own agency to have the ability to work with lots of different companies that maybe wouldn't be able to have a dedicated performance media person or a marketing strategist, and being able to just to see all the different startups around Toronto and Canada that I've kind of talked with since I've launched and in month three of ad scientists going live. And yeah, it's been worth it just to have a company that I can say that it's bigger than me. I can invite people to come in, collaborate with myself on, and yeah, it's exciting stuff.

 

Tracy (03:47): Yeah, absolutely. I think we both went, I self-employed kind around the same time, and it was so exciting kind of hitting this milestone at the same time as you. And one thing that I noticed about you as a performance marketer that I have not necessarily seen in other performance marketers is that you see a role for experimentation and testing, but in a rigorous kind of way, hence why you informed we have sample size requirements. I find that a lot of performance marketers just fly by the seat of their pants and things like statistics just kind of take a backseat. And I know that data is a really big part of what you do in performance marketing.

 

Joe (04:30): Yeah, yeah. It's like the way I look at it, data has so much of a story to tell, and from my first job in marketing, I've been working with C-suite and there was always questions, questions, questions, and nobody seemed to have an answer. And then I kind of quickly realized, hey, you have a wealth of data within this company. It's not structured in a way that you can easily find answers, but if we drill in there, the answers to those questions that every company has, it's there or it's in there, it's somewhere in the corners of the internet that you can get that marketplace information or audience insight. So that's really what led me to be very focused on data. And I think that if you can use data to inform your decision-making, even if the data is sparse, even the process of drilling into data and trying to find the answer clarifies the way you think about the questions you have. And rather than it being a very simple, broad question, you might try to refine what you're trying to ask yourself and what the business needs to understand. And that journey that you go on whenever you're interrogating data and trying to find data is, it is such a rewarding journey for me. So yeah,

 

Tracy (05:59): I love it. Yeah, I can relate to that for sure. Sometimes you come across an issue on why does conversion performance suck, and then you do a bit more digging, you actually do some user research and you start to get answers and you can piece things together. So I'm sure it's very rewarding in that type of way. Speaking of rewarding, I'm going to take the flip side of that and say, I'm sure you come across a lot of reasons why ads don't perform. What do you think are the biggest reasons why that happens?

 

Joe (06:36): So there's a lot of different reasons. Every company I've worked with, there's probably been a unique blend of very similar things. Lack of insight into the audience is probably the biggest one that informs so many other decisions. Whenever I join a company, usually I'll talk to the founder, the marketing team, whoever's there that's dealing day in there with the customers, and I'll all have assumptions about who their customer is. But then I will take that and be like, okay, this is the running theory of who the customer is. Now let's try to prove that's the case. I won't lie, I am not going to say it's a hundred percent of the time it's wrong, but it's not a hundred percent if I say that.

 

Joe (07:34): So that's one of the biggest things that stands in the way of companies having successful performance marketing is that there's assumptions made because maybe that's what the product or the service was initially intended for, but who actually adopts the product or service isn't necessarily the people you intended for. There can be a lot of happy mistakes along the way. So one of the first things, so understanding the audience and then from there, the creative is so important, but the creative should come from a deep understanding of the audience and what are their pain points, what do they value? And so if you don't understand the audience, you end up with this kind of bland messaging. And the other side is people think that you ask companies who are you trying to reach with your advertising? And the question is, oh, everybody. Everybody's our audience.

 

Joe (08:37): You'd be crazy not to buy our product or service. The minute you're born, you should have our product. And that's just not the case. So really probably I'd say there's a laundry list of the technical implementation, tracking, all that kind of stuff that goes into performance marketing. And you know this with CRO as well, there's a million little cuts that you could have in how you track and view your data that could set you up for failure in the future. But I think maybe the two biggest defenders are that too wide a scope and lack of understanding and also not challenging your thought process and not trying to evolve your thought process on who your audience is over time, which is I think a big part of experimentation and trying to narrow in and find new audiences and stuff out there. But yeah, a hybrid of those. And once you have both of those too wide and the lack of understanding the copy sucks, the targeting sucks, nothing good can come out of this. And maybe you might get lucky and you have a really good ad one time, but if you don't understand who your audience are and where they are, then you end up with a sort of like, well, it was a really good ad, but we don't know why because you can't draw a line. And it was like, oh, well, it worked because of this reason. So

 

Tracy (10:15): How do you figure out who the audience is? How do you learn about them, get that insight on them?

 

Joe (10:23): So the first place I go to is I talk to the big stakeholders. Whenever I go into a company like the founder, the product team, the people that I usually get the biggest insight from is customer support. Those are the people that deal with them day in, day out. So a lot of chatting to them, getting their buy-in to kind of work with me to, because it's all one big ecosystem. So if you can work together with those departments, everything kind of clicks better, but listen to a lot of their calls, reading reviews, if your company has reviews, doing all that kind of research on the audience. And then after that, it's looking at the company's data as I mentioned before. So you have most companies, startups, maybe they're limited by the internal data that they have, but there's ways to supplement that with external data competitors, all that kind of research there.

 

Joe (11:33): But the most valuable is often the internal data and internal data could be the transcript stuff like a Zendesk or something like there, doing a kind of analysis on what are the terms, what is the terminology my audience use, what are the big pin points, what are they disappointed by in our product or feature or whatever. And drilling into that and understanding how to connect your CS data and your sales data and all of that and bring it into one place and look at it, and again, interrogated it. But also, again, you're starting with the idea that was imbued to you from the owners and the people within the company, but you're always trying to challenge that. That's your like, well, is this correct? Is this what the data is telling me? And if so, great. If not, why? Just starting that sort of process. Usually my audit process is pretty thorough and usually comes out with quite a small novella of information about the customer and about the company. That's usually beneficial to the companies themselves, but it sets up your performance marketing to be much more effective from the get-go, because you can start off with a, okay, this is what I know and this is what I think I know, and then let's structure our performance marketing to try to help us solve this. Right? Yeah. So that's my kind of take on it.

 

Tracy (13:18): I love that you have voice of customer research in your process as part of your audit. I think a lot of people skip that step because it's easier to just ask the founder, who's your customer? They tell you they want to be their customer or what you said, the person who has historically been their customer, but this may be outdated information and you go into Google Analytics, do a little, oh, where is the dropoff happening? And that's kind of about it. That's at least what I've seen with other performance marketers. I think also this is why we worked so well together when we were both that willful. I remember we were both always hounding Courtney, our customer support manager, also shout out to Courtney.

 

Joe (14:04): Hey, if she's listening,

 

Tracy (14:06): We would always be asking her these questions about the customer. And then I do remember a few points in which we were sharing insights on the customer together because conversion and performance marketing go so hand in hand. We're finding out language that customers use because you would want to use that for keywords. I would want to use that for landing page copy. All these different considerations that I think maybe gets missed in performance marketing school

 

Joe (14:36): For sure. And I think whenever you've got performance marketing and CRO working so closely together and aligned, it's really powerful because CRO makes performance marketing work better and performance marketing can contribute stuff that helps CRO scale and move faster and insights and understanding of audience targeting and stuff. Again, all the marketing, all of the company is one big ecosystem. They're not disconnected, they're not siloed, but it's where those handshakes happen. The customer passes from performance marketing to CRO to email and then the converter, whatever it is. It's making sure those handshakes are really solid and you know what that information has been passed through is and why it's been passed through. Totally. Again, we're working with you and Willful. That was such a great resource to have that so few companies really do have is that person who's owning the landing pages and the website and asking why is this this? Is there a better way to do things? And having that sort of championing the audience and the customer and really trying to improve their journey and the value that they get. And that's a big plus for any performance marketer to have that.

 

Tracy (16:10): Totally. I'm sure it's also, at least what I've found is it's so easy to work with founders who are open to accepting that their ideal customer is not actually who their current customer is, and they want to double down on who's already making the most of what the product is. And I think Aaron and Kevin, the founders of Willful were really, they were always open to new insight. They were not kind of stuck in their ways in, oh, this is our one customer. They always wanted to know how can we reach more people and who are these people?

 

Joe (16:46): So I'm sure that helps. I mean, sharing those kinds of insights where there was a segment that a new segment would be targeting or something, I was always really gladly accepted. There are founders who in the first year of their business, like, oh, all of our customers are early adopters. I guess our customer is early adopters. And it's like, no, that's not. But yeah, no, that's like having a company that's willing to change its thinking about who its customers are also makes it a big plus because there's a difference between who you want your customer to be and who your customer is. And yeah, I think that's just one of those realities that every company has to meet at some point and get over. Y

 

Tracy (17:40): Totally. Well, on the flip side of that, what do you think people get wrong about performance marketing?

 

Joe (17:50): So there's a couple of, I think that when people hear performance marketing, they think numbers. They think it's like Excel spreadsheet, I'll give you X budget, this is a CPC, this is the conversion rate ta, here is the output of that. And it's not accounting that there is. I think performance marketing is very much about out results, and I think the performance marketing has also had the change because of a lot of the updates with how much data you can get on a customer and what sort of targeting you can do. It's not 2016. It has changed and I think for the better, because I think first of all, a lot of the targeting you could do in 2016 was kind of creepy.

 

Joe (18:56): That's kind of creepy level of targeting you can do and people, but I think that what, but the idea that I can just put in this formula and if I want 10 x my customer base, I just 10 x my budget and that's how it works. There's so much more that goes into it. There is the creative element that I really think that has had to come out more as people get more blank to advertising, like the performance marketing ads, that direct response advertising that was so prevalent in performance marketing even four years ago has had to change. You have to stand out. So there has to be a creative element there. It does play a bigger part.

 

Joe (19:49): There's a lot of understanding the channels that you're using with performance marketing as well. It's not, and this is one of the biggest things I think educating clients on is just because two platforms are paid, social doesn't mean they work the same or doesn't mean that they should have the same, oh, we spend 5,000 on this or 50,000 on this. It'll have the exact same impact when we spend it here. That's not how it works. That's probably a big misconception. So I think that the inherent knowledge of the platforms is a big part of it. There's not one size fits all for performance marketing. And then one of the things that, I guess the biggest thing that always troubled me, performance marketing. I worked in a performance marketing agency when I was younger and I would be on calls with brand agencies and very much the thought process is that performance marketing and brand are two, oil and water, the two don't mix.

 

Joe (21:05): And the things that you would do for brand don't have an impact on performance. That's not the case. The things you can do that are common sense that you would do to boost the brand trust and brand perception within a market has an impact on your performance marketing. If you aren't collecting customer reviews eventually, once you're showing your customer reviews on your Google ads, that has a huge benefit to your Google search. Being able to share customer testimonials and reviews, huge benefit whenever, even if you are not spend your Google Review score on your ads, people usually have a very similar search pattern where they'll see an ad, they'll go, is this company valid? I'll just type in the company name. Oh, there's maybe like 200 reviews here. Amazing. I will trust this. And maybe they don't convert right then. But having that sort of third party check and then next time they see your ad, they'll convert. That's like a three part, three step process. But for some reason, whenever you're in performance marketing, the expectation is like, no, you just deal with the ads, the ad creative, the data, the numbers, but don't suggest let's do something to boost our brand trust. And those sort of have a good creative strategy that links in with maybe brand and other things like performance marketing is one arm off your marketing and should really link in with everything else. I think that's a big thing that companies kind of miss whenever they talk about performance marketing.

 

Tracy (22:56): Yeah, I think that's a really good call out. I see the exact same almost kind of adversarial relationship between brand and performance marketing or conversion optimization especially. And I don't know if you also deal with this too, but especially when it comes to copywriting, sometimes brand wants to use language that is creative and makes sense for members of the team internally, but to the customer, it's like, I don't know what that means. Tell me what this is. I have no clue what I'm even looking at right now. I see that a lot in CRO. I don't know if you see the exact same thing with performance as

 

Joe (23:38): Well. Whenever I am doing audits of ad accounts or even coming in to a new team, whenever you're hearing their copy for the first time, you're reading their website for the first time with new eyes, and I'll actually do this whenever somebody joins any team of mine or is a new employer, I'll be like, can you read this and tell me is this internal space, is this jargon that we have made up and talked about in 50 different meetings to make this page or this product, but makes no sense to the lay person? Yes.

 

Tracy (24:20): And that's why reviews and the customer support tickets are so important. You might be selling something by a certain name and then people have no clue what that even means, but you have the answer

 

Joe (24:31): Hidden there. Yeah, it's there. Yeah, they'll describe it maybe not in such a, there's no pros going on there, but they'll describe it as they see it or how they internalize what your product is. And it benefits you to like, okay, well, if I want to reach more of this customer who I know has used our product or service, I should use that verbiage to hit home what we do. There is a point where the two should meet and I think mix and mingle. And when you're writing ads, good ads, sometimes being too clever is sort of a hindrance to yourself. You have to ask yourself, you write something really snappy and sharp and you're like, yeah, but is that true to my customer? No. You kind of have to swallow that a little bit and you'll be like, ah, I'm so proud of that copy. But it's just not what the company needs right now. So yeah.

 

Tracy (25:29): Yeah, ego really does not serve us well in our lines of work. We really have to develop some thick skin and be like, no, I was wrong. It's fine. And just kind of move past it. But speaking of, we've been talking about assumptions. We've been talking about just performance marketing in general is a very experimental activity. You have to try a lot of different things and see what sticks. Where do you find experimentation plays a role in your performance marketing practice?

 

Joe (26:06): Experimentation is I think the lifeblood of performance marketing. The thing that I see that companies struggle with is the fact that they find something that works. They do it, they do it to death, and then when they're doing it, and they can only do that and they've only spent their money doing that, and then whenever that stops working or stops, hits a ceiling, they're like, we're next. And they don't know. And then they're really struggling to hit that next goal because they haven't experimented, right?

 

Joe (27:00): And even if it wasn't about finding new sources of revenue or whatever it is, experimentation should be the way you get better with your core audience as well. What I was saying with finding who your customer actually is, that's experimentation day one. Once I found out who my customer is, what language they use, I want to check myself there. I don't want to just like, oh, I went away and I find all these great insights. Wow, good on me. I want to actually validate that against something. So experimentation to me is like, I've found these insights and I've shared these insights, but now I want to really make sure these insights are true for the entire. I want to see what the limitations of these insights are. Maybe it's true for everybody, I doubt it, but there are probably limitations to how true something is for such a big population or a segment of an audience.

 

Joe (28:11): So from day one, the way I would structure my performance marketing is in a way to gain insights, maybe not at a, you have these optimizations you need to be making and you need to be having best ad copy, and you run those multi-arm bandit tests and stuff right there, and that's given you which horse is getting over the finish line first. But if you have a hypothesis and an experiment you want to run, if you think about what copy cross the finish line, and then you're running a hundred campaigns, and it's like, is it always true? You end up getting this if you structure your campaigns and accounts and everything in a way that whenever you have run many, many different small tactical tests in six months, maybe six, maybe if it's a big enough account in a month, three months, six months, whatever time period, you want to step back and you've got all this macro data or macro data that is so rich because you've thought about what you want to answer and what's important to the business from the get go.

 

Joe (29:26): And you've tied that to, okay, well what's the business subjective? Does it help the business to understand the audience better? Does it help the business to understand what positioning or features sell the product better? How does that sharing that with the product team help them develop new products? But if you've thought about your performance marketing from day one through that experimental lens and thought that in time I'll have a rich set of data to mine for insight, the company's going to be richer, and what you're able to do for the company is going to be so much more valuable. The way I look at it is if you are doing that sort of performance marketing without experimentation, there's only two results. There's win and lose where your ad worked or didn't work. But if you do performance marketing with the lens of experimentation, there's a third option you lost, but you got an insight. You know why?

 

Tracy (30:35): Now we're getting into test to learn, test to win. I love it.

 

Joe (30:38): Yeah. I mean, in my eyes, if you were a three card Monty thing and it's like, oh, you have two chances to win versus one chance to win, I'm going to take the two chances to win every time in terms of what I pick. But so if you can turn your losses into wins, that is such a benefit to any business. So whenever I'm managing a team, I'm like, okay, what you've tried to do didn't work. Okay, why didn't it work? And if I can't get the answer of, oh, it didn't work because of this, then there was no value added to the company. But if they can turn like, oh, it didn't work for this and this reason, I'd be like, great, we won't do that again. Next time we'll make it even smarter test and we'll do something else and we've added 1% to the chance we're going to win next time.

 

Joe (31:35): But do that again and again and again, it compounds success. So maybe at the start, it seems slow. It's like, oh, we're losing more often than we're winning. It's like, but we're losing them. We're getting insights. So whenever the sticks are much bigger, the sticks always grow over time. So when the sticks are really big, especially with startups, they're like, oh, we got one seal a day. Hooray. But whenever it's like, oh, we're getting 50 seals a day, and then you've also paired that with, oh, and it's not a flip of a coin. It's a 60 40 chance of winning, then that's where you start to get that big growth. That's where that comes in. And maybe that's the other thing that I didn't mention about what people don't get about performance marketing. It's a journey. It's a process. You buy into it, you have a strategy, and then you steer the course unless something comes blindsides you out of the way, it obviously then looks like abandoned ship. But if you've got a path to walk getting buy-in early from any founder or CMO, and just staying that course and knowing that that process in time will give you that growth you're trying to unlock. But it is, you have to constantly be cast, always pushing at the boundaries and fringes of what works and getting those small failures so that you can get a bigger win down the line.

 

Tracy (33:04): Totally. I am such a strong believer in you will win eventually when you embrace the quote losing.

 

Tracy (33:17): Yes. Those are not optional. Those are imperative to have a successful, I'm thinking from the perspective of a conversion optimization program, which is, again, it's so similar to performance marketing in that regard. It's not an option to fail. You have to, it's part of the process. And I think, I don't know if you noticed this as well, but for me, the hardest thing is keeping trust with your client or your boss. When they see the flat results, they see the dips. Do you have tips for anyone who is kind of facing that challenge as well to make sure the trust is still there and that the patience and the trust and the process is still there?

 

Joe (34:03): That is a minefield, and there's no silver bullet for that really. I've found that the best thing because I'll throw my hands up and say, that didn't work, but I was like, well, at least we know it doesn't work and it didn't work for these reasons. And some people will take that as, oh, okay, march on. But some people will be like, oh, well, maybe in another pair hands this would be a success. I think that's a real insecurity that people have that they can't, people only want to see the good, but that's just not life.

 

Joe (34:49): But I found that if you are very transparent, transparency is a big way to win a client over. So whenever I go to a client, I'll be like, okay, this is the audit. This is what I found and this, and now because of this, this is what we'll do, and it makes sense. It's logical. And then over every step along the way, it's like, oh, we're at this step. This is what's happening. And if you bring them along for the journey and don't shut them out and include them in your thought process and say like, Hey, this didn't work because of this, but if we do X, Y, and Z now we'll get a win. And whenever they see X, Y, and Z happen and you get to a win, then they go, “oh, I think you might have something here”. But if you were just to go like, oh, we didn't win here, but I am going to do something and then we're going to win something.

And I think it's whenever you find a founder who is willing to embrace the process and respect the craft and the trade and just go along for the ride, that's a super valuable partnership to have because you can go to anybody and just be like, do this and only give me wins, and if you don't have me wins, I'll get the next guy who only get me wins. And then you'd just be in this treadmill or of just hiring and firing. But it's kind of a bad place to be and a very wasteful place to be because you also lose all those insights that maybe the person was getting. So yeah, I don't know. Is there a solution? No. I guess transparency and long-term planning, that's the only two things I can think of.

 

Tracy (36:49): I think those are really, really, really good callouts and they're things that you can just do for free anytime today. What else would you suggest that someone who's in performance marketing can do today to really level up their practice?

 

Joe (37:06): Oh, okay. I mean, I could say blogs and exams, all that kind of stuff, and there is a place for that, but really I think is obviously a big one. More you do the better you get anything, but also do with purpose don't probably, maybe there's a junior performance marketer out there listening to this and they are just going in optimizing their client account and then leaving, and they're just looking at the data sets that are in front of 'em. Like, well, maybe just if you're in an agency or you're in a company that's like, maybe think of the bigger impact of performance marketing and all the things that touch on performance marketing, and maybe just start trying to think of the part it plays in the bigger picture and start thinking, asking those questions, asking those bigger questions and asking yourself, how can performance marketing deliver more than just the transactional focus of performance marketing? How can it actually make the company better in some more substantive way? That's a very, there's not a book or anything I can tell you that's necessarily, it's a state of mind, I guess, or I dunno

 

Tracy (38:37): A good answer. I going to say you broke down some real poetry in this episode. I was not expecting that. I'm going to call it jo a tree It Very,

 

Joe (38:47): Okay.

 

Tracy (38:48): Yeah, no, this is a really great conversation. And I personally, you didn't work with me when I was doing performance marketing, but I was so bad at it, Joe, because I had that mindset of, oh, it's just dollars in dollars out. You get the spreadsheet. That's it. So you really kind of showed me, and hopefully our listeners too, that there's so much more to performance marketing or running ads that's very similar to experimentation and just the mindset around understanding your users. So thank you so much for breaking all that down. I saved the hardest questions for last. What do you have going on that you want to share with our listeners? What should they know about Joe?

 

Joe (39:36): Well, what should they know about Joe? I mean, the launch of ad scientists is really the newest thing with me. I think that is my kind of baby at the moment. And I was just thinking that, yeah, I really dunno how much more I'm willing to take on than launching a company at the moment. But I mean, I'm really enjoying working with a lot of fantastic companies and founders and people at just different stages of the journey. So yeah, I mean that's always exciting. And I think just the more that I'm able to meet and even just good conversations. I mean, that's probably the biggest thing that I've enjoyed since launching my own thing is meeting the other founders and just having chats about what are the roadblocks that are hitting them. So yeah, I mean that's kind of the exciting road in front of me right now.

 

Tracy (40:33): Nice. Where can our listeners reach you?

​

Joe (40:37): So you can reach me either on LinkedIn or on ad scientist.ca.

 

Tracy (40:44): Great. I love your website. You've got your dog Velma in there. Love her.

 

Joe (40:47): She's so cute. Oh, of course. There was nowhere. There was going to be a website without her fierce in there somewhere. Love that.

 

Tracy (40:56): Well, thank you so much, Joe. It's been really nice catching up with you and we'll catch you around.

 

Joe (41:01): Yeah, thanks for having me. Have a good rest of your day.