WEBVTT
00:00:00.320 --> 00:00:05.599
Ad fraud isn't so much a tech problem as it is fundamentally an incentive problem.
00:00:05.599 --> 00:00:10.720
You see, middlemen make more money when more volume flows through their systems.
00:00:10.720 --> 00:00:20.160
So they have little financial motivation to cut off fraudulent traffic, and fraudsters are sophisticated, continuously evolving to bypass detection.
00:00:20.160 --> 00:00:27.519
So today, we're learning about how to combat ad fraud, how it actually happens, and simple frameworks for avoiding it altogether.
00:00:27.519 --> 00:00:40.079
We'll learn about the red flag of low CPMs, why most reporting models overclaim credit for their contribution, and we'll hear about what happened when Uber turned off $100 million in ad spend.
00:00:40.079 --> 00:00:42.240
Welcome back to this day of streaming podcast.
00:00:42.240 --> 00:00:45.119
I'm your host, Tim Rowe, and today I'm joined by Dr.
00:00:45.119 --> 00:00:58.640
Augustine Foo, creator of Foo Analytics, an analytics pixel for monitoring programmatic media investment for the purpose really of identifying fraudulent traffic inventory and optimizing investment.
00:00:58.640 --> 00:01:08.560
So if you want to understand ad fraud once and for all, how it shows up in streaming TV, and most importantly, how to avoid it, then this is the conversation for you.
00:01:08.560 --> 00:01:10.079
Let's get into it.
00:01:10.079 --> 00:01:17.200
Dr.
00:01:17.200 --> 00:01:21.359
Fu, why is ad fraud such a big problem?
00:01:21.680 --> 00:01:24.239
It's gotten bigger and bigger over the years.
00:01:24.239 --> 00:01:28.879
And the way I like to describe it is that ad fraud is not a tech problem.
00:01:28.879 --> 00:01:30.959
It's actually an incentives problem.
00:01:30.959 --> 00:01:45.040
So it's uh it's kind of easy to understand if if you think about the middlemen, uh, you know, like the ad exchanges, even the agencies or other ad tech companies, uh, when there's more volume that flows through their pipes, they make more money.
00:01:45.040 --> 00:01:50.319
So they're not in a hurry to cut down the fraud because that's gonna cut down their revenue.
00:01:50.319 --> 00:01:54.640
So in that case, that's that's why I explained it as more of an incentives problem.
00:01:54.640 --> 00:02:00.079
And because it's an incentives problem, throwing more tech at it is not gonna solve it, right?
00:02:00.079 --> 00:02:02.560
So the bots and the fraudsters are very clever.
00:02:02.560 --> 00:02:06.640
They know how to get around the detection and avoid getting caught, right?
00:02:06.640 --> 00:02:08.719
Bad guys have a lot of practice doing that.
00:02:08.719 --> 00:02:13.280
So that's why fraud has gone up uh over the years and not down.
00:02:13.599 --> 00:02:14.879
That is that is a great point.
00:02:14.879 --> 00:02:22.960
They've gotten better at it, and there is a strong incentive related to uh to figuring out how to be good at being good at bad stuff.
00:02:22.960 --> 00:02:23.840
Being bad.
00:02:23.840 --> 00:02:26.240
Yeah, getting good at being bad.
00:02:26.240 --> 00:02:35.680
You've developed a series of technologies and frameworks and methodologies for thinking through and identifying and ultimately resolving these issues.
00:02:35.680 --> 00:02:43.039
We'll talk about that, but I'd love to talk about some of the really great examples that you shared through some of your published work.
00:02:43.039 --> 00:02:48.400
There's a piece that you shared, 100% click-throughs on pharmaceutical ads.
00:02:48.400 --> 00:02:49.759
Can you tell us about this story?
00:02:49.759 --> 00:02:52.719
How did how did someone get a hundred percent click-through rate?
00:02:52.719 --> 00:02:53.120
Yeah.
00:02:53.120 --> 00:02:54.080
And what does that mean?
00:02:54.319 --> 00:03:05.280
Um that's how I actually got into the fraud detection side of things because I was uh working at an agency in Omnicom, and we were serving a lot of pharmaceutical clients and med device clients.
00:03:05.280 --> 00:03:08.000
So they were primarily doing search campaigns.
00:03:08.000 --> 00:03:12.319
And then when you actually looked at some of the data, it's like, what the heck is going on here?
00:03:12.319 --> 00:03:22.400
If you just take the rolled up average, the campaigns were performing like you know, 9%, 15%, 25% click-through rates on average.
00:03:22.400 --> 00:03:28.960
But then when you actually pull the place report by domain, you'll say, this domain has 100% click-through rate.
00:03:28.960 --> 00:03:33.680
This other domain has 100% click-through rate, sometimes greater than 100%, 100% click-through rate.
00:03:33.680 --> 00:03:37.520
So, you know, if you're just looking at the rolled-up averages, you're gonna miss it, right?
00:03:37.520 --> 00:03:41.120
It's gonna seem like, oh, it's great, you know, this campaign is performing so well.
00:03:41.120 --> 00:03:46.479
But then when you look at the line items, the details, you'll see stuff that just doesn't make any sense.
00:03:46.479 --> 00:03:51.840
So back then, this was you know almost 15 years ago, you know, no one could really explain what it was.
00:03:51.840 --> 00:03:54.960
They were all just cheering, oh yeah, the campaign's working really well.
00:03:54.960 --> 00:04:01.120
It's like that doesn't make any sense because humans don't click on ads that much, including search ads, right?
00:04:01.120 --> 00:04:07.439
1% would be a good rule of thumb, maybe 2%, but definitely not 100 or 200%.
00:04:07.439 --> 00:04:16.160
So, long story short, we now know it's the bots that are clicking on the ads, and they do that because it's a cost per click model.
00:04:16.160 --> 00:04:19.839
So they have to click it in order to get the revenue.
00:04:19.839 --> 00:04:29.279
And so for a lot of the advertisers, you know, when when they run a search campaign, a lot of the ads do run on Google.com, right, where humans Google things.
00:04:29.279 --> 00:04:38.319
But there's also something called the search partner network, where there's all these other sites that run code on the page to run search ads.
00:04:38.319 --> 00:04:46.160
So when there's a search ad that shows up and someone clicks on it, the site gets some of the revenue and Google gets some of the revenue, right?
00:04:46.160 --> 00:04:47.519
It's kind of shared.
00:04:47.519 --> 00:04:57.279
So then you can see how the sites have the motive, right, the incentive to commit click fraud using bots because they can just click on ads all day long.
00:04:57.279 --> 00:04:59.519
So then they can make more money for their site.
00:04:59.519 --> 00:05:01.680
So that's kind of what we're fighting, right?
00:05:01.680 --> 00:05:05.120
And when we looked at the data and you know, it didn't quite make sense.
00:05:05.120 --> 00:05:10.800
That's when I started digging into it and I built some technology tools to help me do the audits, right?
00:05:10.800 --> 00:05:13.279
To kind of say what the heck is going on here.
00:05:13.279 --> 00:05:17.120
Long story short, is uh it's kind of evolved into a platform, right?
00:05:17.120 --> 00:05:22.399
I didn't set out to build flu analytics, uh, it was more of some tools to help me audit campaigns.
00:05:22.399 --> 00:05:25.759
And I was originally delivering it through PowerPoints, right?
00:05:25.759 --> 00:05:29.839
Kind of like McKinsey consulting projects, but that doesn't scale.
00:05:29.839 --> 00:05:35.040
So by 2020, I just decided I'm gonna open up the platform and let others use it.
00:05:35.040 --> 00:05:43.040
So just like people learned how to use Google Analytics for their websites, they can now learn how to use Foo Analytics for their digital ads.
00:05:43.040 --> 00:05:48.319
So that's uh I just gave it a name, Foo Analytics, and then added user accounts.
00:05:48.319 --> 00:06:04.879
And now majority of our customers are the big advertisers because their marketing folks and their analytics folks are now equipped with an analytics platform that they can log into, look at the dashboard, and then figure out where the bad stuff is, right?
00:06:04.879 --> 00:06:11.600
Uh which are the bad sites, which are bad uh mobile apps, and then just add those to a block list and clean up their campaigns.
00:06:11.920 --> 00:06:12.399
Huge.
00:06:12.399 --> 00:06:19.680
Uh how does that change or how does that look different for television, for CTV, streaming, for video?
00:06:19.680 --> 00:06:22.879
How they're not not clickable necessarily.
00:06:22.879 --> 00:06:25.600
How does fraud show up inside of video?
00:06:26.079 --> 00:06:29.199
Well, it's you know, the click is like the outcome of it, right?
00:06:29.199 --> 00:06:32.800
So you show the ad and then some portion of the users might click on it.
00:06:32.800 --> 00:06:37.439
But what's more important is that a portion of the ads are not even shown to humans, right?
00:06:37.439 --> 00:06:40.319
And those a portion of those ads are going to fake sites.
00:06:40.319 --> 00:06:42.399
So that's what we're trying to stop, right?
00:06:42.399 --> 00:06:44.160
Even before the click happens.
00:06:44.160 --> 00:06:48.560
So if your ad is running on a fake site, humans are definitely not seeing it, right?
00:06:48.560 --> 00:06:57.920
And so that's that's the problem where we're measuring the ads where they went and trying to block those before the ads go there and before the money goes there.
00:06:57.920 --> 00:07:01.439
Because once the money's there, it's really hard to get back, right?
00:07:01.439 --> 00:07:04.000
So a lot of people say, oh yeah, we can get refunds.
00:07:04.000 --> 00:07:08.399
Once the money is in the bad guys' pockets, you're never getting it back from them.
00:07:08.399 --> 00:07:16.079
So ultimately, it's going to be some middleman like maybe the exchange that you bought from that's going to basically eat the cost and give it back to you.
00:07:16.079 --> 00:07:23.759
So it's always better to not let the money go to the bad guys in the first place rather than try to get a refund afterwards.
00:07:24.079 --> 00:07:29.920
So step one, prevent the money from going there in the first place by having analytics.
00:07:30.160 --> 00:07:31.839
Yeah, having good analytics.
00:07:31.839 --> 00:07:34.319
Yeah, because first you have to see that you have a problem.
00:07:34.319 --> 00:07:44.160
And the reason I'm using the word analytics as opposed to fraud detection is that I think a lot of people are familiar with the legacy fraud verification companies, which I won't name.
00:07:44.160 --> 00:07:50.319
Basically, they just give you a spreadsheet and a dashboard that says 1% IVT or invalid traffic.
00:07:50.319 --> 00:07:55.360
When you see that and nothing else, there's really nothing you can do with that number.
00:07:55.360 --> 00:07:55.839
Right.
00:07:55.839 --> 00:08:08.720
So the reason I'm calling my platform analytics is that it has the supporting details so that you can go in there and understand why something was marked a bot or why something was marked, you know, there's other forms of fraud other than bots.
00:08:08.720 --> 00:08:14.560
So if you say, okay, I understand that, and these are bad guys, then let me add them to a block list.
00:08:14.560 --> 00:08:14.959
Right.
00:08:14.959 --> 00:08:21.519
So these are kind of a process that you go through maybe once a month, uh, maybe once a week if if you want to.
00:08:21.519 --> 00:08:31.040
You don't have to do it continuously, but it gives you a way to monitor where your ads are going and then continuously kind of improve and optimize your campaigns as you go along.
00:08:31.279 --> 00:08:31.680
Cool.
00:08:31.680 --> 00:08:38.159
So we're building a block list as we go along of our of our known bad actors.
00:08:38.159 --> 00:08:45.200
How is inside of CTV, how how are, I guess, how is the fraud taking place?
00:08:45.200 --> 00:09:00.159
Thinking about, let's think about like our local seller who's trying to explain the advantages of CTV to uh a small business owner, but they're like, hey, I saw an article that said fraud's gonna take my money, so I'm gonna put it on a billboard instead.
00:09:00.159 --> 00:09:02.960
How would you explain that to small business owners?
00:09:03.200 --> 00:09:06.080
So in CTV, um, there's actually a conundrum here.
00:09:06.080 --> 00:09:10.399
So let me let me kind of put this out there and let's see how you how you react to it.
00:09:10.399 --> 00:09:17.840
So in CTV, on the one hand, I can say there's no fraud in CTV because it's impossible.
00:09:17.840 --> 00:09:22.159
And then on the other hand, I can say there's 100% fraud in CTV.
00:09:22.159 --> 00:09:28.720
Now, how can both of those exist and be true at the same time?
00:09:29.679 --> 00:09:31.440
I don't know, you got me stumped there.
00:09:31.440 --> 00:09:32.480
How could that be true?
00:09:33.279 --> 00:09:40.000
So basically, if you're buying from a real seller like Home and Garden TV, Food Network, ESPN, right?
00:09:40.000 --> 00:09:43.200
Obviously, these are ones people have heard of and actually subscribe to.
00:09:43.200 --> 00:09:44.799
Disney Plus, whatever.
00:09:44.799 --> 00:09:48.080
There's no fraud in CTV because it's impossible.
00:09:48.080 --> 00:09:52.159
Because Disney Plus is not trying to rip you off.
00:09:52.159 --> 00:09:59.759
If you buy ads direct from them, the bad guys can't get the fake impressions in there and can't get money out.
00:09:59.759 --> 00:10:03.759
Therefore, they're not doing fraud in CTV.
00:10:03.759 --> 00:10:16.080
However, if you try to buy CTV on programmatic exchanges, and especially if you're trying to buy super large quantities at super low prices, it is not CTV.
00:10:16.080 --> 00:10:17.919
It is 100% fraud.
00:10:17.919 --> 00:10:31.840
That's because any bad guy can just upload a bid request and say it's Disney Plus, say it's HGTV, say it's a food network, say whatever they want because they want to pretend to be the legitimate apps so that they can get bids.
00:10:31.840 --> 00:10:32.399
Right?
00:10:32.399 --> 00:10:36.000
If they put a no-name app in there, nobody's gonna bid on it.
00:10:36.000 --> 00:10:48.399
So it's always better for the fraudster to pretend to be a well-recognized streaming app like ESPN or Disney Plus in the bid requests, but they still have their own seller ID in there because they want to get paid.
00:10:48.399 --> 00:10:55.519
So that money is actually diverted away from ESPN, Disney Plus, uh, you know, Amazon Prime, whatever.
00:10:55.519 --> 00:10:59.440
Those sellers, the the legitimate sellers don't even see that.
00:10:59.440 --> 00:11:06.240
And there's a bunch of long tail Roku apps, but that's actually tiny compared to the mainstream fraud.
00:11:06.240 --> 00:11:11.200
Okay, so in that case, there's no fraud in CTV because it's impossible.
00:11:11.200 --> 00:11:16.320
Applies to the case where you're buying from legitimate sellers, ones that you've heard of before.
00:11:16.320 --> 00:11:22.480
And let me note that legitimate sellers, including YouTube, has unsold inventory.
00:11:22.480 --> 00:11:23.679
How do I know?
00:11:23.679 --> 00:11:29.919
When I'm watching Sunday football, you'll see a little screen that says enjoy the Zen, we'll be right back.
00:11:29.919 --> 00:11:33.440
That's an unsold CTV ad slot.
00:11:33.440 --> 00:11:36.159
Okay, so even the big guys have unsold.
00:11:36.159 --> 00:11:43.360
That's because the advertisers are chasing low-cost CTV inventory that's actually not real.
00:11:43.360 --> 00:11:52.879
Okay, they're buying billions of impressions, and it's it didn't run on ESPN, it didn't run on Disney Plus, it ran somewhere else or didn't run at all.
00:11:52.879 --> 00:12:00.960
Okay, and then the the flip side is there is fraud in CTV and it's 100%, and that's because it's everything outside.
00:12:01.279 --> 00:12:13.759
It seems then like there's a larger conversation about what a CPM actually is, because if you're paying a really low CPM, but you're not actually getting the thing that you're paying for, what the heck was it worth?
00:12:13.759 --> 00:12:15.440
Can we talk about CPMs?
00:12:15.759 --> 00:12:25.679
Yeah, that's actually a very important topic because uh what I hear all day long is when you know agencies and even the advertisers say, oh, cost efficiency, they're trying to save cost.
00:12:25.679 --> 00:12:33.519
Well, let me use a very simple mathematical calculation here, just based on what I've seen evolve over the last 15 years.
00:12:33.519 --> 00:12:43.039
In the early days of digital, you would be paying $35 CPMs to a publisher like New York Times, you know, any any major publisher.
00:12:43.039 --> 00:12:46.080
So that's $35 per thousand impressions.
00:12:46.080 --> 00:12:49.679
Now you're paying $3 CPMs.
00:12:49.679 --> 00:12:53.120
Okay, so people think, oh yeah, we had cost savings.
00:12:53.120 --> 00:12:56.240
But actually that's not true because CPM is a price.
00:12:56.240 --> 00:12:58.720
You still have to multiply by the quantity.
00:12:58.720 --> 00:13:06.879
So now you're buying $3 CPM prices, you're buying 10 times the quantity, you're still spending $30.
00:13:06.879 --> 00:13:08.559
You see what I'm saying?
00:13:08.559 --> 00:13:09.759
That's the cost.
00:13:09.759 --> 00:13:12.480
So CPM is a price, it's not a cost.
00:13:12.480 --> 00:13:16.000
So there's this misuse of the term cost efficiency.
00:13:16.000 --> 00:13:18.159
Oh, yeah, lower CPMs, it's cost efficient.
00:13:18.159 --> 00:13:18.960
It's not.
00:13:18.960 --> 00:13:34.000
So what ends up happening is now, you know, when the advertisers are chasing the really low price like CPMs for CTV, they end up buying the crap, like the non-real stuff, and they're buying lots and lots of quantity of it.
00:13:34.000 --> 00:13:42.399
So the simple advice is don't be afraid of paying higher CPM prices because you're going to be buying less quantity.
00:13:42.399 --> 00:13:49.600
So you're still saving costs because fewer of your ads are going to completely fake sites and uh mobile apps.
00:13:49.600 --> 00:13:50.080
Right.
00:13:50.080 --> 00:13:55.919
So that's kind of how to think about a CPM as a price and how that impacts your cost, right?
00:13:55.919 --> 00:13:59.840
So you can really save costs by buying less of the fraud that's out there.
00:14:00.559 --> 00:14:03.759
And that segues perfectly into reporting.
00:14:03.759 --> 00:14:12.080
If I'm buying a bunch of fraudulent inventory, how is it that my reporting still reflects that it's working?
00:14:12.080 --> 00:14:15.120
It feels like, okay, cool.
00:14:15.120 --> 00:14:28.320
If I understand how fraud works and I understand that I need to take measures to prevent fraud from happening, and here are corrective actions I can take after it happens to prevent it from happening again, and I understand the pricing.
00:14:28.320 --> 00:14:32.159
How is it actually showing up in my reporting like it's working?
00:14:32.399 --> 00:14:33.039
Yep.
00:14:33.039 --> 00:14:40.320
So it's easily explained if you can understand the difference between correlation and causation.
00:14:40.639 --> 00:14:40.879
Okay.
00:14:40.879 --> 00:14:41.840
Let's let's explain.
00:14:42.320 --> 00:14:42.799
Correlation.
00:14:42.799 --> 00:14:43.200
Yeah.
00:14:43.200 --> 00:14:50.559
So correlation just means sales are happening over here while you're doing digital marketing over there.
00:14:50.559 --> 00:14:51.120
Yes.
00:14:51.120 --> 00:14:52.879
They're unrelated to each other.
00:14:52.879 --> 00:15:01.759
So here's the thing: like for a lot of the CPG companies, if you're selling uh soda or if you're selling soup or paper towels or whatever, that happens in the grocery store.
00:15:01.759 --> 00:15:04.080
You're doing digital marketing over here.
00:15:04.080 --> 00:15:12.559
It's not directly causing sales of paper towels, toilet paper, soup and soda in grocery stores.
00:15:12.559 --> 00:15:17.279
Okay, those are happening, whether or not you're doing digital marketing.
00:15:17.279 --> 00:15:25.039
And there's been some experiments over the years, not many, but remember when PG turned off 200 million of their digital spend?
00:15:25.039 --> 00:15:27.519
No change in their sales.
00:15:27.519 --> 00:15:36.080
Okay, so those are data points that I've seen that most people don't want to hear about because they all want to think that the digital marketing is working.
00:15:36.080 --> 00:15:40.879
So now let me get to a little bit more detail to why the reports look so good.
00:15:40.879 --> 00:15:43.440
Okay, so that's a matter of attribution.
00:15:43.440 --> 00:15:47.679
Some of it's fraudulent, some of it's just incorrect attribution.
00:15:47.679 --> 00:16:02.399
So in years past, there, you know, when some of these ad tech companies were trying to push display ads, they were saying, oh, you know, it's so unfair that search ads got all the credit for the sales.
00:16:02.399 --> 00:16:10.720
The reason search ads got all the credit for sales is because it's usually the last click before the customer purchases something.
00:16:10.720 --> 00:16:18.399
So if you see something on TV, you're gonna say, oh, let me go Google that, and then you see a search ad and then you click on it and then you buy something, right?
00:16:18.399 --> 00:16:21.279
Because at that time you're thinking about it, you're ready to buy something.
00:16:21.279 --> 00:16:26.879
So then that sale gets attributed to the last click, which was driven by search.
00:16:26.879 --> 00:16:30.799
So then all the people who are trying to sell display ads say, oh, those that's so unfair.
00:16:30.799 --> 00:16:35.919
Display ads must have some impact and must have helped to cause that sale.
00:16:35.919 --> 00:16:39.600
That is true, but it's not a one-to-one correlation.
00:16:39.600 --> 00:16:45.039
It's not like one display ad, you know, a human sees one display ad and then they go buy something, right?
00:16:45.039 --> 00:16:52.080
You have to see a lot of advertising, and then when you're in, you know, when you're in the right mood and the right time, then you're gonna go buy something.
00:16:52.080 --> 00:17:01.120
But because of the display ad sellers complaining, Google basically developed something called view through conversions.
00:17:01.120 --> 00:17:07.680
So that meant even if you didn't click on the display ad, it has some kind of benefit, right?
00:17:07.680 --> 00:17:20.559
So they're gonna say when a person is exposed to that display ad, if there's a sale or purchase that happens within the next 30, 60, or 90 days, we're gonna attribute that sale to the display ad.
00:17:20.559 --> 00:17:24.480
That's a very, very rudimentary way of doing attribution.
00:17:24.480 --> 00:17:29.119
And it does work in certain cases, but it's also been now abused.
00:17:29.119 --> 00:17:29.519
All right.
00:17:29.519 --> 00:17:37.039
So, for example, you can say, oh, you know, that individual display ad didn't drive that sale that happened 30 days later.
00:17:37.039 --> 00:17:38.720
It just happened 30 days later.
00:17:38.720 --> 00:17:45.920
And you can see a scenario where the person was going to buy the thing anyway, and they just happened to be exposed to the ad.
00:17:45.920 --> 00:17:47.920
It wasn't caused by the ad.
00:17:47.920 --> 00:17:49.279
You see the difference?
00:17:49.279 --> 00:18:02.079
So the problem is these attribution or conversion models over-attribute or overclaim credit for the sales that would have happened anyway, or that would have happened in the absence of the advertising.
00:18:02.079 --> 00:18:12.880
So, what a lot of advertisers are not yet doing is focusing on incrementality, which means the sales that would not have happened in the absence of the advertising, right?
00:18:12.880 --> 00:18:16.000
We actually want to say we spent these dollars in digital.
00:18:16.000 --> 00:18:20.880
Did it actually drive more sales than those that would have happened anyway?
00:18:20.880 --> 00:18:21.279
Right?
00:18:21.279 --> 00:18:23.680
That's really the key question they should be asking.
00:18:23.680 --> 00:18:35.680
But kind of to close out your question, it's they're seeing sales happening, and they have attribution models that say, oh, those sales are caused by the ad impressions, even though they weren't.
00:18:35.680 --> 00:18:46.079
So that's why a lot of advertisers believe that it's working, even though there's a whole bunch of fraud, like it could be 90% ads shown to bots, the sales still happened.
00:18:46.079 --> 00:18:52.400
But then the model said, Oh, yeah, it's it's you know, those sales are attributed to these digital campaigns that you have running.
00:18:52.400 --> 00:18:59.359
That's why you believe, and there's a whole bunch of people who wanted to believe that digital works really, really well.
00:18:59.359 --> 00:19:02.559
Okay, so that's how it covered up the fraud, right?
00:19:02.559 --> 00:19:09.119
Even if the campaign had 90% bots in it, impressions were shown to bots, they'd still had sales going on, right?
00:19:09.119 --> 00:19:11.599
They still had purchases, conversions, all that kind of stuff.
00:19:11.599 --> 00:19:13.279
So they thought it was working.
00:19:13.279 --> 00:19:31.279
But careful marketers and probably more advanced marketers are now kind of trying to tease that apart and saying, okay, well, we can run turn-off experiments, like just turn off the digital campaign in one state and see if the sales in that state continue at the same rate, right?
00:19:31.279 --> 00:19:35.519
Those are simple experiments they can run to say, was there a cause and effect?
00:19:35.519 --> 00:19:40.160
And then now to kind of close out the correlation versus causation thing.
00:19:40.160 --> 00:19:45.519
Correlation is just most of the sales and the digital marketing, it's just correlation, right?
00:19:45.519 --> 00:19:46.799
They happen at the same time.
00:19:46.799 --> 00:19:48.240
It's not causation.
00:19:48.240 --> 00:19:54.240
But I think more and more marketers are kind of turning on to that and getting smart and say, okay, that's not good enough.
00:19:54.240 --> 00:19:56.480
We really have to figure out the causation.
00:19:56.480 --> 00:19:59.759
So there's different methodologies, there's you know, media mix models.
00:19:59.759 --> 00:20:01.039
And that kind of stuff.
00:20:01.039 --> 00:20:07.920
But you have to be very careful where errors are introduced into those models and you're still overclaiming credit for sales.
00:20:07.920 --> 00:20:11.359
So you've got to be careful to only focus on the incrementality.
00:20:11.680 --> 00:20:24.000
Someone that's listening today, they're maybe either buying CTV, they're leading video investment for a brand, or they're leading a local sales team and trying to navigate conversations like this to drive adoption.
00:20:24.000 --> 00:20:28.559
What's the one key takeaway that you want a listener to leave with?
00:20:29.200 --> 00:20:34.480
I think the best way to think about this is approach it as if you were a small business owner.
00:20:34.480 --> 00:20:37.359
Okay, so small business owners have finite budgets.
00:20:37.359 --> 00:20:51.599
If they spend $1,000 and they don't get any more sales, like any more people walking into their grocery store or anyone, any more people sitting in the chair at their barbershop, they can't spend the next $1,000 on digital ads.
00:20:51.599 --> 00:20:55.680
So, you know, the big marketers, they have so much budget.
00:20:55.680 --> 00:21:02.559
And they have these media mix models and all that kind of stuff that kind of attribute and tell them things are working, and you have great ROAS on it, right?
00:21:02.559 --> 00:21:04.240
Return on ad spend on it.
00:21:04.240 --> 00:21:09.200
Those are the ones where they're spending way, way more money than they should be.
00:21:09.200 --> 00:21:15.039
So it's almost like think as if you were a small business owner and you have to actually look at real outcomes.