How To Spot Ad Fraud on Streaming TV (and beyond) Dr. Augustine Fou, Creator of FouAnalytics

Ad Fraud As An Incentive Problem

Tim Rowe 0:00

Ad fraud isn't so much a tech problem as it is fundamentally an incentive problem. You see, middlemen make more money when more volume flows through their systems. So they have little financial motivation to cut off fraudulent traffic, and fraudsters are sophisticated, continuously evolving to bypass detection. So today, we're learning about how to combat ad fraud, how it actually happens, and simple frameworks for avoiding it altogether. We'll learn about the red flag of low CPMs, why most reporting models overclaim credit for their contribution, and we'll hear about what happened when Uber turned off $100 million in ad spend. Welcome back to this day of streaming podcast. I'm your host, Tim Rowe, and today I'm joined by Dr. Augustine Foo, creator of Foo Analytics, an analytics pixel for monitoring programmatic media investment for the purpose really of identifying fraudulent traffic inventory and optimizing investment. So if you want to understand ad fraud once and for all, how it shows up in streaming TV, and most importantly, how to avoid it, then this is the conversation for you. Let's get into it. Dr. Fu, why is ad fraud such a big problem?

Dr. Fou 1:21

It's gotten bigger and bigger over the years. And the way I like to describe it is that ad fraud is not a tech problem. It's actually an incentives problem. So it's uh it's kind of easy to understand if if you think about the middlemen, uh, you know, like the ad exchanges, even the agencies or other ad tech companies, uh, when there's more volume that flows through their pipes, they make more money. So they're not in a hurry to cut down the fraud because that's gonna cut down their revenue. So in that case, that's that's why I explained it as more of an incentives problem. And because it's an incentives problem, throwing more tech at it is not gonna solve it, right? So the bots and the fraudsters are very clever. They know how to get around the detection and avoid getting caught, right? Bad guys have a lot of practice doing that. So that's why fraud has gone up uh over the years and not down.

Tim Rowe 2:13

That is that is a great point. They've gotten better at it, and there is a strong incentive related to uh to figuring out how to be good at being good at bad stuff. Being bad. Yeah, getting good at being bad. You've developed a series of technologies and frameworks and methodologies for thinking through and identifying and ultimately resolving these issues. We'll talk about that, but I'd love to talk about some of the really great examples that you shared through some

100% CTRs And Click Fraud Mechanics

Tim Rowe 2:42

of your published work. There's a piece that you shared, 100% click-throughs on pharmaceutical ads. Can you tell us about this story? How did how did someone get a hundred percent click-through rate? Yeah. And what does that mean?

Dr. Fou 2:54

Um that's how I actually got into the fraud detection side of things because I was uh working at an agency in Omnicom, and we were serving a lot of pharmaceutical clients and med device clients. So they were primarily doing search campaigns. And then when you actually looked at some of the data, it's like, what the heck is going on here? If you just take the rolled up average, the campaigns were performing like you know, 9%, 15%, 25% click-through rates on average. But then when you actually pull the place report by domain, you'll say, this domain has 100% click-through rate. This other domain has 100% click-through rate, sometimes greater than 100%, 100% click-through rate. So, you know, if you're just looking at the rolled-up averages, you're gonna miss it, right? It's gonna seem like, oh, it's great, you know, this campaign is performing so well. But then when you look at the line items, the details, you'll see stuff that just doesn't make any sense. So back then, this was you know almost 15 years ago, you know, no one could really explain what it was. They were all just cheering, oh yeah, the campaign's working really well. It's like that doesn't make any sense because humans don't click on ads that much, including search ads, right? 1% would be a good rule of thumb, maybe 2%, but definitely not 100 or 200%. So, long story short, we now know it's the bots that are clicking on the ads, and they do that because it's a cost per click model. So they have to click it in order to get the revenue. And so for a lot of the advertisers, you know, when when they run a search campaign, a lot of the ads do run on Google.com, right, where humans Google things. But there's also something called the search partner network, where there's all these other sites that run code on the page to run search ads. So when there's a search ad that shows up and someone clicks on it, the site gets some of the revenue and Google gets some of the revenue, right? It's kind of shared. So then you can see how the sites have the motive, right, the incentive to commit click fraud using bots because they can just click on ads all day long. So then they can make more money for their site. So that's kind of what we're fighting, right? And when we looked at the data and you know, it didn't quite make sense. That's when I started digging into it and I built some technology tools to help me do the audits, right? To kind of say what the heck is going on here. Long story short, is uh it's kind of evolved into a platform, right? I didn't set out to build flu analytics, uh, it was more of

From Audits To Foo Analytics

Dr. Fou 5:20

some tools to help me audit campaigns. And I was originally delivering it through PowerPoints, right? Kind of like McKinsey consulting projects, but that doesn't scale. So by 2020, I just decided I'm gonna open up the platform and let others use it. So just like people learned how to use Google Analytics for their websites, they can now learn how to use Foo Analytics for their digital ads. So that's uh I just gave it a name, Foo Analytics, and then added user accounts. And now majority of our customers are the big advertisers because their marketing folks and their analytics folks are now equipped with an analytics platform that they can log into, look at the dashboard, and then figure out where the bad stuff is, right? Uh which are the bad sites, which are bad uh mobile apps, and then just add those to a block list and clean up their campaigns.

Tim Rowe 6:11

Huge. Uh how does that change or how does that look different for television, for CTV, streaming, for video? How they're not not clickable necessarily.

How CTV Fraud Actually Works

Tim Rowe 6:23

How does fraud show up inside of video?

Dr. Fou 6:26

Well, it's you know, the click is like the outcome of it, right? So you show the ad and then some portion of the users might click on it. But what's more important is that a portion of the ads are not even shown to humans, right? And those a portion of those ads are going to fake sites. So that's what we're trying to stop, right? Even before the click happens. So if your ad is running on a fake site, humans are definitely not seeing it, right? And so that's that's the problem where we're measuring the ads where they went and trying to block those before the ads go there and before the money goes there. Because once the money's there, it's really hard to get back, right? So a lot of people say, oh yeah, we can get refunds. Once the money is in the bad guys' pockets, you're never getting it back from them. So ultimately, it's going to be some middleman like maybe the exchange that you bought from that's going to basically eat the cost and give it back to you. So it's always better to not let the money go to the bad guys in the first place rather than try to get a refund afterwards.

Tim Rowe 7:24

So step one, prevent the money from going there in the first place by having analytics.

Dr. Fou 7:30

Yeah, having good analytics. Yeah, because first you have to see that you have a problem. And the reason I'm using the word analytics as opposed to fraud detection is that I think a lot of people are familiar with the legacy fraud verification companies, which I won't name. Basically, they just give you a spreadsheet and a dashboard that says 1% IVT or invalid traffic. When you see that and nothing else, there's really nothing you can do with that number. Right. So the reason I'm calling my platform analytics is that it has the supporting details so that you can go in there and understand why something was marked a bot or why something was marked, you know, there's other forms of fraud other than bots. So if you say, okay, I understand that, and these are bad guys, then let me add them to a block list. Right. So these are kind of a process that you go through maybe once a month, uh, maybe once a week if if you want to. You don't have to do it continuously, but it gives you a way to monitor where your ads are going and then continuously kind of improve and optimize your campaigns as you go along.

Tim Rowe 8:31

Cool. So we're building a block list as we go along of our of our known bad actors. How is inside of CTV, how how are, I guess, how is the fraud taking place? Thinking about, let's think about like our local seller who's trying to explain the advantages of CTV to uh a small business owner, but they're like, hey, I saw an article that said fraud's gonna take my money, so I'm gonna put it on a billboard instead. How would you explain that to small business owners?

Dr. Fou 9:03

So in CTV, um, there's actually a conundrum here. So let me let me kind of put this out there and let's see how you how you react to it. So in CTV, on the one hand, I can say there's no fraud in CTV because it's impossible. And then on the other hand, I can say there's 100%

The CPM Trap And False “Cost Efficiency”

Dr. Fou 9:21

fraud in CTV. Now, how can both of those exist and be true at the same time?

Tim Rowe 9:29

I don't know, you got me stumped there. How could that be true?

Dr. Fou 9:33

So basically, if you're buying from a real seller like Home and Garden TV, Food Network, ESPN, right? Obviously, these are ones people have heard of and actually subscribe to. Disney Plus, whatever. There's no fraud in CTV because it's impossible. Because Disney Plus is not trying to rip you off. If you buy ads direct from them, the bad guys can't get the fake impressions in there and can't get money out. Therefore, they're not doing fraud in CTV. However, if you try to buy CTV on programmatic exchanges, and especially if you're trying to buy super large quantities at super low prices, it is not CTV. It is 100% fraud. That's because any bad guy can just upload a bid request and say it's Disney Plus, say it's HGTV, say it's a food network, say whatever they want because they want to pretend to be the legitimate apps so that they can get bids. Right? If they put a no-name app in there, nobody's gonna bid on it. So it's always better for the fraudster to pretend to be a well-recognized streaming app like ESPN or Disney Plus in the bid requests, but they still have their own seller ID in there because they want to get paid. So that money is actually diverted away from ESPN, Disney Plus, uh, you know, Amazon Prime, whatever. Those sellers, the the legitimate sellers don't even see that. And there's a bunch of long tail Roku apps, but that's actually tiny compared to the mainstream fraud. Okay, so in that case, there's no fraud in CTV because it's impossible. Applies to the case where you're buying from legitimate sellers, ones that you've heard of before. And let me note that legitimate sellers, including YouTube, has unsold inventory. How do I know? When I'm watching Sunday football, you'll see a little screen that says enjoy the Zen, we'll be right back. That's an unsold CTV ad slot. Okay, so even the big guys have unsold. That's because the advertisers are chasing low-cost CTV inventory that's

Correlation, Causation, And Bad Attribution

Dr. Fou 11:42

actually not real. Okay, they're buying billions of impressions, and it's it didn't run on ESPN, it didn't run on Disney Plus, it ran somewhere else or didn't run at all. Okay, and then the the flip side is there is fraud in CTV and it's 100%, and that's because it's everything outside.

Tim Rowe 12:01

It seems then like there's a larger conversation about what a CPM actually is, because if you're paying a really low CPM, but you're not actually getting the thing that you're paying for, what the heck was it worth? Can we talk about CPMs?

Dr. Fou 12:15

Yeah, that's actually a very important topic because uh what I hear all day long is when you know agencies and even the advertisers say, oh, cost efficiency, they're trying to save cost. Well, let me use a very simple mathematical calculation here, just based on what I've seen evolve over the last 15 years. In the early days of digital, you would be paying $35 CPMs to a publisher like New York Times, you know, any any major publisher. So that's $35 per thousand impressions. Now you're paying $3 CPMs. Okay, so people think, oh yeah, we had cost savings. But actually that's not true because CPM is a price. You still have to multiply by the quantity. So now you're buying $3 CPM prices, you're buying 10 times the quantity, you're still spending $30. You see what I'm saying? That's the cost. So CPM is a price, it's not a cost. So there's this misuse of the term cost efficiency. Oh, yeah, lower CPMs, it's cost efficient. It's not. So what ends up happening is now, you know, when the advertisers are chasing the really low price like CPMs for CTV, they end up buying the crap, like the non-real stuff, and they're buying lots and lots of quantity of it. So the simple advice is don't be afraid of paying higher CPM prices because you're going to be buying less quantity. So you're still saving costs because fewer of your ads are going to completely fake sites and uh mobile apps. Right. So that's kind of how to think about a CPM as a price and how that impacts your cost, right? So you can really save costs by buying less of the fraud that's out there.

Tim Rowe 14:00

And that segues perfectly into reporting. If I'm buying a bunch of fraudulent inventory, how is it that my reporting still reflects that it's working?

Turnoff Tests And The Uber Case

Tim Rowe 14:12

It feels like, okay, cool. If I understand how fraud works and I understand that I need to take measures to prevent fraud from happening, and here are corrective actions I can take after it happens to prevent it from happening again, and I understand the pricing. How is it actually showing up in my reporting like it's working?

Dr. Fou 14:32

Yep. So it's easily explained if you can understand the difference between correlation and causation.

Tim Rowe 14:40

Okay. Let's let's explain.

Dr. Fou 14:42

Correlation. Yeah. So correlation just means sales are happening over here while you're doing digital marketing over there. Yes. They're unrelated to each other. So here's the thing: like for a lot of the CPG companies, if you're selling uh soda or if you're selling soup or paper towels or whatever, that happens in the grocery store. You're doing digital marketing over here. It's not directly causing sales of paper towels, toilet paper, soup and soda in grocery stores. Okay, those are happening, whether or not you're doing digital marketing. And there's been some experiments over the years, not many, but remember when PG turned off 200 million of their digital spend? No change in their sales. Okay, so those are data points that I've seen that most people don't want to hear about because they all want to think that the digital marketing is working. So now let me get to a little bit more detail to why the reports look so good. Okay, so that's a matter of attribution. Some of it's fraudulent, some of it's just incorrect attribution. So in years past, there, you know, when some of these ad tech companies were trying to push display ads, they were saying, oh, you know, it's so unfair that search ads got all the credit for the sales. The reason search ads got all the credit for sales is because it's usually the last click before the customer purchases something. So if you see something on TV, you're gonna say, oh, let me go Google that, and then you see a search ad and then you click on it and then you buy something, right? Because at that time you're thinking about it, you're ready to buy something. So then that sale gets attributed to the last click, which was driven by search. So then all the people who are trying to sell display ads say, oh, those that's so unfair. Display ads must have some impact and must have helped to cause that sale. That is true, but it's

Practical Takeaways And Foo Audits

Dr. Fou 16:38

not a one-to-one correlation. It's not like one display ad, you know, a human sees one display ad and then they go buy something, right? You have to see a lot of advertising, and then when you're in, you know, when you're in the right mood and the right time, then you're gonna go buy something. But because of the display ad sellers complaining, Google basically developed something called view through conversions. So that meant even if you didn't click on the display ad, it has some kind of benefit, right? So they're gonna say when a person is exposed to that display ad, if there's a sale or purchase that happens within the next 30, 60, or 90 days, we're gonna attribute that sale to the display ad. That's a very, very rudimentary way of doing attribution. And it does work in certain cases, but it's also been now abused. All right. So, for example, you can say, oh, you know, that individual display ad didn't drive that sale that happened 30 days later. It just happened 30 days later. And you can see a scenario where the person was going to buy the thing anyway, and they just happened to be exposed to the ad. It wasn't caused by the ad. You see the difference? So the problem is these attribution or conversion models over-attribute or overclaim credit for the sales that would have happened anyway, or that would have happened in the absence of the advertising. So, what a lot of advertisers are not yet doing is focusing on incrementality, which means the sales that would not have happened in the absence of the advertising, right? We actually want to say we spent these dollars in digital. Did it actually drive more sales than those that would have happened

Closing And Where To Learn More

Dr. Fou 18:20

anyway? Right? That's really the key question they should be asking. But kind of to close out your question, it's they're seeing sales happening, and they have attribution models that say, oh, those sales are caused by the ad impressions, even though they weren't. So that's why a lot of advertisers believe that it's working, even though there's a whole bunch of fraud, like it could be 90% ads shown to bots, the sales still happened. But then the model said, Oh, yeah, it's it's you know, those sales are attributed to these digital campaigns that you have running. That's why you believe, and there's a whole bunch of people who wanted to believe that digital works really, really well. Okay, so that's how it covered up the fraud, right? Even if the campaign had 90% bots in it, impressions were shown to bots, they'd still had sales going on, right? They still had purchases, conversions, all that kind of stuff. So they thought it was working. But careful marketers and probably more advanced marketers are now kind of trying to tease that apart and saying, okay, well, we can run turn-off experiments, like just turn off the digital campaign in one state and see if the sales in that state continue at the same rate, right? Those are simple experiments they can run to say, was there a cause and effect? And then now to kind of close out the correlation versus causation thing. Correlation is just most of the sales and the digital marketing, it's just correlation, right? They happen at the same time. It's not causation. But I think more and more marketers are kind of turning on to that and getting smart and say, okay, that's not good enough. We really have to figure out the causation. So there's different methodologies, there's you know, media mix models. And that kind of stuff. But you have to be very careful where errors are introduced into those models and you're still overclaiming credit for sales. So you've got to be careful to only focus on the incrementality.

Tim Rowe 20:11

Someone that's listening today, they're maybe either buying CTV, they're leading video investment for a brand, or they're leading a local sales team and trying to navigate conversations like this to drive adoption. What's the one key takeaway that you want a listener to leave with?

Dr. Fou 20:29

I think the best way to think about this is approach it as if you were a small business owner. Okay, so small business owners have finite budgets. If they spend $1,000 and they don't get any more sales, like any more people walking into their grocery store or anyone, any more people sitting in the chair at their barbershop, they can't spend the next $1,000 on digital ads. So, you know, the big marketers, they have so much budget. And they have these media mix models and all that kind of stuff that kind of attribute and tell them things are working, and you have great ROAS on it, right? Return on ad spend on it. Those are the ones where they're spending way, way more money than they should be. So it's almost like think as if you were a small business owner and you have to actually look at real outcomes. And then by the way, when you turn off the spend, the sales should be going down, right? Or should go away. So if it's not, then you can tell the digital marketing or the CTV ads that I'm buying did not cause those sales, right? Did the sales continue when you turn it off? This was this goes back to, you know, like an the Uber uh fraud example from 2018. They were, they thought they were very clever, and they said we would only pay when we get app installs, right? That's the ultimate conversion, right? Get someone to install the Uber app. So what do you think the bad guys did? They faked the reporting. Yeah, exactly. Well, not even real app installs. They just faked the reporting to make it look like there were app installs. There were some of these bad guys that were so bold, they didn't even run ads. They didn't even have real devices that installed the app. So literally, it was all fake. So what Uber did, and this was an analytics person there, it wasn't because of any special tech or anything. Kevin Frisch just said, let's let me just pause the spending for a week and see what happens. He paused the spending for a week, app installs continued. He paused the app the spend for another week, app installs continued. And he said, Okay, F this, I'm gonna just cut it all off, app installs continued. Because humans wanted to install the Uber app anyway. It wasn't because all the spend in those mobile exchanges caused the app installs. So, based on that, I mean, most advertisers can do that themselves. They don't need any special tech to detect the fraud. They can just run these turnoff experiments. And you don't have to turn everything off, right? Because you still have to do marketing, but just choose a market, right? New York or California or whatever, turn it off and see if the velocity of sales and conversions in that market changed. If it didn't, then that's a data point for you. Okay. Not that many sales and conversions were caused by the digital marketing. There may be other things like actual billboards in the physical world that you can buy because people actually walk by them or drive by them. Right? In digital, it's so easy to hide the fraud because everything's just bits and bytes. There's so many fake sites, so many fake apps. No advertiser is going to review an Excel spreadsheet that has two million rows in it. Right? So they just ignore it and they just assume, okay, yeah, they the vendors tell me it's working, so it's working. So that's where we've gone wrong for many years. But I think it's time to make digital marketing better.

Tim Rowe 23:50

I agree. And you're working on solving that again in 2026. Foo Analytics has been out for a decade and a half, but you are launching audits. Can you tell us about that?

Dr. Fou 24:01

Yeah, it's it's very simple, but the word audit, I think, is both scary, but it's also something that people know they need to do. So we've basically put Foo Analytics tags into the ads, say, for example, CTV ads. The audit is basically taking a look for the advertiser to see where the ads are going and whether the ads even ran. Okay, so if it didn't run, you're not getting what you paid for. And then if those ads went to crappy websites and mobile apps, those are not CTV, right? You paid CTV prices and it went to crappy mobile apps and websites. So those are the kinds of things that the audit will help surface that I don't think the advertisers currently know because they're not getting any of those insights from the placement reports or even the log level detail data that they get. So it's really taking an independent look and seeing if there's areas of improvement. I'm almost certain that there will be, especially in CTV, because the fraud is so rampant and so easy to do.

Tim Rowe 25:02

Dr. Fu, this has been a great installment, I think, in reimagining the way that we approach digital marketing. I think that that's the great takeaway from this is hey, here's a permission slip to do it different. The way that it's been done doesn't necessarily mean that it's the only way to do it. And there's clearly a lot of opportunity here for us to all get better. If folks want to learn more about foo analytics, learn more about the audits, connect with you, read some of the great content, where should they go?

Dr. Fou 25:31

Fooanalytics.com, so F-O-U analytics with an S.com, or just Google my name, Augustine Foo, and I'm all over the LinkedIn, and I have articles uh with charts and stuff like that, so you can see the data.

Tim Rowe 25:44

Definitely not hard to find. We'll make sure that it's all easy to find close by to this episode. Dr. Foo, thank you. Thank you, Tim. And if you found this conversation to be helpful, please share with a colleague or a client. Start a conversation today, and we'll see you all next time.