Note: This is a really interesting post - please don't get my commentary wrong, in general I think he's spot on.
> Modern marketing is all about data and however hard you might try, you can't spend any time around marketers online without being subjected to endless think pieces, how-to guides, ebooks or other dreck about how we need to track and measure and count every little thing.
Yeah, there's a ton of chaff out there from marketers selling to, well, marketers. It's a fair point about the state of things that there is way too much noise and not much signal.
> We've got click rates, impressions, conversion rates, open rates, ROAS, pageviews, bounces rates, ROI, CPM, CPC, impression share, average position, sessions, channels, landing pages, KPI after never ending KPI.
Yep, too many KPIs! The main problem facing marketing is often a plethora of information that breeds useless analyses, diverting focus away from just buckling down to gauging incrementality.
> That'd be fine if all this shit meant something and we knew how to interpret it. But it doesn't and we don't.
I disagree with this. When we get way too far into the weeds we can lose perspective, but plenty of these metrics cited above are not just meaningful but _critical_.
> The reality is much simpler, and therefore much more complex. Most of us don't understand how data is collected, how these mechanisms work and most importantly where and how they don't work.
Truth.
> 36% percent of people in the UK use an adblocker
This is probably skewed, as the data is based off a survey where respondents using an adblocker are probably more likely to respond. The best way to get information like this broadly across the industry is comparing server and JS-based analytics logs to get real ratios, and it is closer to 8% when I last was part of a large scale test (2.2bn sessions across 84 countries). Even that number varies highly by country, device, demographics and what the user is viewing.
You could probably solidly bet that ~40% of HN users have ad blockers, but <1% of CNN viewers do. That's how big of a swing it is.
> which means your javascript based website tracking is meaningless
Not entirely true. JS based analytics is useful, to a point. It should always be considered one signal of many, and use it for insights, but not necessarily as a source of truth. _Anyone relying on web analytics at scale for insights should also be using server-logs as their source of truth_.
> Email open rates don't actually indicate that an email was opened, merely that a request was made to a server
Amen! They are generally BS and should only be considered an indicator, and your email CTR should always be based on send->click. That's it.
> The black boxes inside Facebook and other ad exchanges give you flat out wrong data about how your ads are performing
Meh. The video thing is old news and frankly only mattered about completion rates. If we're talking about FB, what matters is reach and impressions that drive overall lift in your business. FB has _by far_ the best holdout testing in the industry that gives a solid gauge of your total return on impressions. Clicks on FB, YT, display, etc. are just a signal - the real goal is who saw your ad and then takes action afterward.
> The audiences you're targeting on Google, Bing, etc are fraudulent and don't even exist
This is a tricky one. If you're talking self-reported content targeting on display, yeah - it's crap. IF you're talking about FB/Google/Bing/Oath audience data based on behavior? Solid gold, in descending order on that list. Don't forget that lookalike audiences are also an amazing tool, and they're definitely not fraud if they perform as well as they do.
> The exchanges you're purchasing media space from are cheating you
No comment on the linked article. And yes, there is massive amounts of fraud in the mobile ad ecosystem.
> And even if we know how the data is collected, what it means and what it's actually tracking, most of us don't have the technical chops to analyse the data we've collected1. I don't mean to rag on anyone by saying this, but we do need a reality check.
This is getting better imho. Over time Marketing Analytics has become a solid discipline and I've been incredibly fortunate to work with many people that have produced amazingly solid insights with massive impact. It is getting better, but like I said before we often get way too far in the weeds and lose perspective. Simple is often the solution.
> And look. I get it. Having tangible data allows us to demonstrate that we're doing our job and we're trying to measure and improve what we're doing. But as Bob Hoffman rightly points out - that's not how brands are built.
> The numbers are often all we have to prove our case, to get more budget and in extreme cases, to continue to stay employed. We'll remain in this mess until we can separate marketing from short sighted and poorly informed decision making. Until leaders can lead on the strength of their conviction and experience instead of second guessing themselves and their staff based on the inadequacy of data.
> I don't know what the way out of this mess is, or what the path to success looks like. All I know is this.
> We're addicted to bad data.
I _absolutely_ agree about separating marketing from short-sighted and poorly informed decision making. This is the single biggest issue facing us right now and it continues to hold growth back in so many companies.
Note: This is a really interesting post - please don't get my commentary wrong, in general I think he's spot on.
> Modern marketing is all about data and however hard you might try, you can't spend any time around marketers online without being subjected to endless think pieces, how-to guides, ebooks or other dreck about how we need to track and measure and count every little thing.
Yeah, there's a ton of chaff out there from marketers selling to, well, marketers. It's a fair point about the state of things that there is way too much noise and not much signal.
> We've got click rates, impressions, conversion rates, open rates, ROAS, pageviews, bounces rates, ROI, CPM, CPC, impression share, average position, sessions, channels, landing pages, KPI after never ending KPI.
Yep, too many KPIs! The main problem facing marketing is often a plethora of information that breeds useless analyses, diverting focus away from just buckling down to gauging incrementality.
> That'd be fine if all this shit meant something and we knew how to interpret it. But it doesn't and we don't.
I disagree with this. When we get way too far into the weeds we can lose perspective, but plenty of these metrics cited above are not just meaningful but _critical_.
> The reality is much simpler, and therefore much more complex. Most of us don't understand how data is collected, how these mechanisms work and most importantly where and how they don't work.
Truth.
> 36% percent of people in the UK use an adblocker
This is probably skewed, as the data is based off a survey where respondents using an adblocker are probably more likely to respond. The best way to get information like this broadly across the industry is comparing server and JS-based analytics logs to get real ratios, and it is closer to 8% when I last was part of a large scale test (2.2bn sessions across 84 countries). Even that number varies highly by country, device, demographics and what the user is viewing.
You could probably solidly bet that ~40% of HN users have ad blockers, but <1% of CNN viewers do. That's how big of a swing it is.
> which means your javascript based website tracking is meaningless
Not entirely true. JS based analytics is useful, to a point. It should always be considered one signal of many, and use it for insights, but not necessarily as a source of truth. _Anyone relying on web analytics at scale for insights should also be using server-logs as their source of truth_.
> Email open rates don't actually indicate that an email was opened, merely that a request was made to a server
Amen! They are generally BS and should only be considered an indicator, and your email CTR should always be based on send->click. That's it.
> The black boxes inside Facebook and other ad exchanges give you flat out wrong data about how your ads are performing
Meh. The video thing is old news and frankly only mattered about completion rates. If we're talking about FB, what matters is reach and impressions that drive overall lift in your business. FB has _by far_ the best holdout testing in the industry that gives a solid gauge of your total return on impressions. Clicks on FB, YT, display, etc. are just a signal - the real goal is who saw your ad and then takes action afterward.
> The audiences you're targeting on Google, Bing, etc are fraudulent and don't even exist
This is a tricky one. If you're talking self-reported content targeting on display, yeah - it's crap. IF you're talking about FB/Google/Bing/Oath audience data based on behavior? Solid gold, in descending order on that list. Don't forget that lookalike audiences are also an amazing tool, and they're definitely not fraud if they perform as well as they do.
> The exchanges you're purchasing media space from are cheating you
No comment on the linked article. And yes, there is massive amounts of fraud in the mobile ad ecosystem.
> And even if we know how the data is collected, what it means and what it's actually tracking, most of us don't have the technical chops to analyse the data we've collected1. I don't mean to rag on anyone by saying this, but we do need a reality check.
This is getting better imho. Over time Marketing Analytics has become a solid discipline and I've been incredibly fortunate to work with many people that have produced amazingly solid insights with massive impact. It is getting better, but like I said before we often get way too far in the weeds and lose perspective. Simple is often the solution.
> And look. I get it. Having tangible data allows us to demonstrate that we're doing our job and we're trying to measure and improve what we're doing. But as Bob Hoffman rightly points out - that's not how brands are built.
> The numbers are often all we have to prove our case, to get more budget and in extreme cases, to continue to stay employed. We'll remain in this mess until we can separate marketing from short sighted and poorly informed decision making. Until leaders can lead on the strength of their conviction and experience instead of second guessing themselves and their staff based on the inadequacy of data.
> I don't know what the way out of this mess is, or what the path to success looks like. All I know is this.
> We're addicted to bad data.
I _absolutely_ agree about separating marketing from short-sighted and poorly informed decision making. This is the single biggest issue facing us right now and it continues to hold growth back in so many companies.
It is getting better - slowly, but surely.