That proof can be relatively straightforward when ad dollars are spent online – with analytics able to tell us who’s responding to our ads, what they’re doing on our websites, how much time they spend there and whether or not they complete a purchase.
The detail gets a little fuzzy when we start measuring offline mediums like TV. Sure, most advertisers know audience reach, some demographics and probably some level of top-line results. But it doesn’t match the depth and granularity of data we can get for online campaigns.
Most of us are true believers that TV works. An Adweek article from Nov. 3, 2008 says that, even with media budgets in peril, there’s no severe pullback in network TV spending for the first quarter of 2009. Likewise, cable is business-as-usual, so far.
But too many TV advertisers concede defeat in the measurement game – unnecessarily. I continually confront the attitude that nothing less than “perfect tracking” will do. What this often leads to is a constant reinvention of metrics and tracking procedures in search of this elusive perfection. In some cases “perfect tracking” seems so difficult or impossible that anything beyond high-level measurement of results is postponed or avoided.
The fact is, TV (and offline tracking in general) can be much closer to perfect than most people think is possible. Just because TV analytics are “fuzzy,” doesn’t mean the numbers are soft.
An understanding of “fuzzy logic” can help us make TV ad measurement nearly as precise as online.
Fuzzy Logic: Embracing the Reality of Imperfect, Incomplete Information
One glance around a typical office reveals the reality of fuzzy logic. Try to answer one question simply by gathering data from what you see when you look around. Here’s the question: Is your office hot or cold?
First clue: Phil the receptionist is wearing a sweater. I guess it’s cold. But his desk is in front of the main door, and every time someone opens it a gust of cold air from outside slaps him across the face.
Another clue: Diane, who’s got an office on the west side of the building, is wearing short sleeves and is continually wiping sweat from her brow. It must be hot. Of course, it’s almost the end of the month and she hasn’t met her sales quota. Plus, the sun’s shooting darts through her window.
Clearly, answers like “the room is hot” or “the room is cold” don’t begin to reveal the true complexity of the situation. There’s no way to accurately answer without supplementing what we see in the room with what we know about the people and the environment. This is why we have “fuzzy logic.”
The Association for the Advancement of Artificial Intelligence explains here: “The human brain can reason with uncertainties, vagueness and judgments. Computers can only manipulate precise valuations. Fuzzy logic is an attempt to combine the two techniques.”
To gather data, and make decisions based on that data, requires a system that can take into account imperfect, incomplete and even subjective information.
Fuzzy Analytics: Start with Imperfection, Improve as You Go
To deal with the relative ambiguity of TV ad measurement, I adapt the concept of fuzzy logic into one I call “fuzzy analytics.” Here’s how it works: Find a level of tracking we can do, accept its imperfections, gather data, analyze it and improve our ability to understand it as we go. It will evolve into a system that is nearly as accurate as following a click online.
Here’s an example: Hotwire.com. My agency has been working with the online discount travel site for about four years, and we’ve focused primarily on television.
As is the case with many dot-coms, the name – Hotwire.com – is both the brand and the destination to which we’re driving consumers. Since we’re not driving to a unique URL or phone number – as many DR campaigns would do – ROI is more difficult to parse.
The key is to determine what the baseline is: What would we expect the business to do today if there were no advertising running? Hotwire started with a measurement model built on assumptions, and applied new information from day to day, honing the model over time.
Today we have a good sense how our advertising is performing. We know where and when we’re getting results, and we can determine the cost of each incremental customer. We’re about 90 percent confident in the decisions we make about how to allocate our budget across networks, shows, days, dayparts, and even our creative mix (when and where to run :30-second spots, :15s and :10s).
Here are some of the guidelines we follow:
Know the company intimately. Get inside access to key company-wide performance indicators. If no advertising were running, how many calls, website visits, or in-person visits would the company receive from prospective customers? From what geographic areas? How many would result in sales? There’s no shorthand here: you need to know the patterns and trends that already exist. Year-over-year historical data is very helpful, but even new businesses can use whatever data they have.
Get buy-in. Talk to each of the stakeholders and make sure that everyone understands how the baseline model is built and how success will be measured. As results come in over time, get input from this group to hone the model to the point that everyone understands the value of the data being examined.
Do the work. Gather the data from your campaigns and compare it to your baseline. Analyze it and make the incremental changes as you go. Make note of the peculiarities that may have existed for that time period, and make allowances for those. It will evolve into a highly efficient analytics machine.
Compare what your performance model is telling you with big company-wide metrics like profitability. Perhaps this seems obvious, but I’ve seen many professionals get lost in their own analytics and forget the big picture. If your model is telling you the campaign is going well, then overall metrics for the business should be going well.
Never burden your potential customers with your tracking. We fall into this trap when we create custom URLs simply to track the campaigns (yourcompany.com/offer6, for example). It’s likely to reduce your overall response rate. It also dilutes the brand message and puts the burden on consumers to remember a longer URL simply so the advertiser can measure results.
Measurement doesn’t exist for its own sake, of course. The more we know about how our campaigns are performing, the more we can optimize our strategy for increased success. The Hotwire.com campaign is ROI-positive – meaning it achieves a positive return on every ad dollar spent. And we have the analytics to prove it.
— Express your opinion, commnet below.