Fox in the Henhouse: Should You Trust Your Agency with Analytics?


trust_small.jpgADOTAS — Advertising agencies—and especially digital agencies—are among the most fascinating companies in the world. They are staffed (most of the time) by whip-smart, sometimes arrogant, fun, creative, young, attractive people.

Doesn’t it always seem like the good-looking smarty-pants you knew in high school ended up working at an agency? Sometimes, maybe when you’re slogging through yet another column of analytics numbers or trying to figure out how to make your analytics tool recognize a measure it wasn’t born to measure—admit it—you get a little jealous. Those agency people are probably skateboarding down the hall right now, coming up with a new way to make oven-cleaner cooler than Lady Gaga.

Agencies are soaking in the suds of what everybody now calls “spin.” You know—lemons become lemonade. Pigs can fly. And ho-hum conversion numbers can be made to seem either “not that relevant” or hidden behind a pitch for new creative. Agencies are amazingly good at taking things that are not so wonderful and making them seem like just the thing you wanted.

All of which is precisely why you can’t let your agency perform your analytics work.

What do you want most out of your hard numbers? You know—like the ones you get from the bank? Or the ones you get from your sales database? If you’re like me, there are a couple of things: accuracy of course, but also you need to trust that they are not getting “spun” for you by somebody who’s got a vested interest in making them look better than they have a right.

You can see where this is heading.

Let’s get to the “accuracy” part first, because that one is easier to handle. Like I said before, agencies and their digital counterparts are creative shops. Only in recent years have they attempted to “do” technology—mostly creating web sites. Which, as any real geek knows, is definitely technology-lite compared with crunching application code or running a massive network in real time—or getting analytics to work the way it’s supposed to.

Agencies have done a great job building sites. Nearly all the major sites today have been built by agencies. And agencies are chock-a-block with experts in Flash, Photoshop, HTML, XML and all the other “developer” skills that go into building award-winning sites.

What they lack is the really deep technology DNA that drives careful, accurate measurement. More and more agencies these days have talented web analytics people on staff, but they are often talkers and aggregators, not coders and configuration experts. And with few exceptions, agencies are certainly not devoting as much effort on ensuring accuracy in the numbers as they are on the creative for that next banner campaign.

Agencies will tell you, quite justifiably, that you go to them for “best of breed” expertise when it comes to creating awareness for your brand. But too often, those that attempt to bundle analytics with their creative projects are giving you “best of breed” creative with lukewarm analytics skills. Now I’m not saying agencies are hopeless when it comes to analytics skills. But best-of-breed? I think not. And if you’re in the market for the best (as you should be), the technology alone should send you looking in other directions than your agency for analytics.

The other problem comes down to the mind-set of agencies. You know, that whole “spin” thing. Here’s where I go back to my title: fox in the henhouse. Here’s how I recommend looking at it: if you knew your bank had a vested interest in showing you that you had more money in the bank than you really had, would you give them your business? Of course not.

Agencies often—and I emphasize often rather than always—sublimate the facts generated by analytics to the message that the account team wants to portray to the customer (that’s you). They will create some very pretty charts and graphs—prettier, probably, than your analytics tool makes on its own and prettier than a web analytics consultant might make—but in many cases the charts and graphs are built to tell a story that fails to be objective. And they can’t really help it.

Can most agencies really afford to tell you that the site you just spent a million dollars on is tanking? That the old, clunky one was better at conversions than the new, Flash-based one that takes forever to load and where everybody gets lost because the clicks don’t quite get to the server in time before they click again and cause havoc? The answer is no, in most cases, they cannot.

What’s the remedy? Let the agencies keep doing what they’re great at: building your brand online.

But think hard before you let them do the measurement too. Find an objective, third-party source for measurement—and the interpretation of the measurement. Then, schedule a meeting with your agency and either reward them for creating an amazingly effective site; or chew them out for creating that bloated, Flash-site where everybody gets lost.

In either case, you’ll know you’ve got accuracy and objectivity on your side.

— Express your opinion, comment below.


  1. I don’t agree for a couple of reasons.

    1. Your agency is supposed to be a trusted partner to your business. If you can’t trust them to be straight with you, then you should fire them.

    2. The analytics data exists, it’s out there in the form of real numbers and you, as a brand, should request it if you have any questions. If the agency can’t or won’t share the data with you then see #1.

    The convergence between creatives and crunchers is already happening and the idea that they should somehow remain separate is, in my opinion, a false dichotomy.

    Alex Fortney

  2. Is this an infomercial? What an immensely self-serving piece of junk. Seriously, you expect us to trust your opinion given that you own a web analytics shop? Would rather Adotas print the word “paid advertisement” at the top of the page. Despicable

  3. I have to agree with Alex on this. If you are paying your agency for analytics services, they should be able to give you objective insights into your analytics data. As Alex said, if you can’t trust them for this information, you haven’t hired the right agency.

  4. I cannot speak for all agencies, but the solution is simple:

    Grant access and provide training for your client to use the analytics software. By allowing complete transparency of the data, you no longer risk auditing issues when a discrepancy is inferred from home-grown reports.

  5. I think that Alex nailed this on the head. Your agency is a partner if you can not trust them then don’t retain them. As a metrics professional at a digital agency I consider accuracy and integrity to be my top priorities. I lay out a measurement plan be for a campaign or site launch that identifies all the metrics that will be tracked and then update the plan when metrics are unavailable for what ever reason. The measurement plan becomes the guide for all reporting.

    I have worked with some of my clients “objective” third party analytic teams and they refuse to share how and what they plan to measure. That is a real issue.

  6. I appreciate the spirit of this article. Leave the numbers for all the mortgage industry refugees, who seemed to have escaped into the digital wonderland, to pour over.
    Creative agencies, even today in this niffy era of impressive analytics, should focus on being interesting and provocative. Animate a baby so it talks like a real adult! Dream up Sponge Bob Square Butt for Burger King etc.

    And this is a self serving comment from somebody who works for nerd heavy company. I’m tired of you chairs that don’t accommodate us squares.


Please enter your comment!
Please enter your name here