comScore, one of the leading web-traffic measurement companies, dropped a bombshell of sorts on the web analytic community when they released a study last week concerning the effects of cookie deletion on web visitor counting. This may not sound like the sort of topic to stir up angry debate, but the implications of the study are serious and far-reaching. They could potentially change your view of how much traffic your site (or partner sites) actually gets, how well your online campaigns are performing, how reliable many of your web analytic reports may have been and even how you are going to proceed with online measurement in the future.

Let’s start at the beginning. A cookie is a small data file placed on your computer when you visit a site. The cookie is put there by the site your visiting, and it’s used to identify your specific computer as one that has visited the site before and tie together behavior from visit to visit.

But before I go on, that last statement needs clarification. Because while it technically applies to every kind of cookie there are two different cookie types and the way they work makes for a subtle difference in understanding this last statement. The first and most common type of cookie is called a 1^st Party cookie. And it’s really what I’ve been describing. A 1^st Party cookie is placed on your machine by the site you visited. Simple.

But the clever folks who do advertising and measurement on the web had a kind of sneaky idea. If they place a single object on a site you visit (like a tiny image) and that object is served by them not by the site your landing on, then the image request is made to THEIR computers. And, along with that request, they can drop a cookie on your machine. And they can read THEIR cookie, whenever you visit ANY site that has one of their images on the page. This is called a 3^rd Party cookie. It doesn’t work any different than a 1^st Party cookie — but it does mean that the owner of the 3^rd Party cookie can see something that the owner of the 1^st Party cookie can’t. Namely, what site(s) you visit — with the emphasis on the plural.

Who drops third party cookies? Pretty much every Internet advertising and measurement company. Google, Yahoo, MSN, Omniture, WebSideStory, DoubleClick, ValueClick, yata, yata, yata. Keep in mind, the 3^rd Party cookie is anonymous — it doesn’t know who the visitor is. It isn’t a threat to privacy and it isn’t spam.

But despite all these “is nots”, the 3^rd Party cookie has gotten something of a bad name. And, unlike 1^st Party cookies that often serve a clearly useful function (like enabling portal customization), 3^rd Party cookies have no tangible consumer benefit. So as browsers evolved and Spyware companies latched onto the idea that 3^rd Party cookies weren’t good, problems arose in 3^rd Party cookie measurement.

You see, a user can configure their browser to reject all cookies, no cookies or just one kind of cookie. Since many users realize that 1^st Party cookies are actually useful, they choose to accept 1^st Party cookies but not 3^rd Party cookies. This led to a rapid increase in 3^rd Party cookie rejection — a problem that meant many visitors were slipping through the measurement cracks. In addition, Spyware companies built lists of common 3^rd Party cookies from advertising and measurement companies and generally set themselves up to delete these cookies whenever they found them.

Bad news for measurement. But the web analytics vendors (at least) had a pretty good response. They changed the way their systems worked so that the cookie being used for measurement was actually one dropped by the site you’re visiting. Suddenly, it’s a 1^st Party cookie. It’s much less likely to be rejected and much less likely (we thought) to be deleted. All is again good in the world of web analytics.

Because, you see, the cookie is vital to the web analytics world as the sole way of tracking visitors. Not page views, which are easily tracked. Not visits, which can be tracked sans cookie. But visitors. Cookies are the only web analytics technique in popular use for tracking the behavior of visitors and for counting how many visitors a site actually has.

Here’s a short list of measurement tasks you can’t do without a cookie:

o Count Visitors

o Track Campaign Success over time

o Distinguish between New and Repeat Visitors

o Distinguish between Prospects and Customers

o Analyze behavior across more than one of your sites

o Analyze your Sales Cycle

o Measure over-time Engagement / Loyalty / Frequency of Visitors

The list could be extended indefinitely, but the basic idea is this. Without cookies, web measurement has no understanding of any “visitor” behavior — only visits and page views.

Which brings me to the comScore study. comScore’s study tracked one very large internet portal site and one large ad-serving network. The portal site was dropping a 1^st Party cookie and the ad-serving network was dropping a 3^rd Party cookie. Here is a summary of their results from the comScore Press Release:

The study examined the degree to which users cleared cookies from their computers, thereby causing servers to deposit new cookies and potentially leading to overstated estimates of unique users when relying on cookie-based server data.
Average Computer Receives 2.5 First-Party Cookies per Site Each Month
comScore observed that 31 percent of U.S. Internet users cleared their first-party cookies during the month. Within this user segment, the study found an average of 4.7 different cookies for the site. Among the 7-percent of computers with at least 4 cookie resets, comScore counted an average of 12.5 distinct first-party cookies per computer, accounting for 35 percent of all cookies observed in the analysis.
Using the total comScore sample as a basis, an average of 2.5 distinct first-party cookies were observed per computer for the site being examined. This indicates that Web site server logs that count unique cookies to measure unique visitors are likely to be exaggerating the size of the site’s audience by a factor as high as 2.5, or an overstatement of 150 percent.

*Excludes log-in cookies
“While past studies from other research companies have shown a similar proportion of computers that clear their cookies, the comScore study is the first to highlight the disproportionately high percentage of cookies represented by those computers,” commented Dr. Magid Abraham, President and CEO of comScore. “For example, with just 7 percent of computers accounting for 35 percent of all cookies, it’s clear that a certain segment of Internet users clears its cookies very frequently. These ‘serial resetters’ have the potential to wildly inflate a site’s internal unique visitor tally, because just one set of ‘eyeballs’ at the site may be counted as 10 or more unique visitors over the course of a month. The result is a highly inflated estimate of unique visitors for sites that rely on cookies to count their audience.”
Third-Party Cookies Deleted at Nearly Same Rate as First-Party Cookies
comScore’s analysis of third-party cookies revealed an average of 2.6 distinct cookies per computer in December, indicating a similar rate of overstatement as the first-party cookies. For those computers where at least one cookie reset occurred, the number of third-party cookies observed was slightly higher than first-party cookies at 5.5.
Everyone in the web analytics community has long understood that people sometimes delete their cookies. However, there are three aspects of the comScore study that are particularly significant. First, the percentage of cookie deletions in a month is on the high-end of expectations (31%). Second, the frequency of cookie deletion is much higher than expected — meaning that cookies aren’t just deleted once a month. Indeed, the biggest impact comes from a group of visitors who appear to delete cookies after every session (a settable option in Firefox). These “serial deleters” — if they are frequent visitors to a site — will get counted over and over again as distinct visitors. Third, 1^st Party cookies fare no better than 3^rd Party cookies when it comes to deletion. This last result is very surprising to most of us — and indicates that cookie deletion is not typically a function of spyware protection software (much more likely to delete 3^rd Party cookies) but of browser settings or manual erasing.

Since comScore focuses on visitor counting, most of the discussion around these results has also focused on this. And it’s an important issue. Some in the web analytics community argue that errors in estimating Reach aren’t that important — since these numbers are used for comparative and trending purposes.

Unfortunately, that isn’t really true. First, most companies these days are evaluating Reach by marketing channel — so it’s important to know how the web site compares to other possible marketing efforts. And second, even on a comparative basis, there is every reason to believe that different sites (and even the same site over time) will be differently impacted. At least two (and probably more) factors will obviously drive differences in site measurement: the degree to which a site has lots of heavy repeat users will significantly impact the degree of overstatement in counting and the degree to which the user population uses Firefox will probably have a significant impact as well.

Because of this, a portal site with a very high return rate will likely show a much higher degree of traffic inflation than an eCommerce site. A daily news site will show higher traffic inflation than a health portal. A site for technical users that is heavy on Firefox adopters might be much worse off than a cooking site. Sites that raise privacy issues or concerns (like pornography or medical sites) may be harder hit. And so on. Worse, if you change your site in ways that increase Engagement, this change will artificially increase your Reach numbers as well. Meaning your own site may not be comparable to itself over time.

In addition, this implies that no simple adjustment to web analytic numbers is applicable. You can’t just take web analytic visitor counts and divide them by some correcting factor.

As troublesome as the implications are for Reach, they are even worse when you consider the impact on your broader web measurement issues. Remember the list of tasks that rely on cookies? Many of these analyses (including campaign attribution) are going to be seriously distorted unless the analyst controls carefully for the cookie deletion problems. Even with thoughtful planning, some types of analysis (such as how New Visitors behave) may be nearly impossible for some sites.

Are there good alternatives to using half-baked cookies? Not always. Certainly, there are alternatives for measuring your Reach as comScore is no doubt happy to point out. But from a web measurement perspective, there aren’t many easy solutions. However, being aware of the extent of the problem may be at least some protection. A deep understanding of the problem may keep you from acting on an analysis based on flawed data and secondly it may help you screen-off your analysis from the effects of cookie deletion. There are, in many cases, techniques that will accomplish this. In addition, understanding the extent of the problem may cause some sites to focus on the possibility of “opt-in” measurement — getting visitors to register and log in consistently. It may also drive the measurement community toward a better technical solution.

No matter how you look at it, the comScore study isn’t good news. But the right answer is probably not shooting the messenger. Web analytic data has always had a high degree of messiness. But the fact that data quality isn’t perfect doesn’t also mean that any level of data imperfection is acceptable.
An analyst must be able to understand the issue and all its measurement implications to be able to make a sensible judgment about what to do differently and how to adjust site measurement. If you fob off the comScore study with a Gallic shrug, “That’s just the way web data is…” then it won’t just be your cookies that are half-baked!

That’s dangerous. Because measurement — if it isn’t ignored — drives change in particular directions. And if you are measuring only the conversion efficiency of your site, then pages and tools that drive to conversion will get all the attention. Pages and tools that may drive significant brand value will be ignored or even dropped. In the long run, this will lead your site to be poorly optimized — even when stakeholders are aware of the multiplicity of functions on the site.

This phenomenon of measurement driving misoptimization is surprisingly common. It appears regularly in Search Engine Marketing — whenever companies choose to optimize to clicks. And the potential for it exists whenever measurement has been focused on only one part of a larger picture. The inevitable human and organizational tendency is to optimize to the numbers you have.

So if you believe that branding/marketing is a significant function for your website, it’s imperative that you bake those concepts into your measurement in a way that will drive intelligent decision-making. To do this, you need to come up with a measure of branding/marketing that is:

• Weighted Appropriately vis-ÃÂ-vis Online Conversion
• Designed to encourage appropriate optimization
• Measurable using your tool set

I’ve ordered these starting with the most difficult first. Getting a handle on the value of a brand impression on the web is far from trivial. Many organizations have done media research in the past to understand the potential value of an “impression.” But such values — when taken from traditional media — may not apply particularly well to the web site. In addition, web site engagement is much more variable than with other forms of media. When a consumer watches TV or hears a radio spot — they pretty much have the same experience. Granted their level of attention may be dramatically different — but there is a sameness to the experience (as well as an outright impossibility to measure differences in engagement) that make it both necessary and plausible to establish a single value to the impression. The same cannot be said for the web — where the depth of engagement on the site is both visible and dramatically different by user.

There are several studies underway to try and measure the “brand” impact of various pieces of the web channel — including banners, search, and the web site. These studies track consumers over a long period of time for a single property and strive to measure the effect of each type of consumer interaction. Without denigrating the value of such studies, you should be cautious about assuming that whatever results they document are necessarily germane to you.

All too frequently, a single case-study becomes the conventional wisdom for an industry — with few people understanding just how variable real-world experience can actually be. A perfect example of this is the issue of Organic Cannibalization. When we’ve actually measured this, we’ve seen real-world experience ranging from organic listings actually supporting paid listings (as conventional wisdom says they should) to significantly (to the tune of 50% or more) cannibalizing it. And we’ve measured effects at pretty much every level in-between (See here for one example of this).

On a similar theme, a recent study of the effect of TiVO on buying habits (Advertising Age, March 5, 2007) revealed a wide range of impacts by brand — from significant declines in advertising effectiveness to virtually no impact. The researchers had theories why this was the case (type of program advertised on, effectiveness of advertising, cost-competitiveness of the market space), but these are really only guesses. And, as with so many real-world cases, the deep implication is that there was no one right answer applicable to every advertiser.

This drive to explain everything with one simple rule is everywhere. If you listen to reports on the stock market, you’ll hear the same impulse to simplistic explanation (and with a much higher nonsense factor) — the market was “spooked” by some piece of news or was “reacting to comments” by someone. Maybe. Maybe not. There’s always something to explain a movement, but in a market movements will inevitably occur and some outside fact can always be found to match. Variation happens.

So depending on the importance of this issue to your business, it may be worth investing in a tracking study of your own. If that’s too elaborate, then you’ll have to be satisfied with building a value answer out of a subjective weighting of key factors: the known value of a traditional media impression, survey research on the brand impact and satisfaction from web site usage, and the long-term study of website usage and retention.

Once you’ve decided on a relative weighting of branding/impressions vs. conversion, you’ll have to decide how to measure for that value. The key, in making this decision, is avoiding a measure that encourages misoptimization.

Eric has a webinar coming up with Aquent and the AMA on web analytics targeted to a very broad audience. He and I “sat down” via IM and had the following conversation about the webinar, his books, building web analytics into a company and web analytics as a discipline. If you’re in online marketing (and why else would you be reading ADOTAS?), then you need to have at least a baseline understanding of web analytics. And Eric is one of the very best people to gain that understanding from.

G. Angel: Eric, I take it that your upcoming AMA webinar is about getting people started in web analytics. You’ve probably had as much experience with that as anybody. What kind of things will the webinar cover and (by inference) what kind of things does it take to get people started in web analytics?

E. Peterson: Yep, when Aquent and the AMA approached me about this web event they said they wanted to provide a presentation that would level-set web analytics for a very broad audience. They felt that “Web Analytics Demystified” would be an excellent basis for this.

In my presentation I will cover “Ten Steps to Web Analytics Success” ranging from “Defining Your Business Objectives” to “Hiring the Right People” to “Adhering to the Process”

All ten strategies are, in my experience, critical to companies getting the type of return on investment from web analytics that is universally being sought.

G. Angel: You mention Web Analytics Demystified – but you’ve written several books – talk about the books you’ve written — how they fit together and who/how you think people should approach them. Is Web Analytics Demystified the best place to start?

E. Peterson: Well, I’m biased in this regard but yes, I think that Web Analytics Demystified is an excellent starting point for companies either just now considering an investment in web analytics or considering a re-investment in web analytics.

I recently re-read the book, something I’d written back in 2003 and 2004, and was pleasantly surprised to realize that almost all of what I said then remains true. Web Analytics Demystified is about the basics — what you need to know to get started and where to go when you’re up and running.

My second book, Web Site Measurement Hacks (O’Reilly 2005) is a collaborative effort with many of the great thought leaders in our space — folks like John Marshall and Dr. Stephen Turner from ClickTracks, Jay McCarthy of WebSideStory, Bryan Eisenberg from FutureNow, and Jim Novo from the Drilling Down Project. WSMH is good for folks who are a little more advanced.

And my third book, The Big Book of Key Performance Indicators really drills down into the topic of reporting using KPIs — their definition, distribution, and use.

You’re description of Web Analytics Demystified sounds like beginning from a company perspective – what about an analyst who’s a new hire or someone trying to learn the business – same story?

E. Peterson: Well, if you’re new to the space you should start with Demystified but most folks in the space tell me they’ve read the book and have moved on to Hacks and TBBOKPIs to deepen their understanding of their jobs.

I was lucky enough to get into writing about the subject early on … for years it was just the great Jim Sterne and I.

One of the most frustrating aspects of SEO is that it is, in many respects, deeply stupid. When a user types “marketing” into Google (a query that will return something like one billion entries), it’s absurd to think that any search engine can possibly pick a “best fit.” Absurd because the task is inherently impossible — there is no best fit. And while the problem is most obvious for the broadest terms, it exists at every level of specificity. Take the key term in our (Semphonics) industry — web analytics.

When a user enters web analytics should they get the site of a web analytics software package, a web analytics consultancy, a web analytics association or a web analytics blog? And between software sites, all of which are essentially 100% about web analytics, how would it be possible to choose between WebSideStory, Omniture and Webtrends (or a half-dozen other vendors)?

It isn’t possible. But the static nature of the returned search page places an enormous value on being in the top position — more, for example, than in the phone book when the alphabetical listing places some premium on being named triple-A but not nearly as much as the top position in an engine.

I think this is going to change dramatically in the next couple of years. The Search Engines are going to have to deliver results in a much richer — Ajax driven fashion. When your search results can be instantly sorted, filtered and categorized, being number one on return is going to be mean much, much less. So I think in couple of years the practice of SEO is going to look more like the rest of marketing. It will be easy to get included in a germane result set. And marketers will have to focus on making their listing as attractive as possible. That should be an improvement in every respect.

But at the moment, the technical discipline of SEO is still very much with us. And since it’s with us, it’s important to do it as well as possible.

In Search Engine Marketing, doing something well is always ultimately about your ability to measure and tune your results. And in this respect, at least, our black sheep runs with the flock.

Yes, measurement. SEO and measurement haven’t exactly been soulmates. But SEO will reward disciplined measurement just as much — perhaps even more — than PPC.

Measure for Problems

So many people get lost in the nitty-gritty of SEO or intimidated by its intricacies that they miss the big picture staring them in the face. The first place to start when measuring your SEO traffic is to look at the overall organic traffic on your site by search engine. For most sites we work with (classic BtoC eCommerce), Google is going to be the dominant organic traffic source — typically sourcing anywhere from around fifty-percent to as much as seventy or eighty-percent of organic traffic. Yahoo is usually the next largest organic traffic source — typically pushing somewhere between 15-30% of traffic. MS Search is usually number three — with numbers ranging from nearly equivalent to Yahoo to percentages as small as 1-2%. Ask is the most common fourth source — typically with volumes about half of Microsoft.

I’ve expressed these as wide ranges because almost every site will show a different mix. But it’s useful to know the general expected ranges so that you can quickly place your site in comparison. That will give you a good idea of where you have opportunities by engine. If your Google results account for 95% of total organic volume, then you need to concentrate on optimizing for Yahoo, Microsoft and other engines. If Google is sending you less volume than Yahoo, then you’ve got a serious Google problem.

In the main, we find that companies discover issues with Yahoo and Microsoft. These engines aren’t as exhaustive in their coverage as Google, are less commonly optimized for, and are defeated by a wider variety of spidering issues. If you find that your site is under-represented in one or more engines, that provides you with direction both where to look and, in some cases, what kind of problems to look for.

Like most such decisions about “DIY” versus outsourcing, there aren’t likely to be one set of right answers. Every company is different and no two vendor partnerships are ever quite alike. But there are some good reasons why a company might want to pursue this direction — and, if you are thinking about bringing your Search Marketing in-house, here are some key factors to consider.

Most outsourcing decisions — regardless of whether it’s payroll or PPC – share a common set of decision-factors. These include:

How core is the function in question to your business? You need to be good at your core — the things that will really differentiate you from your competitors. For most organizations, completing Payroll isn’t core. It’s necessary, but it’s not a differentiator. Is Search Engine Marketing a core function in your business? The more important internet acquisition is, the more likely the answer is to be yes. If all of your customer acquisition is via the web, then Search Engine Marketing is probably just too important to you to give over to a vendor — and if you don’t have the necessary skills in-house you should probably look at a path toward acquiring them.

How well does the function fit your culture? Business culture is very real. Some organizations are great at selling. Others at product development. Others at customer relationship building. The less suitable your overall business culture is to a function, the less likely you are to be able to succeed at it. And if you can’t succeed at it, then you’re probably better off cultivating a vendor relationship — even the function is extremely important to you.

What kind of culture is best suited to Search Engine Marketing? I’m not sure there is one right answer to this. For PPC advertising, a culture heavily influenced by direct response (DR) marketing is ideal. DR types usually take to PPC with a vengeance. PPC also rewards cultures that are analytic, measurement oriented, technical, aggressive and younger. Defining the best culture for SEO is less easy. SEO when well done tends to be a methodical process that rewards considerable attention to detail and patience.

These cultural questions aren’t just apropos to taking Search in-house. They are factors you should consider when evaluating agencies as well. After all, what was it about most Interactive Agencies that made them a logical choice for doing PPC or SEO in the first place? Nothing, really. Interactive Agencies generally grew up as Design Shops — they had virtually no experience in media buying (possibly the closest old-world non-direct-response function to PPC), and — despite words to the contrary — are usually persistent foes of any kind of analytic or measurement function. In short, it made business sense for Interactive and Creative Agencies to get your PPC business but it often made no cultural sense at all. And that’s probably why, at least in part, they’ve so often made a mess of it.

Which brings us to the next factor in thinking about outsourcing: the quality of your potential partners. Unless you’re budget is very large, you are probably looking at choosing from a relatively small set of local or regional agencies. When you look at a vendor, you should have good reason to believe that not only are they better at Search Engine Marketing than you are right now, but that they would remain better for the foreseeable future.

For many shops, all of the expertise is concentrated in one or two people. That’s really dangerous. If that person walks out the door, so does their competitive advantage. And Search Engine Marketing growth has increasingly made hiring program managers feel like operating a revolving door at a busy hotel.

Unfortunately for web marketers, however, Search is one of the most difficult parts of a website to measure and optimize. Why? Search is — in a way — the ultimate “dynamic” page. It changes with every term entered and, potentially, with every new day on the site. It potentially routes to every single page on the site. It is entered from almost any page on the site. And, with entry, is associated a Search Term that drives the results. And that search term typically has tens of thousands of different values on a well trafficked site.

Each of these factors make Search challenging to measure. Most of the popular web analytics tools go at least part way to solving these problems. With fairly minimal setup, they’ll provide several types of analysis that are quite useful. You can typically analyze each of the following:

• Paths to Search
• Search Terms Entered
• Failed Searches (returned no results)
• Endpoint Conversion by Search Term
• Paths from Search

This seems like a fairly comprehensive list — but in actual practice it’s decidedly unsatisfactory in situations where Search is a major component of a site.

Paths to Search can be interesting for sites looking to understand when Search is being used as a fallback in a failed navigation session. By comparing Search routes (search is the next action from a page) to Total routes, the rate of fall-out to Search can be mapped for a complete site. And there are sites, in fact, where this is the most interesting Search analysis. But where search is an important first recourse, by far the most common path to search is from the home page, with most other pages having a fairly similar (and not necessarily bad) drive to search rate.

Measuring Search Terms Entered can also be revealing. For sites where Search is a “fallback,” highly-used search term may actually be indicative of failure (and often suggest holes in site navigation). For sites where Search is a primary option, search term volumes can often help track visitor interest patterns over time. Neither analysis, is really helpful for understanding or tuning Search performance.

Failed searches (where a visitor enters a term and gets no results) are a useful tool for tuning the operational performance of search. A failed search is nearly always a bad thing (assuming the term entered is at least borderline relevant). With a decent internal search tool, failed searches are usually quite a small percentage of the total. Where problems do exist, fixing them will have a big upside. In most cases, though, this analysis will be only sporadically useful in tuning search.

Search is often analyzed in terms of endpoint conversion. In this regard, Search is primarily viewed in comparison to non-search sessions — with designers trying to analyze whether search sessions are more or less productive than browsing sessions. This analysis is complicated for ad-based sites by the fact that search sessions are nearly always shorter than browsing sessions.

Conversion analysis can also be quite interesting and useful — but it’s necessary to put it into perspective. On both retail and ad-based sites, the visitor who searches often has a fundamentally different mind-set than one who browses.

On retail sites, search behavior is often indicative that the visitor is further along in the sales-cycle and has focused in on a specific product. On publishing sites, a search is generally indicative that the visitor has a particular information need. In either case, the main point is that a simple apples-to-apples comparison of search vs. non-search sessions may be misleading. Marketers should generally look askance at statements like “Search visitors are 40% more likely to buy than non-search visitors. They may be meaningful, but, equally possible, they may not!

This leaves the analysis of paths from Search. The function of search, after all, is to move visitors to the appropriate place in the web site. Typically, search competes with other pages (called Router Pages) that do the same job by presenting likely links to visitors as they move through one or more pages. To understand how well Search is doing this job compared to the alternative it’s essential to understand how Search routes visitors. Hence the importance of path from Search. Unfortunately, there are two problems with actually understanding paths from Search that make this by far the most problematic aspect of search to actually measure.

In my last installment, I described a host of problems that beset sites when trying to use conversion as THE metric of choice. But no matter how many issues surround tracking conversion, it’s still a lot easier to deal with and more powerful than anything available to sites that don’t have conversions as a convenient yardstick for success.

For ad-based sites, there is no such thing a single conversion event. When a page is a billboard (banner-based), then the publisher most often makes money on a per-impression basis. In this model, page views are the primary driver of revenue. But there is a secondary dynamic — the more visitors click through on the banner, the more likely advertisers are to be happy. Yet every visitor that clicks through on a banner is lost to the site — so no further impressions get recorded (in that session). In other words, there is a built-in conflict in the basic site goals — the publisher benefits most from impressions without click-throughs but the advertiser benefits most from click-throughs.

For content-network ads placed on a pay-per-click basis, the publisher only makes money on the click-out. That’s straightforward enough, but if a site mixes banners and PPC links, then PPC links have the same impression loss that banners do.

The more engaging content is and the more tantalizing navigational paths that a publisher provides, the more page views (and impressions) a site will drive. But engaging content and tantalizing navigational paths may reduce click-outs — making advertising less successful and therefore less profitable.

So unlike ecommerce sites, ad-based sites have to struggle with a conversion event that is amorphous (revenue is driven by every page view) and contains within it at least two contradictory impulses (maximize views vs. maximize click-throughs).

For the most part, publishers resolve these conflicts sensibly enough by focusing on driving as much consumption of content as possible while selling to appropriate advertisers and letting “nature” take its course.

There is, of course, a usability tradeoff when it comes to maximizing impressions. The quicker and faster a visitor finds the content they are looking for, the better for them and the higher their likelihood of return. But the quicker and faster they arrive at a destination, the less impression opportunities they provide.

Most ad-based publishers recognize that optimizing the site experience — even at the expense of short-run impression maximization — is almost always the right strategy.
But here is where epiphenomena come in. Because in trying to figure out how valuable content areas are, publishers (or their web analysts) often completely miss the boat.

So just what are epiphenomena? Epiphenomena are effects or symptoms which are associated with a deeper effect or condition, but are not the primary cause of this condition. For example, rich people drive more BMW’s than poor people. But driving a BMW isn’t what makes you rich (quite the contrary)! Catholics are more likely to have a Christmas tree in their living room during the winter holidays. This doesn’t mean that setting up a Christmas tree in somebody’s living room will make them a Catholic.

On a website, epiphenomenal effects are incredibly common and surprisingly easy to misunderstand. In one recent analysis, we at SEMphonic segmented visitors based on whether or not they viewed some specific content areas on the site. The goal was to understand how effective these site areas were in driving same session impressions, overtime visits and lifetime impressions.

This problem is especially serious for website analytics — where the traditional measure of site effectiveness (conversions / visitors) is deeply flawed. But similar problems can exist in Search Engine Marketing programs and other online advertising campaigns. Techniques for solving these problems do exist, but you’ll have to think carefully about how you measure your web channel and what you report on.

Conversion as a Mantra
In the bad old days, web site effectiveness was measured by hits. Then by page views. Then by visitors. None of which meant a darn thing when it came to meaningful business results. So it wasn’t all that long before people finally agreed — it’s conversion that matters. You measure your website the way you measure the rest of your business — by its contribution to the bottom line. In no time at all, this became a mantra for good measurement. Are you measuring to conversion? No — that’s bad. Yes — that’s good.

And there is point in denying that measuring to conversion is a vast step forward compared to the earlier paradigms. It is not, unfortunately, quite as simple, straightforward or complete a solution to good measurement as has been thought.

It turns out there are a whole bunch of ways in which measurement to conversion can result in seriously misleading information. I’ve covered some of these in previous articles — especially Organic Cannibalization — where PPC programs “borrow” clicks from organic listings — potentially inflating conversions and understating the true cost per lead or acquisition by a significant amount. You can read more about the phenomena of cannibalization and its implications here on ADOTAS.

Surprisingly, perhaps, not only can different channels cannibalize each other, but the same channel can cannibalize itself. This is quite common in PPC programs (and, to a lesser extent with SEO) — cases where visitors frequently source from different search terms to the same site in one or more sessions. Since campaign tracking in PPC generally gives credit to the last campaign that drove a lead, one PPC campaign can cannibalize another. This is especially common with “brand” terms, and tracking conversion in this way can significantly mis-represent actual conversion performance.

Today, I’m less concerned with the myriad ways that different marketing channels can cannibalize each other and the effects that this can have on conversion and more concerned with understanding the effectiveness of the website itself as a channel.

There are five big problems with measuring the effectiveness of your website by conversion:

• Just as online channels compete and cannibalize, your web channel may compete and contribute with other channels. This can make your website look much better or worse than it otherwise might.

• Few websites are directed only to conversion goals. When you don’t factor out traffic that has other intentions than purchase, you unfairly penalize a website and you may commit a series of optimization mistakes.

• The effectiveness of a web site is heavily influenced by the mix of visitor sources you drive — so measuring effectiveness without regard to source efficiency can radically skew your perception of how well your site is doing.

• Many visitors may show up at a website “ready to convert.” Giving your website credit for these conversions is fine operationally, but it can hide significant problems in the ability of your website to actually convince prospects.

• Many components of your website have too small an impact on a conversion decision to actually measure if changing them has improved your efficiency — but cumulative changes can have a significant effect. So how can you measure these “micro” components in a way that makes sense.

Finding Qualified Buyers
The success of Pay-Per-Click (PPC) campaigns is largely determined by the precision with which you can place your ad in front of qualified buyers. For mass marketers, the emphasis is more on buyers — since almost everyone is potentially qualified. But for companies occupying a smaller niche, finding highly-qualified eyeballs is critical.

This is especially important when a web site is lead-generating — not converting. The cost of an unqualified lead to a web site that handles conversion is generally only the cost of the initial click. When a web site generates leads, however, the cost is often precious time lost handling or identifying useless leads. This cost can dwarf the initial PPC charges and can make a PPC program untenable even when it might be working on a straight cost-per-qualified-lead basis.

Competitive Terms are Inherently Well-Qualified
It’s a generally accepted fact in Search Engine Marketing that your own brand words will convert particularly well on your site, versus more generic keywords. But what about your competitor’s brands? Until earlier this year, buying competitor words was a relatively cheap way to get good conversion rate. After all, you are pre-selecting customers you know are savvy enough about your industry to know who the main players are, and thus these potential customers are ahead of the curve when it comes to search browsing. And again, these words tend to be much cheaper than generic words, “buy stocks” being much more expensive per click than “Scottrade”.

Traditional Buying Techniques Have Become Less-Effective
However, many of the search engines don’t allow you to purchase competitor terms — and new rules on Google make purchasing competitor keywords much less attractive than was once the case. Google’s Minimum Bid Pricing in effect places a minimum floor on your per click costs. Since your click rates are usually quite low when buying competitor brand terms, you will almost certainly be asked to bid more for the placement than the actual auction will warrant. The net result is that you can no longer buy exposure (and clicks) on competitor search terms for the nickel or dime that it used to cost.

In addition, there is always some risk when buying competitor names — both of confused branding and the threat of litigation for trademark infringement. Most companies are rightly conservative when it comes to exposure to the legal system. Whatever the right or wrong of this type of advertising, nobody ever wants to be sued!

In general, what our studies found is that PPC buys will often cannibalize organic listings — sometimes quite heavily — sometimes not very much at all. Cannibalization is likely to be heaviest for brand listings (but even this varies greatly). It is also generally confined to listings where the PPC buyer has a top 3 position in the organic listings. Finally, they found that in every case, PPC buys provided at least some incremental lift over the organic listings alone.

From this set of findings, we’ve generally recommended that companies that have strong SEO programs (or just good organic listings by dint of brand/site) weigh the effects of organic cannibalization when making PPC buys. That doesn’t mean not buying words where cannibalization might (or does) occur. It means that PPC buyers should measure if cannibalization exists and weight the cost of these words appropriately — so that when they optimize their PPC buy, these words aren’t over-bought in comparison to other keyword opportunities.

This made it especially interesting to hear about the take on this same data produced by the major search engines for some of their significant clients — which were strikingly similar to each other and bore a simple and unsurprising theme of “spend more.” What is this “Yooglesoft” view? Well, let’s start by saying that the Yooglesoft view is based on data that might easily (but I’m assuming didn’t) come from one of our studies. However, the data they talk about typically illustrates only one part of the equation — the increase in incremental lift. The Yooglesoft view essentially boils down to this: with PPC Ads in addition to organic listings you get incremental lift.

True. Undeniably true. Uninteresting. But true.

But even though you should have every reason to be wary of Yooglesoft’s best practices — and while they often don’t make a particularly compelling argument — there is more to be said on their behalf.

Some practitioners will argue that the type of company most effected by this (big players with significant organic presence) simply can’t afford to give up clicks to the competition. On this view, the simple fact of incremental lift would be sufficient to justify ignoring organic cannibalization. This argument doesn’t carry much weight. If it were true, then these companies would be obliged to buy nearly every search term in existence — since they all must provide at least some incremental lift. This view would, no doubt, be congenial to Yooglesoft — but even they wouldn’t really consider it a best-practice!

So the argument should probably be re-stated as this: the words that a company owns high organic position on are generally the most relevant to them (especially our branded terms). And for the big company’s with strong organic presence that we’ve been talking about, perhaps they shouldn’t be willing to sacrifice any of these heavily qualified clicks to a competitor.

I think there’s some sense in this view — and depending on the short-term goals for a company in terms of market-share it might even underscore a viable strategy.

But even this proposition, however, seems mostly doubtful — as any argument which ignores cost is bound to be. There are a myriad of ways to spend marketing dollars — and as the cost per click for any specific term goes up — so too do the number of alternative methods of spending that money which might make more sense. For any organization with a fixed marketing budget over a specific term (which is, I take it, every marketing organization in the world), knowing the actual cost of an acquisition by channel is essential to constructing an optimal balance. And the gist of our claim is that you can’t assign a true cost to the PPC program for these words unless you measure the organic cannibalization.

In the end, the argument that you should measure as well as spend is every bit as self-serving as the Yooglesoft view — coming as it does from measurement practitioners. But not all self-serving views are created equal — as thinking about PPC and organic cannibalization from both perspectives will surely lead you to realize!