Context Matters More Than Anything

Ok, I seem to have kicked up a bit of a storm again with my post “The Importance of Critical Thinking When Reading Industry Posts“. That post arose from my frustration at seeing data presented from numerous sources as being representative or even universal in its applicability to paid search accounts. In my post, I talked about how averages can be really dangerous – I stand by that statement.

Larry Kim’s post entitled “Why Averages Still Matter In PPC” takes me a bit to task on that thought. He argues that averages really are ok to use and I don’t totally disagree with that notion. Here is where I do differentiate though – averages presented without data context are dangerous. For an average to be truly representative of a data set, the data must adhere pretty strictly to a bell curve in its distribution. Larry’s post talks about this as well, on this we totally agree. But most of the posts I see that talk about averages or benchmarks rarely provide what I would consider very, very necessary context for the data that went into the calculation of the reported averages. Are we to just assume that all averages being presented are based on a bell curve shaped data distribution? If that the data sample used to calculate the average meets this criteria, just say so and it instantly adds more validity to the average being discussed.

By not disclosing more details about the data set used in these types of calculations, readers are left to either blindly accept that anyone who is presenting averages has done it in such a way that their data actually represents a true average figure or question and/or ignore it all. I appreciate that as data sets get larger they *tend* to normalize – that is math speak for basically making outliers present in the data much less significant. But, again, a large data set in and of itself does not guarantee this. More to the point, if performance information is being shared with the idea that a person reading the post might want to apply or think about the data relative to their situation, additional context for the data is essential.

For example, even if a dataset is large, if it is made up of mostly small spend, local advertisers and the accounts I manage are mid-sized spend and regional, that matters. It really is as simple as wanting to know if what I am reading about is an apple to my account profiles’ apples. I don’t this this is a revolutionary concept – it seems like something pretty foundational.

Part of the problem stems from the fact that there is a real hunger for benchmarks and absolute answers when it comes to any kind of marketing, but particularly for PPC. And we as practitioners famously respond regularly with “it depends” as our default answer to questions like “what is a good clickthrough rate?” or “how much should each click cost?”. And while it may grate to not have specific answers at-the-ready for these types of questions, there is a good reason why we generally don’t. Because accounts are like snowflakes – no two are exactly alike. And for this reason, applying anything universally from one account to another, from one industry to another from one geography to another is fraught with peril. What performs well in one scenario might completely tank in another that, on its face, seems similar.

If you have worked with clients who target different geographic regions, you may have already seen this in microcosm – I know I have. Campaigns for the same product or service can perform VERY DIFFERENTLY FOR THE SAME CLIENT just in different geographic areas. Knowing this to be true, averages used as benchmarks make even less sense to me. I take these types of benchmarks, when not provided with what I have described here as adequate data context, much in the way I take data from search research tools – as perhaps a relative number to keep in the back of my mind, but not something I’d dare hang my or my clients’ hat on in any kind of absolute terms.

Bottom line, as I said in my original post too, is to not turn your brain off when you read anything. Be sure of course that the source is credible, but beyond that try to really understand what is being presented, how the conclusions were reached and if they are taken from a data set that includes entities and accounts similar to those you are managing. And if you’re presenting this kind of information either in posts or during presentations be mindful of providing detail and context. It’s like your middle school math teacher always told you – show your work! PPC nerds want to see it.

As always, share your thoughts with me either in the comments or on Twitter (@NeptuneMoon).

Comments

  1. You did it again! Great post!

  2. I every time spent my half an hour to read this weblog’s articles all the time along with a mug of coffee.

Speak Your Mind

*