Marketers have relied on research from the time that marketing truly began. Whether that research had to do with finding out what customers wanted to hear or how other campaigns actually worked, data has been a marketer’s secret weapon. If you could show research behind your methodologies, your knowledge, not to mention your credibility, seemed to increase in strength. Unfortunately, we’ve come across a few examples of market research lately that seemed heavily biased, inaccurate, or that simply did not make sense. It makes us wonder if we are seeing the beginning of a disturbing trend.
The first questionable study surfaced back in February when highly respected SEO and inbound marketing company HubSpot published a report indicating that blogging out-performed television ads during the Super Bowl. I had a lot to say about that report, and to his credit, Mike Volpe of HubSpot responded in the comments section.
Flash forward to just a couple of weeks ago. An email, sent by Penton Marketing began with the following note: “B2B buyers are 70% through the buying process before contacting the vendor.” That seemed pretty interesting given all of the talk about how social media is enticing people to interact with brands. However, an effort to click to download a report simply led to a page that said “We’ll contact you.” Further research led to a May 2012 report on lead nurturing, where Penton cited the same statistic from SiriusDecisions, a marketing research firm. The statistic was not linked in any way to the Sirius site or to a report, and after nearly a hour of searching, we have not been able to find the Sirius report that actually cites this statistic as being fact.
Over this past weekend, another disconcerting event was added to this laundry list. In skimming through my Twitter feed, I saw a promoted tweet (one that a company pays to have appear prominently) mentioning a new study on Social Media and brands. For the first time ever, I clicked a link from a promoted tweet and I even handed over my phone number and email address so that I could download the report. Why was I willing to hand over that information? Well, the report was authored by Forrester Research, so I felt the report would be well worth my while in addition to the minor inconvenience of being harassed by sales people.
Sadly, and surprisingly, the report is rife with disconnects and flaws. In order to highlight how market research seems more misleading these days, let’s analyze the report carefully.
A Shaky Foundation
Problem one, the title of the research is misleading based on who was surveyed to complete the report. The report is called, “How Social Media is Changing Brand Building.” This would imply that social media definitely *is* changing brand building. It’s true that according to the study, 93% of marketers polled said that they need to reinvent how they approach brand building because of social media and mobile. However, let’s take a look at the businesses surveyed for this report.
33 Across – Here’s what their “about” page says, in part: “Reaching over a billion users, 33Across processes tens of thousands anonymous social engagement, influence, and interest actions that surround marketer and publisher brands each second. The company has offices in 11 cities including New York, San Francisco, Sunnyvale, Salt Lake City, Chicago, Detroit, and Boston.” Obviously, this is not a small company and it’s already immersed in social media as part of its business.
360i – From their “about” page – “360i is an award-winning digital agency specializing in search engine marketing, social media, mobile marketing and web design and development.” OK, so this is an agency that specializes in social media. Not to criticize our fellow agency people, but it’s possible they might have brought a biased perspective to these questions.
Digitas – Another huge agency that in part specializes in social media marketing.
General Motors – Primarily a consumer-oriented company, and another extremely large organization.
imc2 – Now called “Me Plus You,” this is yet another digital agency.
JetBlue – Another huge consumer-oriented company.
McDonalds – We certainly know who they are.
Radian6 – Radian6 is one of the top sellers of social media monitoring tools. Again, they might have brought a bit of a bias to this survey.
Rockfish Interactive – Another digital agency
Tenthwave Digital – Agency
Vivaki – Still another agency, this one combining personnel from Digitas and Leo Burnett (a very well-respected agency).
Weber Shandwick – A PR firm
These companies are the only ones listed as having been interviewed for the report. Given that, is it surprising that 93% of people polled said social media was changing everything? To us, it seems a bit self-serving.
What’s even more difficult to understand is that several of the findings indicate that B2B companies were surveyed. While the results often differ from the provided B2C results, who out of these companies represents a B2B company that is not an agency? Were the digital agencies listed above counted as B2B? That makes the report seem even less credible given the mentioned agencies’ proclivity for social media.
Fuzzy Math
Obviously, questioning the foundation for a study is going to cast a shadow on the study as a whole, but let’s look at one more facet of this report that raised our eyebrows.
According to Forrester, out of 57,924 online adults, 59% visit a social networking site on a daily or weekly basis. Of those, 33% have become a fan of a brand or company over the last three months.
Our initial reaction? Who cares about these numbers? Facebook just marked one billion users worldwide. Is it surprising that a little over half of adults polled said they visit a social networking site pretty regularly? Not really. Moreover, the small percentage of people who have clicked “like” on a company or brand page would seem to argue against the idea that brands are seeing a windfall of engagement on Facebook. In fact, Advertising Age recently published an article titled, “No, Brands Aren’t People — and Consumers Don’t Want Them to Be” (only abstract is now available) noting, “we found that only 23% of consumers have brand relationships – and they are already fans of the brand in question. The rest aren’t interested in a relationship, regardless of whether they like a brand or not.”
Taking it a step further, we were able to find a SiriusDecisions study on how the C-Suite prefers to validate vendors. According to this study, 31% of CEOs polled said they validated a vendor based on a previous experience with the company. CIOs and HR personnel also indicated that previous experience with a company was the most important factor in choosing a vendor. Social Media has nothing to do with that unless it is to help keep your company top-of-mind after an interaction with your customer.
Assuming those other cited studies are correct, the fact that half of adults polled visit an online network is virtually useless to most brands. If people are not even liking your page, you are not reaching them. Your brand has remained as unknown to those people as it was before you joined the world of social media.
Why the avalanche of poor research?
The real question is why so many studies are published when they are clearly shaky at best. In the case of the Forrester report, one almost wonders if the companies interviewed also paid for the study, which would explain the bias. Passing such a report off as objective market research seems to us like a major hit to the credibility of market research itself.
But that’s just how we see it. What do you think?
Image Credit: http://www.flickr.com/photos/thomashawk/3186635150/ via Creative Commons
Marketing Wisdom for the 21st Century 
So yeah, heard this on Reddit earlier this week. It was from an business major giving advice to a art/art history major. I am paraphrasing but:
“Drop the art and stick with art history. Either work as an appraiser for an auction house or specialize in a niche market where you can still find masterpieces for $5,000. Use your expertise and connections to acquire or bid up as many masterpieces as you can. Blog, write, and otherwise contribute to the niche’s popularity. Grow it.”
“Then, after you’ve inflated the price high enough to make a profit, sell.”
That’s what these guys are doing. I heard a wonderful quote from Ford’s Scott Monty “Hubris often breeds in a tin ear.”
That’s what’s happening here. Is SM important? Yes. Across the board you can find a way to leverage SM sites, tools, and concepts to better market for your clients.
But the people who got in early are now inflating the value of what they do. Like the charlatans that bought 1000s and 1000s of followers early on and used that inflated follower count to weasel their way into keynote speaking gigs.
More of the same.
And yes, I’ve always been this jaded.
You raise good points, Vincent. I guess the charlatan factor doesn’t bother me as much as the fact that these studies present themselves as being objective when they clearly have some motivation behind them. In fact, this last study was done for Wildfire, another social media immersed company. The title and goals for the study are just simply misleading, at least the way I read it all.
Marjorie – your analysis is superb and spot on. I have long been an advocate of companies using 3rd party qualitative research – findings and insghts communicated through buyer personas – to validate or complement quantitative market research. Companies today need to get a full 360 degree view but it cannot come from quantitative market research alone – and some of it can be very self-serving and faulty as you point out. Great post.
Tony
Thanks Tony. You’re exactly right – understanding the big picture, not just within your own walls but throughout your industry, is 100% essential. Unfortunately, a lot of people are looking for shortcuts these days when time is short, and I fear they will look to these studies and not be alerted to the motives behind said reports. That could cause a lot of misinformation and misperceptions to spread like, well, like wildfire.
Margie,
It seems obvious that the studies were conducted & presented only as a sales tool. Much the same as consultants are reluctant to put forth hard truths to a paying client it appears pollsters are in the business of “selling” their services and obviously taking direction from the client otherwise how could a reputable polling company choose those businesses as candidates for the polling data? I believe their strategy was 2 prong, keep your paying client happy and they chose them as perspective new clients for their data. Social Media fuzzy math is used as scare tactics to intimidate management that are largely out of touch with the social media world but still control marketing budgets.
You know, if the study had been called a white paper, I think that would have been more appropriate. You expect a white paper to offer some good information, but you know the information is there to support what the company is selling. When you call it a report or a survey, however, and especially when a company like Forrester is behind it, that sets a different family of expectations. Muddling those worlds can have a lot of negative repercussions.
Aside from blatant marketing material posing as market research, I think the media world is suffering from a bad case of confirmation bias. The same reduction in the barriers for entry to “create” have given us a challenge to vet and curate.
With all the exuberance surrounding digital and social media, we need to remember: wanting something to be true doesn’t make it true. 🙂
I think you hit the nail on the head there, Alan. I’d love to know the inner workings of how that study for Wildfire came about. “We want to prove that social media is important to people who are in charge of branding.” “Oh, ok, let’s talk to digital agencies that work with social media! I would rather have seen a much broader range of survey participants, including people from companies a bit smaller and a bit less consumer-oriented than McDonalds. Call me crazy 🙂
Manipulative, self-serving, and inaccurate. And this from Forrester? Prompts me to rethink the impartiality of their research from here on out. Thank you for posting this as a warning to dig into who is getting researched before trusting the outcome.
Thanks Miller!
It really is quite sad how these companies provide information to consumers as fact when they clearly are not. However it’s something that happens across the spectrum these days, from politics, to advertising products, and more, and (I hate to generalize but,) most consumers are quick to believe it; questioning it is definitely not their first instinct. I think companies are more willing to conduct surveys and studies in this way because it’s easier then conducting a proper study that may not turn out the results you hoped for. But in the case of social media, I can’t really see why they would want to do a botch study in the first place, when it seems to me they probably would have gotten the result they wanted. Great post! – Hannah, BizBrag Team
Hi Hannah,
That’s *exactly* what concerns me. People are looking for quick fixes as provided by other people. So, you get a study, you see it’s sponsored by Wildfire (reputable) and that the research was done by Forrester (reputable) – you may not take the time to say, “Hmm, who was surveyed for this study?” You’ll say, “Oh, OK, everyone is saying social media is important for brand building.” And you’d be ok to an extent, but that gets to be a dangerous path to travel if you don’t do your due diligence.
Forrester is not a market research company, contrary to popular belief. And HubSpot is REALLY not a research company. One is in the paid analyst racket; the other, content marketing. I hope that these examples all reflect poorly on the discernment and standards of the companies who choose to call these studies Market Research, and not on the honorable standards of my profession and the hundreds of MR agencies who adhere to and exceed the standards of the Council for American Survey Research Organizations. But you are right that this proliferation of thinly-veiled marketing pieces does, at the very least, commoditize research.
You’re right Tom – I should have worded the part about HubSpot more carefully. They presented a case study that, while it was transparent it was in part promotional, was extraordinarily shady. But they did not present it as objective market research, so I should have differentiated that.
I think you hit the nail on the head with your comment – the information in the Wildfire report would have been entirely suitable for say, a white paper. You expect those to bring a bias because the ultimate purpose is for them to create leads. But if you present something as a “report” that leads me to think it’s objective, and I get cranky when I find out it’s not.
Market researchers provide data – I think the only thing your profession needs to monitor is how your data is being used as a tool. If a company takes information you have provided and uses it say something the data doesn’t actually support, I think that *could* sully the researchers themselves, unfortunately.
A correction on the methodology for the Forrester report referenced. There are two forms of methodology noted in the methodology section of the referenced report. 1) The data cited was from “an online survey fielded in February 2012 of 99 marketing leaders. The sample draws from a curated list of self-described marketing leaders. Respondents who participate in online surveys have in general more experience with the Internet and feel more comfortable transacting online. The survey is global but draws primarily from the US. Incentive for completion was a deck of the survey results. Exact sample sizes are provided in this report on a question-by-question basis. Panels are not guaranteed to be representative of the population. Unless otherwise noted, statistical data is intended to be used for descriptive and not inferential purposes.” 2) In addition to the quantitative survey, Forrester conducted qualitative interviews with leading experts in the field being researched, such as ad agencies, research companies and marketers. These interviews provide a range of opinions and experiences which help validate and inform Forrester’s point of view on specific topics.
Very engrossing article. I felt your blog is highly intriguing. Kudos again – I will come back.
Marketing Research Companies
Credibility of market research for me is not decreasing and i can say it is still in its own level,in Finland country market research is still an advantage to some people which many of services still doing the best to make their customer satisfied which lead to a market research a good image.
Türkiye’nin en etkin inbox garantili Toplu Mail Server ve toplu mail gönderim hizmetleri. Online olarak yönetilebilir ve raporlarınızı mail gönderim esnasında bile alabilirsiniz.