How to do a simple A/B Test

In digital marketing we often want to test to see what works best. What is most attractive to a potential customer? What message resonates best? What advertising campaign is most likely to get attention? How can we maximize the completion rate on a website landing page?

These kinds of questions often involve a simple test of two competing ideas or approaches. For example, we might have two banner advertisements or two registration pages, and we want to know which results in the most click-throughs or which is most likely to result in more leads. This is called an A/B test. Here’s how to do one right and not have to worry about the stats too much.

What is an A/B Test?
In marketing, an A/B test is a simple way to understand if there’s a meaningful difference between two competing approaches to a problem, such as a banner advertisement or a direct mail campaign. Using the test, we can measure the results of the two approaches and decide if there’s a real (statistically significant) difference.

A/B testing isn’t a new idea. I can remember decades ago talking to an executive at AMEX where they performed extremely elaborate versions of A/B testing on their direct mail campaigns supporting new credit card applications. They were looking for tiny differences in response rates across millions of mailings, and tested everything down to the color of ink. Usually we’re not nearly as sophisticated but the same principles apply.

How do I set up the Test?
Setting up the test isn’t that hard. Normally, you’ll have two approaches to a problem and a desired outcome. Here’s a few examples:

• Your agency comes back with a new creative idea for a banner ad, and you'd like to test it against an existing design. You want to see if the new creative idea gets more click throughs over the other.
• You have a new layouts for a web page. Which keeps people on the page the longest, the new idea or the old one you already have in place?
• You have an email newsletter. You want to test a different subject line to see which gets the best open rate
• You’ve a new direct mail campaign, but can’t agree whether to feature the discount offer again on the envelope, or picture of the new product. Which gets the best response to your order hotline?

In each case, you have a couple of alternative approaches to a problem, and an obvious way of measuring (counting) which is best.

What else should I know to make sure the test will work?

Here’s a few things to be careful about:

•  Ideally, make sure you’re actually measuring the right thing. For example, if you have two banner ads, and each features a different offer, different graphics and is placed on different websites – well, it’s really hard to know what is affecting click-throughs. Is it the websites that make a difference, or the offer or the graphic? The test will not tell you the answer. As much as possible, try and keep everything constant across each test except the specific thing you’re testing.  
• Try and randomize as much as possible. What does this mean? Here’s some examples:
•  If you’re testing two direct mail pieces across a sample of your database (say, 500 people), then make sure to randomly select people from the list and randomly place them in each group. This way, the selection process can’t influence the outcome.
•  If you’re running two different banner ads on a website to see which performs best, try and make sure they’re seen randomly, or as near as you can get.
• Try and make sure that you’re testing across a big enough sample. Bigger is usually better. Statistically, the power of the A/B test is related to the amount of data you gather, and hence the sample size. And remember, your data is often measured in terms of response rates, which in marketing can be very low. There’s no magic number, but try and get large.

How do I perform the test, and do I need to understand statistics?

Most people who do A/B testing never perform an actual test – they “eyeball” the data instead. As we’ll see, this can be a big mistake. It’s absolutely worth doing the tests because it’s easy to come to a wrong conclusion. The test involves statistics, but luckily for us there are online tools that do all the math for you. Here’s two examples:

Example One: Banner Advertising

You run two banner ads on the same website page that appear randomly to viewers (Banner A is old, and you think Banner B might do better). You use 5,000 impressions, 2,500 of each banner, and here’s the results:

	Impressions	Click Thrus
Banner A	2,500	25 (1%)
Banner B	2,500	32 (1.28%)

Eyeball this data, and most people would conclude that Banner B is easily better than Banner A. So let’s run a test to see if this is statistically the case – I’m going to use a simple tool supplied by the good people at Optimizer. (The tool is designed for a specific example for a website registration page, but we can easily adapt it for our use.)

To use the tool, let Control = Banner A and Variation = Banner B (remember, in this example you’ve a hunch that B is better than A). The “Number of Visitors” can equal the number of impressions, which is 2,500 in each case. The number of Conversions is the number of click-thrus for each banner (25 and 32 in our example).

Ignore the “p-value” unless you’re a stats geek. Hit Calculate and the tool will tell you if the difference in click throughs is really statistically significant. And the answer is – NO!

How can this be??? Surely, if there’s a difference in the results, it must mean our Banner B is better than Banner A!

Well, imagine you’re flipping a coin, and you flip it 100 times – assuming the coin is fair, you should get 50 heads and 50 tails, but you wouldn’t be surprised if you got 48 heads and 52 tails. Or even 45 heads and 55 tails. The same random chance with coin flips can influence our simple A/B test, so even though there’s a small difference in results, we can’t be certain this isn’t just random. You could be making a mistake going with Banner B. This is why it is so important to actually do a test – especially in situations where “responses” are very low, as is usual in most marketing activities.

Example Two: Website registration

You now run a test of two landing pages where visitors are asked to register. Page A is the old page you’ve used for a long time, and Page B is “variation” you want to test. You want to see which gets the most registrations, and here’s the data:

	Impressions/Visitors	Registrations
Landing Page A	13,000	83 (0.64%)
Landing Page B	8,900	78 (0.87%)

Plug in the numbers and… the answer is YES! But what does this mean?

It means, Page B performs better than Page A, within the margin of error of the statistical test. Let’s explore this.

In this case, we should note that the number of visitors on each page is very different (which is not to say that they aren’t random across each page, we assume they are. It’s pretty common to have different numbers given the practical realities of market research). Page A has 13,000 visitors, compared to only 8,900 on Page B. Page B has a better registration rate of 0.87% compared to 0.64% for Page A (the absolute numbers of registrations are very close, but are misleading). Eyeballing the data could be confusing, but the test makes it clear that Page B is the better bet.

Happy testing.

What’s at the core of Apple’s social media strategy?

Get into a discussion on marketing strategy, or attend any meeting with “brand” in the title, and it’s usually only a few nanoseconds before somebody invokes Apple and rattles on about how we need to be more like them and do exactly what they do. You can work for any kind of company, from a biotech start-up to a manufacturer of concrete, and this rule applies: Apple is the gold standard on how to do marketing right and create a winning brand that has a gravitational attraction for customers and a magic ability to mint cash.

Yet, when it comes to social media, Apple is doing exactly the opposite of everyone else.  Lemming like, most companies have leapt head-first into the social world, and most experts agree that having a solid social strategy is critical for building a successful brand. Yet Apple is largely absent from social. So, how come the most successful brand on the planet has been such a contrarian when it comes to social media, and been so successful doing it?

‘Reputation management‘ is often the reason given for an active social program, especially for established brands like Apple, the logic being that reputations are made or lost amidst the babble of customers, prospects and competitors. But the linkage between reputation, brand, and company performance is fluid, with one not clearly being dependent on another. Indeed, there’s plenty of crappy companies that have award-winning social programs  - Kodak comes to mind, or Dell.

Apple’s attitude to social is more likely bound-up with their fetish for control. They control the message and they control the user experience. They close others out and control the supply chain. They operate on a principle that information scarcity will drive anticipation and demand. None of this would argue in favor of an aggressive social program, where control is relinquished and being closed just doesn’t work. Nevertheless, it’s hard to argue against the success of Apple’s approach, at least under Jobs’ tenure.

This approach wouldn’t work for most companies.  At a practical level, most of us don’t have the discipline to make it work. But more importantly, ignoring social won’t work because, sadly,  none of us actually are Apple.  Until recently, Apple was the most valuable company in the world. They command attention, people actively listen to the slightest rumor from them, the media fawn. You and I need to fight for these things. It’s the difference between playing offense and defense.

But there’s a bigger lesson to be taken from this.

Apple is good at many, many things, among them having a very clearly articulated business strategy that permeates everything they do. Their approach to social is an articulation of this strategy – it just so happens that this means they do very little social at all. Too often, social is seen as an end in itself, and having a social media strategy as something distinct from broader business objectives. At its core, that’s a recipe for failure.

The Crowdsourcing Myth and the White House

A while back the Obama administration, in a wild fit of optimism, created We the People, an online forum where any citizen can petition the government on a question, and if sufficient people sign the petition they’ll get a response. Great idea, right? Government needs to be more open and responsive, and this is open to all. It’s a form of crowdsourcing, where citizens with common cause can directly engage with elected officials.

Turns out, the idea is great in theory and absurd in practice, much like crowdsourcing itself. Within weeks of launching We the People petitioners from 50 states were asking to secede from the union, and a petition to build a Star Wars’ Death Star attracted over 34,000 signatures (but was regrettably turned down). The Brit CNN talk show host Piers Morgan, a fierce gun control advocate, was quickly the subject of a petition to have him deported. In response to all this nonsense the White House raised the threshold for getting a response from 25,000 to 100,000 signatures, a move only likely to encourage the crackpots and fringe dwellers even further. The UK governments’ e-petitions site has a 100,000 signature threshold, and still gets petitions attempting to save chocolate bars.

Opening the opinion floodgates is admirable, but we forget that those of us with moderate views are labeled the “silent majority” for a good reason. It’s not that we’re lazy, though that could be true, but that the frenzied extremists are the people who are always going to exert themselves the most. If you doubt me, take a look at the comments on any online coverage about the Obama administrations recent efforts to enact gun control legislation – the ferocity of opinion is itself a great argument for keeping kitchen utensils out of the hands of most Americans, never mind semi-automatic firearms.

Crowdsourcing is a dangerous myth. The wisdom of the crowd is far too easily drowned out by the madness of the masses.

US Election Costs $50 Per Vote –Money Well Spent?

The presidential election of 2012 may not be remembered for who won or lost, or the issues, or the storm that wrecked the east coast: There’s a good chance it will be remembered by how much it cost.

By some estimates, over $6 billion (with a ‘B’) has been spent collectively on the various election races nationwide, with something close to a billion being spent on the presidential election alone. With at least 120 million Americans expected to go to the ballot, that’s a staggering $50 per vote. This compares with under $20 per vote in 2000.

This is unquestionably great news if you’re a TV station owner, especially if you’re Ohioan, but of dubious merit for just about everyone else and for democracy at large. But let’s ignore the Big Question about the wisdom of mixing all that money in a democratic election process. I have a more fundamental question: Is this money well spent? Said another way, can you buy an election?

There’s no question that advertising is persuasive to some degree, but can it influence voting? Not so much. Stephen Dunbar at Freaknomics  finds that campaigns managers and insiders from all political parties don’t see a nice linear relationship between spending and outcome, but they recognize it contributes. While researchers acknowledge that the biggest spender is also most often the winner, when all other factors are taken into account, the spending isn’t the root cause of success. To cite recent examples, Meg Whitman in California or McMahon in Connecticut each spent a considerable part of their private fortunes on loosing campaigns. So, the good news is you apparently can’t buy an election.

But wait. There has to be a reason all this money is sloshing around in politics – why are we all subjected to so many crappy TV ads unless they do something?

Advertising does a couple of things very well. First, and most obviously, advertising raises awareness. In this regard, political candidates are no different to dog food or Viagra, at least from an advertising perspective (there may be other similarities which I won’t pursue). This is why so many ‘third party’ candidates rail against the political system – they aren’t able to compete for awareness is a noisy, cluttered political environment. (Indeed, many Americans are surprised and confused to find that there’s actually four candidates on the ballot for president this year. Most voters have never heard of two of them).

The second effect of advertising is to set the agenda. The famous adage about the news business –  news organizations can’t tell us what to think, but they can tell us what to think about – is also true of advertising. This election cycle, the Democrats did a masterful job of framing the political debate, so that the focus became less about the economy and more about social, environmental and health issues. This didn't directly persuade voters to vote for any specific candidate, but it did provide evaluation criteria that favors one over the other.

Advertising can have others effects, especially for so-called ‘low involvement’ buying decisions (think breakfast cereal or washing detergent), but for most situations it’s influence is weak, indirect and fleeting.

As goes political advertising, so goes advertising everywhere.  Those we wish to influence in business may not be heavily swayed by our own advertising and self-promotion. We can make them aware of our company, we can raise issues we think are significant and warrant attention – but we will need to rely on other tools to really influence decision making.

The Value of Social Media

Ever since the Facebook IPO debacle last May there’s been a lot of fretting about the real value of social media platforms. Facebook’s stock opened at $38 a share, and today trades at about $21, a loss of 45% or about $25 billion dollars, or the equivalent of the GDP of a small country – say Cyprus or Panama.

Of course not all social platforms have disappointed so magnificently. LinkedIn, the social professional networking platform, IPO’d in May 2011, and first day investors have made a healthy 50% return. There are other success stories.

The valuation of social platforms isn’t just about investor returns. We need to consider the value proposition to users and to those that will provide income to these platforms – potential sponsors. Most social platforms have at their core an advertising model for making money, and advertisers have also begun to express their value judgments. Responses have been mixed. Back in May this year, shortly after Facebook’s roller-coaster IPO, GM announced that it was withdrawing advertising support, citing poor performance. Many other companies have taken a measured approach to all online advertising, although according to Forrester, advertising revenues for social platforms are expected to rise by 34% through 2014. To some extent, advertising on social is a victim of its own success: clutter and saturation make any advertising or sponsorship difficult. And many potential sponsors see an opportunity to disintermediate all media, and reach out directly to potential customers with their own tailored content. Then there’s the Transparent ROI Problem – online is so amenable to measurement, smart companies are able to precisely gage the hard returns on investments (or lack thereof), and increasingly are discounting softer brand benefits.

One thing is certain – the value proposition to users of social is beyond compelling. For most teens, Facebook is as necessary to life as oxygen. For most everybody else, social is an ingrained and everyday part of life, like the morning cup of coffee or, dare I say it, like the daily newspaper used to be. It is this collective addiction to social media that is fuelling a social media investment bubble, but until we find a clearer conjunction of shared value between users, sponsors and the platforms, the real potential won’t be realized.

To some extent, LinkedIn shows the way and for sponsors and users, the meeting point for shared value is bound up in the idea of community. This is where we find the common values of like-minded users of a product, or passionate followers of a brand. In LinkedIn's case, they attract professionals who want a mediated way to network with colleagues and companies, and businesses want a way to find talent. Everyone sees value. Today, too many social platforms are hoping to exploit business value by mining users’ personal information, marketing this data to companies. Trouble is, users don’t see the shared value – they see exploitation, and that’s not good news for social media companies.

The Emperor Wants Clothes

We used to have an assumption that is the digital world we remained cloaked – almost anonymous – unless we choose to reveal ourselves, and even then we were able to invent a self of our choosing. Not anymore.

As I’ve mentioned in a previous post, everyone now knows that as we rummage around in the virtual world, our digital trail is soon followed: What we do, where we go, who we are, and what we think can all be discovered and refactored with unnerving ease. Our online selves are laid bare.

Us marketing types are very happy with this situation. We like naked consumers who cavort online as if well-dressed, because this unrequited intimacy allows us to target them very effectively. Knowledge is always powerful, and in marketing circles the manifold details we can gather online make us giddy with excitement about how we can tailor loving entreaties the better to woo prospects.

The whole thing is a parody of the old children’s story of the Emperor’s Clothes, expect that in this version the Emperor is starting to demand we give him his old wardrobe back.

Punters know they’re being digitally stripped searched and they’re not thrilled.  They want some protection, some dignity, some rights. Enter Do Not Track. As early as 2007 the US Federal Trade Commission was approached about creating a “do not track” list, similar to the “do not call” lists that exists to suppress telephone solicitations. Over time, a more practical, technical solution has become favored: The implementation of a HTML header field that automatically signals a user’s willingness to allow tracking. This is a small step in the inevitable direction of giving consumers more rights to privacy, but it may not be enough to placate regulatory bodies like the FTC or EU regulators.

So, what do we marketing types do when the Emperor has robes again? More permission-based marketing is one outcome. A reliance on cultivating trust and a relationship with prospects is another. Certainly, the crutch of unfettered access to consumer information may soon be gone.

The Gladwell Effect

A virulent meme of the moment is 10,000 Hours – the idea, popularized by Malcolm Gladwell in his book Outliers, that brute force persistency is the route to greatness.

 In his best-selling book, Gladwell examines the factors that contribute to high levels of success and acclaim. Looking at everyone from Robert Oppenheimer to Bill Gates, he concludes that success is largely predicated on having the stamina and determination to work at a task for a total of around 10,000 hours – what he calls the 10,000-Hour Rule.

Really?

One of his examples is The Beatles, who Gladwell points out spent much of their early days in a daze, in Hamburg, playing and playing and playing. Having read about their exploits in Germany, it’s actually amazing they even survived the experience, never mind rose to acclaim. But to say that the critical ingredient that led to their fame was being on stage together for months on end is, at best, misleading. There’s just so much more to account for.

For one thing, The Beatles adroitly (or fortuitously) surrounded themselves with great talent. To take an example, their producer on most of their recordings, George Martin, was enormously influential on their musical development, making the studio an instrument in itself and pushing the band to explore more complex sounds. Then there’s Brian Epstein, who took their raw talent and turned it into a mop-haired product for worldwide consumption. Oh, and in case we forget, Lennon and McCartney wrote some pretty good tunes, a talent that transcends anything they might have picked-up at the Ratskeller.

Perseverance is undoubtedly a characteristic of greatness. So is its near-neighbor, obsession. I’d actually argue that the real driver here is passion, an ardor for what you do. But this is never enough and to argue otherwise is an oversimplification.

Gladwell can’t be entirely blamed – although some, like scientist Stephen Pinker, have claimed his whole argument is flawed. Gladwell’s idea is more nuanced, but the popular interpretation is a reduction to a direct cause-and-effect: If only we all tried harder, we’d be rock stars.

It’s a very human failing to try and account for all results by isolating a single variable. In marketing, we do this all the time – be it attempting to understand what led to a sale, why that video went viral, or what caused our competitor to beat us on a deal. It’s too easy to say it was all down to the salesman, or the clever script, or the fact we didn’t have that one specific feature in our product. Usual this reductive reasoning is all wrong.

In marketing, don’t expect simple answers. And don’t anticipate that repetition and persistence alone will drive success.

Can social media predict the future?

We all know reporters love to prognosticate, their opinions respecting no temporal boundaries. But now a whole other industry has grown up that lives off their stories, trying to predict future events based on today’s news coverage. Companies like RavenPack have built a business by sifting through news stories in near real-time, looking for the sentiment of coverage related to publicly-traded companies. They claim this information can be profitably used to predict fluctuations in stock prices.

Predicting stock price moves based on current media sentiment has logic to it – readers are also traders, so if we see a string of gloomy stories about IBM’s recent product announcement, then some of us will likely dump the stock. And our likelihood to take action is proportional to our trust in the news source: If we see negative reporting in the Wall Street Journal that outweighs positive opinions on iLoveIBM.com.

Now we have social media and the prediction game has got a good deal more complicated. Researchers at the USC Annenberg School have been working hard to see how social media traffic can be used to try and predict future events. They started by looking at new movies and tried to correlate Twitter messages to first weekend receipts. They managed to get very good at estimating this, often better than industry experts, and what they found has broad implications for social media marketing.

The Annenberg team discovered that the best predictor of box office outcomes wasn’t the volume of traffic related to a movie, but the net sentiment expressed. In other words, quality beats quantity in predicting outcomes. A new movie might get a lot of buzz, but that wasn’t predictive of making a lot of money. This confirms what companies like RavenPack have long known: it’s the tonality, not the totality of coverage that really matter.

The takeaway from this? We need to be very careful how we measure social media outcomes. Measuring retweets, mentions and the raw volume of coverage for your brand just isn’t enough. Yet this is what most people do today.

Annenberg, in co-operation with the LA Times, are now trying to see if they can use their system to predict the outcome of this week’s Oscars. If you believe them, we should expect a big upset: The winner for best movie will be Midnight in Paris. Here I’ll make a prediction of my own: Much as I liked it, Midnight will lose.

Remember, I said that being able to predict stock fluctuations worked because readers are also often traders; the same logic does not apply to Oscar voting. The Oscar outcomes are based on the opinions of a mere 5,700 members of the Academy. Winning others awards such as BAFTA are a much better indicator of Oscar success.

The lesson here is that listening to everyone’s opinions is often a mistake. Instead, we need to target on influential audiences – potential customers, shareholders, employees – and understand clearly how they impact our brand and business.

The social deluge and market saturation

Here’s a fun statistic: every 10 days, over a century’s worth of video footage gets uploaded to YouTube. Here’s another: there are over 27 billion likes and comments added to Facebook every day. Or try this: last week, the number of Superbowl tweets peaked at over 12,000 per second.

If you’re a marketing type, then your first reaction to all this is probably salivation – all those eyeballs, all that attention!! – but dwell on this for a moment and you’ll quickly despair.  The astonishing growth in social media  –  the unbelievable volume and velocity of messages, news, and information – is quickly leading to saturation. As users of social, we’re all increasingly unable to deal with the cacophony and clutter.

There’s another, related effect. In his book Data Smog, David Shenk estimated that the average American consumer was exposed to about 50 commercial messages a day in the 1970s; by 1997, that number had grown to 3,000 messages a day. Today, some put the number of marketing messages as high as 5,000 per day, with most of the increase coming from online and social sources. Commensurate with this dramatic increase in message density is a dramatic decrease in advertising effectiveness. The effectiveness of commercial messages is inversely proportional to the number of messages received.

Getting attention in social media is getting harder and harder. The effectiveness of pushing any messages through social channels will only diminish with time. This isn’t solely an advertising problem.

There are several consequences to this. First, social platforms that rely wholly on advertising for revenue will slowly see growth-rates falter as smart, data-driven companies begin to see waning returns from their advertising investments. The irony here is that the runaway popularity of social platforms will be their undoing.

Second, marketing pros will be forced to rethink old ways of engaging with consumers. Heavy-handed corporate marketing in a social world won’t work. Getting heard amid the social din will require an authenticity and empathy that is alien to many old-school marketing pros. It will require careful targeting and impeccable timing.

Finally, marketing professionals need to educate their organizations on what can realistically be achieved with social media. Engaging through social marketing will require clarity of message and intention, as well as focus and agility – and resources.

Pottingers, Wikipedia, & WikiLies

Last week, The UK newspaper The Independent revealed that Bell Pottinger, a leading PR agency, had engaged in the covert manipulation of Wikipedia entries related to some of its clients. Using multiple, anonymous accounts, staff at the agency had eradicated negative information, inflated positive references, and altered the facts of numerous Wikipedia entries. It also became apparent that agency executives routinely pitched to clients and prospects their ability to alter Wikipedia entries as part of their services.

I propose that henceforth we name an entry in Wikipedia that has been willfully manipulated a "Pottinger" in their honor. I challenge readers to create the "Pottinger" Wikipedia entry.

Your immediate reaction to this story might reasonably be this – how stupid can you get??? My reaction was a little different: I’m a marketing professional with over 15 years’ experience running PR programs for big organizations, and I must confess to having altered Wikipedia entries too.

A few years back I started a new job at a well know technology company, and on my second day got a call from the CMO: Could I come to his office immediately. When I got there he showed me the Wikipedia entry for a senior executive. I was shocked. The entry had been altered by several anonymous people and contained openly slanderous statements. Some of the changes seemed downright bizarre. I had a Wikipedia account (I’d created a number of entries years ago), and I worked to get the changes removed and details corrected. It was an uphill struggle because I didn’t disguise who I was, but eventually things were made right (then right again, as the entry continued to be changed). We made no attempt to catch the perpetrators – that was too complex and time-consuming.

Of course, unlike Bell Pottinger, my actions didn’t breach any of Wikipedia’s guidelines (that I know about) or hide any truths – in fact, the reverse. And I’m not alone: I know of many instances where company employees or agency staff altered entries related to their employer or client, almost all correcting wrongs, adding missing information, or providing context. Wikipedia is a crowdsourced entity, open to anyone; laudably democratic, but ripe for abuse, neglect or simple error. Fixing things can feel unnecessarily arduous and often frustrating.

Needless to say, none of this excuses the stupidity of Bell Pottinger. They’re dolts, with a history of ethical issues.

However, you can’t argue BP is a unique case, or even out-of-the-ordinary. Far from it. There’s a long history of Wikipedia abuse. WikiScanner, Wikipedia Review and others have catalogued many examples over the years, and others have pointed to the inherent problem with a trust-based, crowdsourcing model for gathering information.  Indeed, Wikipedia deletes over a thousand entries every day.

In general, crowdsourcing anything should invite scrutiny and skepticism. We certainly shouldn’t assume that Wikipedia is the bastion of unalienable truth. And unfortunately, nor should we think of Bell Pottinger as an anomaly – expect continued revelations of WikiLies as Wikipedia, already the sixth most visited website worldwide, gains in significance.

Fortune 500 Lag in Social Media Adoption, To Their Great Cost

At the end of last month McKinsey published the results of their fifth annual survey on the ways organizations use social technologies.

For the most part, the results are a snooze: The adoption of social media tools, from Twitter to Quora, is steadily rising; measurable benefits are steadily if slowly increasing; and the sophistication in the way organizations use social media has seen significant gains. All good, although hardly ground-breaking news.

But wait. The McKinsey people aren’t anything if not thorough, so they also tried to show a correlation between adoption of social media and self-reported organizational performance (that is, market share gains, operating margin compared to competition, and being first in industry market share). To my surprise, on the latter measure – showing a link between market share leadership and adopting social tools – the correlations are mostly negative. This suggests that the adoption of social media is adversely associated with being a market leader, a counter-intuitive and strange result. If it were so, then the good people at McKinsey should be telling the titans of industry to flee Facebook, ban blogging, and terminate Twitter post-haste.

But wait. The researchers’ explanation of this is that “while market leaders may use social media technologies within the organization, they might be less inclined than market challengers to push for a full range of benefits [and use social media externally].” So, according to McKinsey, they suspect that market leaders as a group are actually under-utilizing social media.

This sounds plausible. Indeed, very detailed research by UMASS Dartmouth shows that for blogs, about 23 percent of the Fortune 500 (large, market-share leaders) have corporate blogs, compared to well over 50% if the Inc. 500, a list of the fastest-growing companies compiled by Inc. magazine. There are similar results for corporate adoption of Facebook and Twitter – the Fortune 500 lag the rest of industry, especially the fastest-growing companies.

Put these two pieces of research together and we have strong data suggesting that large, market-share leaders – think companies like WalMart, Exxon-Mobil, Proctor and Gamble, Hewlett Packard, Boeing and Dow Chemical – are collectively failing to extract value from the social media revolution. These findings play into the stereotype of the lumbering, bureaucratic multinational that often enjoy a market-share lead but fail to take first-mover advantage of new innovations.

And the research clearly shows that the price of late adoption of social media is very high indeed.

Latest Update: A Definitive List of Social Media Measurment and Monitoring Tools

Way back in April 2010 I compiled what I boldly claimed was a definitive list of social media measurement and monitoring tools. Since then, I've updated the list a couple of times and critically reevaluated my 'definitive' claim: Most likely, I've missed a lot of solutions and have certainly struggled to keep pace with the constant changes in the evolving social media measurement market.

My latest update saw numerous changes, reflecting increasing competitive pressures in a very cluttered marketplace. A half-dozen companies have disappeared completely, and several others have moved on to provide different solutions only tangentially related to social media metrics. There's also been a small number of acquisitions, most notably Salesforce.com buying the market-leader Radian6 in March this year. The net result is that the total number of solutions has fallen, although the quality of the survivors is very high.

If you're looking to buy a social media measurement solution there's a few important takeaways from all these changes:

• It's a Buyers Market
  Despite the predicted consolidation, there's still an abundance of companies supplying solutions. Companies are eager to get your business and you should be able to cut a deal for the right solution.
• Buyer Beware!
  Expect the consolidation to accelerate over the coming year. Be cautious about companies that aren't able to demonstrate good market traction, supply a full list of references, or just seem too desperate; they may well be gone in a year or two.
• Understand Your Requirements
  There's a broad spectrum of solutions out there, and every day new features are being added. Make sure you have a strong understanding of what you need to do and how a tool will help you do it. Take a look at the guides from many industry analysts (mine is here).
• Recognize that the Market is Still Maturing
  There is no perfect solution out there. Sourcing data, avoiding spam, providing adequate filtering, integration to marketing automation systems, providing easy-to-configure dashboards and reporting, alerting for your customer services folks... these are just a handful of important issues that are still being worked out.
• The Giants are Coming
  Finally, we 're still waiting for the obvious market giants... Google, Microsoft, maybe Facebook... to really enter this market. There's been ongoing rumors that Google was going to release a full-blown media monitoring solution; if they ever do, it could upturn the market and put an end to many of the companies I list. Stay tuned...

Any Ideas What Public Relations Is?

When you’re in the business of managing the image of others it’s a little embarrassing to acknowledge you’re suffering from an identity crisis.

This week, the good people at the Public Relations Society of America (PRSA) began an effort to better define what “public relations” is. This isn’t their first attempt: Two previous tries at a definition, in 2003 and 2007, ended in failure.

A perfectly reasonable question to ask is ‘why doesn't a working definition for public relations already exist?’ After all, the modern discipline of public relations was pioneered at the turn of the last century by Edward Bernays, Ivy Lee and others; the PRSA itself was formed in 1935.  Isn’t it fairly obvious what PR is about? And you don’t see physicists, lawyers, or dog trainers agonizing over what their chosen profession is all about, so why the debate with PR?

One answer is that PR and Corporate Communications are enduring monumental change. The economic collapse of conventional journalism has permanently altered the way news is created and shared. Opinions are formed and reputations altered through a labyrinth of social connections. Managing a public image has become more complicated, and the role of a PR pro less clear.

Another, less palatable reason is that most things in the world of marketing and communications are badly defined. If we were to take Voltaire at his word – “if you wish to converse with me, define your terms” – then a discussion with marketing pros would be very abbreviated indeed. As a profession, we bandy about overloaded terms like “brand”, “image”, and even “marketing” itself with only a fuzzy and shifting sense of what we mean.

So the PRSA, in an act of either abdication or inclusion, depending on your perspective, has asked for crowd-sourced inputs on what a definition should be. In my view, they’re asking the wrong question. We know full-well what PR is. The issue is how to make PR effective.

Bernays, the grandchild of Sigmund Freud, was very blunt in his assessment of what PR is about and its underlying intent, with his notion of “engineered consent” being rooted in ideas borrowed from propaganda. Ivy Lee was gentler:

"In brief, our plan is frankly, and openly, on behalf of business concerns and public institutions, to supply the press and public…prompt and accurate information concerning subjects which it is of value and interest to the public to know about." 

Modern PR hovers uneasily between the two truths offered by Bernays and Lee. Not much has changed at this level. The PRSA is thoroughly confused; we don’t need a new definition of what PR is, but rather we need to understand how to make PR more effective in a new communications landscape. The goals of PR are the same; the mechanisms for reaching those goals are changing and uncertain. The PRSA’s energies would be better spent on addressing these real challenges.

Marketing by the Numbers

One of the side-effects of our always online existence is that everything has become visible and measurable. As we all rummage around in the virtual world, we leave behind a trail of digital detritus that others can find, accumulate and sequence: What we do, where we go, who we are and what we think can all be discovered and refactored with unnerving ease. This has raised many alarms about privacy and security, but has also introduced opportunities for marketing professionals.

I’d argue that the new world of digital marketing is upending the whole marketing profession.

Marketing used to be a largely subjective, qualitative, artful enterprise. For sure, we could do research, conduct elaborate focus groups, and painstakingly gather data to inform decisions and discover the impact of our marketing activities, but all this was arduous and often ad-hoc. We’ve moved from an environment of information sparcity to information overload. Instead of ‘mining’ for data, we’re dealing with the avalanche.

Today, data-driven marketing is becoming the norm. Expectations of what marketing can achieve are changing. Most important, there’s a new level of expected accountability.

Some years back I got a Ph.D. and as a result accumulated more knowledge about statistics and research methods that I thought was healthy, or useful. Turns out, my old stats texts are the books I’m referring to most. I’ve been interviewing for a new job and a common ground for questioning is “how do you measure the effectiveness of what you do?” I’ve even seen job descriptions that single out the ability to conduct A/B and multivariate analysis of campaign data. Being able to read data – and to conduct marketing from a data-driven perspective – is a vital skill today.

Of course, data isn’t wisdom, as Wharton Professor George Day has pointed out. According to him, the amount of data a company faces is doubling every 18 months, while our ability to sift and assimilate the data is remaining pretty much static. Day and his colleagues advocate ‘adaptive marketing experimentation’, an approach to marketing that fosters data-driven decision-making by continually testing variations on different solutions – a fail-fast, discover-quickly methodology. Their views are informed by a recent IBM research report created from interviews with over 1,700 CMOs. The leading issues for these CMOs: data overload, social media, channel proliferation and shifting demographics.

Professor Day’s recent article is a great read. And after you’ve finished, see if you can find those old statistics textbooks in the attic.

Klout, Qwikster, and Mob Marketing

There’s been a lot of chatter recently about Klout, a tool that attempts to measure an individual’s online social influence. As with all scales that try to quantify individual prowess – think IQ to SAT – there’s a healthy debate about the basic validity of what Klout purports to do: After all, what exactly is “influence”?

I’d hazard a guess that whether you’re a fan or a foe of Klout has a lot to do with how well you score on their 100-point scale, though I may be being overly cynical. But whatever you’re opinion of Klout I think we can agree that finding some way of articulating relative influence is a big marketing problem we need to solve. We desperately need a way to sort the wheat from the chaff, because in our noisy online world there’s an awful lot of chaff.

Personally, I don't see Klout as a permanent fixture of the social media landscape. Klout thinks it is selling a solution when they really only have a feature: Most social media monitoring tools of any worth have in them a way of determining salience, aka influence. Most search engines will get there soon too. This is where this "feature" belongs, in a context that has some value.

But there’s another problem, neatly exemplified by the well-publicized and stock-shrinking antics at Netflix. To recap, after doing a great deal of research with users, Netflix decided to split the company’s identity in two, and launched Qwikster so they can focus on their rapidly growing steaming media business. About the same time they also changed their fee structure. Within days the online hordes were screaming foul, droves left the service, and as of today the stock price is down about 70 percent from its high this year. A quasi- apology was made.

There are many complex financial and business issues at play here, and there's no question that Netflix management failed on many levels.  There's also a consensus that, from a strictly business standpoint, Netflix was making the right decisions. All that aside, my question is this: Given that Netflix did extensive research and consulted with their community of users before they made any changes, why was the subsequent reaction so profoundly negative?

One answer gets at the root of the real problem with Klout, which measures an individual’s influence. Often, this is the wrong unit of analysis. I’d argue that Netflix, like many brands before it, fell victim to a mob – a highly vocal minority that individually may have no clout at all, but collectively exert enormous influence. Worse, this vocal and passionate minority may not even represent the feelings of the silent majority of users, but they exert a disproportionate control.

Mass Marketing is passé. Welcome to Mob Marketing.

The communities that care about a particular brand or organization are diverse – they’ve always been so. What’s changed is the leveling effect of our online world: Everyone has equal voice, which means that amid all the babble it’s very hard to discern who matters individually and collectively. And it’s almost impossible to guard against a loose coalition of marginal naysayers once they’re mobilized.

What to do? Here's some suggestions:

• First, make sure you are engaged with all constituents of your brand. Listen widely, respond selectively. Make sure that your communities feel appreciated. This is the responsibility of everyone in your organization.
• Remember that all change attracts enemies. No matter what you do, it’s likely someone will take offense. Remain in control and have the courage of your convictions. Recognize that, despite what believers in crowd sourcing may say, giving over decision-making control to an unfiltered community may be unwise. Consult, inform and listen.
• Make sure you understand what the valuable – and often silent – majority want and do everything you can to get them involved. Activating your core base is critical: The weight of their collective opinion is the best defense against a marginal mob. Find ways to amplify their views and champion your brand.
• Finally, learn to recognize the marginal fanatics. Don’t overly invest in trying to change their hardened views – your energies are better spend cultivating and engaging your loyalists, and attracting new fans and supporters.

Brand America

In these fractious and partisan times it’s interesting how U.S. politicians of all stripes are eager to distance themselves from their chosen profession. Democrats and Republicans alike, no matter their actual tenure, are all suddenly Washington “outsiders.” Many have also developed a newly found appreciation for how businesses are run, and think government could learn a trick or two from corporate America. We don’t need a President, they seem to be saying, we need a CEO.

Government-as-business is an interesting concept, which got me thinking: Why not elect an American CMO, a kind of Marketer-in-Chief?

The question isn’t as facetious as it might seem. Lots of countries actually do elect or appoint someone – often a “president” with limited legislative clout – to represent their country without having any overt political baggage. Truth is, in Europe that’s what royalty’s for.

America desperately needs a Marketer-in-Chief. America the Brand isn’t so brave anymore. You don’t need to conduct an audit to see the signs of dwindling brand loyalty and a confused brand identity. A good CMO would have read the signals long ago: The latest Rasmussen Poll shows that only 16 percent of likely voters think their country is “heading in the right direction,” while only 35 percent think America’s best days are to come. This is Quickster bad. The truth is, behind all the Tea Party bluster and Take Wall Street theatrics, there’s a shared disquiet that the US has lost its way and compromised on some ideals.

Overseas, Brand America is being bashed as bad as tainted Tylenol or New Coke. A couple of foreign wars certainly don’t help, and a worldwide financial pandemic is also souring the mood. Interestingly, Obama’s reputation abroad remains strong – his personal brand is relatively unscathed.

But Brand America has dealt with crisis before and come through, so, what’s the problem now? I blame the politicians. As some famous American once said, “a house divided cannot stand,” and Brand America has some deep-sea-trench divisions on strategy and values. No self-respecting CMO would stand for this. It used to be that the country coalesced around a universally accepted brand promise: Liberty, justice and the pursuit of happiness, etc. This was considered enough. Now, while we might agree on the brand promise, we dogmatically disagree on how to deliver on that promise. Even the role of government is being questioned. Meanwhile, as our fearless political leaders seem to encourage polarization, we flounder dealing with looming competitive threats from Brand China and the rest.

As any CMO knows, brand equity is a fragile commodity, hard to earn and too easily lost. However, in the case of Brand America, I think the gloomy prognosis is overstated.

A couple of months ago I was sworn-in as a US citizen, along with 260 others from over 30 different countries. It was a moving and sobering experience – I sat next to a Somali woman and a Nigerian man, both with harrowing stories of lost relatives and exile. An Indian man told me about the Pharma start-up he was creating – in the US. That same week, I listened to American scientists from the Kepler project describe discovering the first planets from other solar systems. And this year U.S. citizens netted seven Nobel prizes. On the global stage, America is still a place of invention and promise. The old cliché is true: American is a land of opportunity.

All we need is a Marketer-in-Chief to sell it better.

Is social media saturating – and what are the implications for marketing?

According to the latest data from Nielsen Research and Experian Simmons, the adoption of social media in the US has skyrocketed to 80 percent of those with online access, or over 40 percent of the overall population. Putting this in perspective, among those 35-and-younger, the adoption of social media is approaching that of US car ownership. Safe to say, it is clearly foolish to think of social media as something new or novel – it is an everyday part of most people’s lives.

Collectively, Americans now spend almost a billion hours a month exchanging news, information and gossip at social media sites. Last I checked there are still only 24 hours in a day: As you look at these stats, especially for the college-age set, the use of social media starts to look like borderline addiction. For many teens and tweens being connected online is an umbilical necessity for sustaining life. You wonder if Facebook and the rest shouldn’t carry warning labels, like cigarettes: Using Social Media Is Not A Substitute For Food, Sleep Or Reality.

More seriously, as social media adoption becomes mainstream, what are the consequences for marketing?

First, marketing pros should understand that media use overall is usually a zero-sum game: If we’re all spending more time online with social media, it usually means we’re doing less of some other media activity. There’s evidence to support this view, with online activities eating away at everything from watching television to reading. Second, we should anticipate that the use of social media is saturating. The amount of time we spend on social media is cresting, especially among those 35-and-younger.

This is a familiar scenario for marketing folks who have lived through other communications and marketing revolutions. I can remember when email marketing was a novelty, with response rates for well-targeted campaigns routinely reachig 3-5 percent (I’ve seen similarly impressive stats for the novelty-of-the-moment, QR codes). As social media saturates we should expect it to become harder and harder to create and sustain relationships between prospects and our brands. Saturation equates to clutter, attentions wane, and people become weary and guarded.

We should also remember that social media got that name for a reason – it’s a tool for staying in touch with friends and family, not primarily for finding your company, not matter how lovely it may be. Understand that social media is only one part of an integrated marketing program. Have realistic expectations - and make sure you communicate these to your management teams. As you work with social media, be true to yourself and your brand. The fundamentals of good marketing have not been re-invented: Be engaging, be credible, be trustworthy, and be unique.

Sales and Marketing May be Better Aligned Than We Think

When I was asked by Bob Johnson at IDG to be on a SMEI panel to discuss the divide between sales and marketing, I had a strong feeling of déjà vu: I can remember ten years ago being invited on another panel to debate the exact same issue. That event had been in London, hence there was an open bar before the panel convened; I recall the discussion getting very heated indeed.

As I prepared for the SMEI panel I began to wonder if the marketing/sales feud had improved much in the intervening decade, so decided to pull together a short survey to find out. Using social media I got just under 100 responses from an assortment of friends, followers and colleagues  – certainly not a representative sample, but diverse nonetheless.

Q: Over the last 5 years, do you think the relationship between sales and marketing has: 

Grown Worse:                                                 24%
Stayed about the same:                                   33%
Grown Better:                                                 43%

The results surprised me; I’d guessed that as we endure a drawn-out recession, relations between sales and marketing would have become more frayed. I was wrong, and by a large majority the audience at the SMEI event also believed sales and marketing were getting along much better these days.

I then asked was might be the root cause of any conflict:

Q: The main cause of conflict and disagreement between marketing and sales is (pick one): 

Lack on alignment on goals and objectives:        43%
Resource allocation:                                          11%
The quantity and quality of leads:                       23%
No clear demarcation of responsibility:                4%
Lack of processes to coordinate activities:         12%
Lack of professional respect:                               8%

By a wide margin, most respondents think that conflict is caused by a lack of alignment around goals and objectives. This view was echoed by my fellow panelists, Ann Marie Beasley from CA and Mark Blessing at Bright Computing.

I discussed how the most common model used for engineering alignment across sales and marketing – the ubiquitous marketing funnel – may be long overdue for an overhaul. Whatever model you use, successful alignment across the two organizations starts with a candid discussion on joint KPIs and metrics. Measured accountability, agreed and shared, is the root to success.

Marketing and Sales: Closing the Great Divide

I'm excited to have been invited to speak at the Sales and Marketing Executives International event here in Boston on September 14th. The subject is a perennial favorite -- Marketing and Sales: Closing the Great Divide.

I'm interested in your views - over the last five years has the relationship between sales and marketing improved, grown worse, or stayed about the same? What causes friction? Leave me a comment -- or better yet, take 30 seconds to complete this three-question survey. 

I'll share results of the survey in a later post. Thanks!

The Science of Marketing?

Reading recent headlines and watching the gyrations of the stock market it’s easy to see why economics earned the epithet “the dismal science.” But is marketing any better? Indeed, is there any science to marketing at all?

Usually not.

Take social media as an example. Normally, marketing treats social media data in the same way that air traffic controllers treat blips on a radar screen – as signals that require careful help in landing, or emergencies demanding evasion on interception. The only difference is we marketeers attempt to land customers and intercept naysayers, rather than planes. Don’t get me wrong – marketing needs radar – but this hardly qualifies as science.

Data by itself isn’t science: Marketing pros usually have an excess of the former and very little of the latter. A scientific approach would require proposing a hypothesis to explain observations or ideas, then devising an experiment to test the hypothesis. It requires the rigorous definition of terms and ways of quantifying things. It demands objectivity.

Tom Webster gets at this distinction is his recent blog post, and argues that we’ll always need a mix of qualitative and quantitative information. It’s tempting to agree with Seth Godin and others and say marketing is both art and science, but this feels like a cope-out to me.

What we can say is that marketing is getting much more quantitative. This is a good thing. I can remember from my teaching days that marketing undergrads hated even the most rudimentary classes on quantitative analysis and statistics – today, they’d be well-advised to take these courses very seriously indeed. Science or not, the future of marketing will clearly move toward a more disciplined and measured approach. But the real gains will be with those organizations that go the next step, and apply more scientific rigor to their marketing investments.