Sentimental Software

Software generally isn't sentimental.

Ask a software program to find everything on the Web that talks about Barack Obama or Flat Panel TVs or Interocitors and it does pretty well. It doesn't matter how simple or obscure, how abundant or scarce, computers are very good at finding stuff.

However, ask a software program to find everything on the Web that says nice things about Obama, TVs or Interocitors and the program gets queasy and confused and apparently random in how it responds. Humans may understand nice, nasty and neutral in astonishing detail, and pick up on the slightest nuance in how things are expressed, but computers don't. We understand plain spoken feelings, we laugh at a joke, we grok sarcasm. Said another way, humans feel things, computers can't.

Lots of people have tried to make computers more sensitive creatures by writing elaborate programs that attempt to disentangle the tone of what humans write. They usually start by trying to get computers to understand written expression in the way we do, grammatically, syntactically and lexically. It's tricky. Stuff my three year-old understands in I Love You, Goodnight might be obvious to most software programs that try and detect tone, but my nine year-olds' Harry Potter would be a real stretch. And my New York Times would bamboozle most sentiment detection systems a lot of the time.

A lot of people are working on this problem, because there's a lot of commercial applications (Wikipedia has a nice summary of the technology, the business potential and some of the companies that are trying to solve this problem). It has great potential in automated trading applications, brand management, PR, politics... the list is long. Today, automated tone or sentiment detection is often built into media monitoring systems, but most companies selling these products acknowledge the results can be erratic and instead rely on human readers -- now there's a really, really interesting job . In my day job, I'd love an application that could accurately assess the nice-nasty-neutral tone of the stories my company gets. And I have to believe there's a great start-up opportunity in this space, especially serving some vertical market sectors.

If anyone knows of good sentiment detection software, let me know.

Obama in Hong Kong

Newspaper headline in Hong Kong daily: America Changes Color, next to a picture of Lincoln on the left and Obama on the right.

Living another man's dream

I'm in Hong Kong this week with work, the first time I've been here in over 20 years, and all eyes are on the US.

On Nathan Road and in Central they're selling Obama t-shirts, and the local papers are filled with the rich prospect of not if he'll become president, but what he'll do when he gets the office. No more White House is a joke I heard on the Star Ferry.

He's living another man's dream, voiced over 45 years ago. Just when you completely loose faith in American politics and values, this happens. If it wasn't for an exquisite confluence of events it would never have happened at all, but it is happening and I never thought it would. Amazing.

If this doesn't get Obama elected, nothing will

CNN Money: Biggest Government Bailout in History.

PR Measurement is giving me a headache...

I've spent most of my career in marketing and lately have been concentrating on PR, which I manage for a publicly-traded company.

To say that PR and the media business have changed in the last few years is a bit like saying Bill Gates is comfortably well-off, or Neil Armstrong is a seasoned traveler, or Sarah Palin is low key; it's really hard to overstate the turmoil in the media business, and as a consequence the upheavals in PR (for a great take on how this has impacted tech publishing, see Tom Steinert-Thelkeld's blog).

Nowhere is this more apparent than in the media measurement business. I'm old enough to remember when clips really were clips: pieces of newsprint cut-out from a magazine or newspaper by some exceedingly patient, far-off reader, then painstakingly collated, annotated and mailed to me in a big bulging brown envelope. Today, almost all the news is online and much of it doesn't come from a traditional news outlet, yet most media metrics and coverage monitoring still function as if in an ink-smudged era.

For sure there's a bunch of new companies that have addressed the new media reality, and focused on social networks and brand management: Biz360, Cymfony, BuzzLogic, Vocus and RatePoint are some examples. They all essentially follow the same formula of aggregating digital news using some kind of search and filter system (you can still get the pieces of paper if you need them, but each little clip will cost you more than the newsstand price of the whole publication). The algorithms at the heart of these systems are usually based on fixed keywords (company name, ticker symbol, product names, etc.) then some additional processing based on either rudimentary rules of grammar and syntax or or a series of logical operations, and is often called natural language processing (NLP), since it tries to emulate how humans read and understand text. The results, based on my limited experiences, range from the amazing to the bizarre, and most systems need human intervention to get at subtle things like tone.

In an attempt to add value and differentiate themselves from free services like Google News, these companies also have a vast array of reports and dashboards that slice and dice data to show share-of-voice, on target messaging, competitive coverage, salience and on and on. Again, result may vary from those advertised...

Pricing does not seem to vary much: all cater to a similar audience of complex multinationals, usually in the financial services, pharma, or legal businesses, and costs are high. Or at least they seem high to me.

This complexity has spawned a lot of blogs. K.D. Paines is excellent on PR measurement, although recent posts suggest a level of complexity in getting truly comprehensive metrics that is daunting and might account for the high costs. Ed Moed has a lot of good stuff to say, too (although Ed, I think “What's so funny...” was written by Nick Lowe), and intelligent measurement has a lot to say about social media.

But at the end of the day I get a headache. It should be easy. It should be straightforward. It should be inexpensive. And it isn't. If anyone has ideas on how to crack the PR measurement problem, let me know.

Novel Novels

In 2007 there were a staggering 50,071 new works of fiction published in the US, or about a book every 30 minutes. This reminded me of the famous Kurt Vonnegut quip that there’s really only two plotlines in existence – ‘man falls in hole’, and ‘boy meets girl’ – which if he’s right would make for an awful lot of repetition.

More than anything else this might account for the increasing, willful absurdity of a lot of modern fiction. Even in the last few years we’ve had books narrated by a murder victim (Lovely Bones) and an autistic teenager (The Curious case of the Dog at Night Time), both very successful, and both upending ideas of the ‘unreliable narrator.’

All of this came to mind as I read for the first time Nabakov’s excellent Pale Fire, published back in 1962. It’s a great example of a novel novel: a twisty tale masquerading as a definitive, annotated edition of the last work of a (fictional) famous poet. I guess Nabakov was teaching at the time at Cornell, and he has great fun skewing academics and the academic interpretation of literature. The poem of the title, which is at the heart of the book, alternates between gorgeous, haunting imagery and great jokes, and the daft annotations that follow are laugh-out-loud, like Lolita. I feel I’m missing 90 percent of the references, but this is still one of the best books I’ve read in a long time.

Not many writers can pull-off literary tricks like Pale Fire, but here are a few other novel novels that derive a lot of impact from pulling apart the conventional narrative structure.

Time’s Arrow is probably Martin Amis’ best book, and mostly overlooked. It starts from an end, and end with a beginning: the books is entirely in reverse, following the death to birth passage of a man’s life, a Nazi war criminal who from old age moves back through youth and to Germany, where in a concentration camp he brings back to life millions of Jews. The book works in a lot of different ways, not least as a comment on redemption and atonement.

Vikram Seth, like Amis, was the golden boy of literature for a while, and wrote Golden Gate just before the fame hit. The whole novel, set in contemporary San Francisco, is written in iambic pentameter. At one point midway through the book Seth breaks down the “fourth wall” and directly addresses the reader – not as some postmodern trick, but out of exasperation with having to invent more plot with the right meter.

Cloud Atlas by David Mitchell got raves in the UK but far less adulation in the US. Some of the criticism has merit, but this book is still wonderful. Here the multiple narrators begin their tales then suddenly switch mid-sentence, and the six stories are dovetailed together in an ascending and descending sequence, each written is a distinctive style and voice that is utterly convincing.

And lastly, I read the latest Bond opus, Devil May Care, written by Sebastien Faulks writing as Ian Fleming. This is a hoot, and Faulks really is writing as Fleming here – pitch perfect, sexist, racist, and with a subtly subversive plot perfectly preserved in some early 1960s world that never really existed at all.

Skin Deep Politics

American politics can often appear shallow, but who would have thought it was only skin deep.

Last week the New York Times, in conjunction with CBS News, published their latest poll on the US presidential race. And Oh, what a shock:

After years of growing political polarization, much of the divide in American politics is partisan. But Americans’ perceptions of the fall presidential election between Mr. Obama, Democrat of Illinois, and Senator John McCain, Republican of Arizona, also underlined the racial discord that the poll found. More than 80 percent of black voters said they had a favorable opinion of Mr. Obama; about 30 percent of white voters said they had a favorable opinion of him.

On the other side, about 35% of whites have a positive opinion of McCain, compared to only 5% of blacks. Layer on this more research showing that about 30% of Americans admit to feelings of racial bias.

Then this week John McCain, who has a history of melanoma, announced that he's had a spot removed from his face, which is undergoing further testing. In a speech yesterday, McCain urged all Americans to use sunscreen and stay out of the sun if possible.

Both presumptive candidates have skin in the election game, literally. For one, it is about race, and for the other, age.