Questions to ask your prospective social media listening, monitoring and measurement vendor

Last I checked, there are an astonishing 88 products that claim they can support social media measurement. When shopping for a solution, this overabundance renders most marketing types catatonic. We're confused by this surplus. The vendors themselves don't help us; desperate to differentiate themselves, they claim increasingly improbable capabilities and use clever obfuscation to suggest they are somehow unique and better than anyone else.

All this came to mind this week when I attended the Monitoring Social Media event in Boston. Various vendors and consultants – and sadly, not enough end-users – presented their views on how best to listen, measure and respond to social media activity. Whether you decide to use free tools or invest in a commercial solution, here's my takeaway on the key questions to ask vendors about their products:

Sources:
The output of any social media tool is only as good as the inputs. Verify exactly what primary social media properties the tool monitors, and the depth that it can reach on each. Find out how blogs are monitored, and how you can add sources and if that's easy.
Also ask how much history the tool supports – how long is content archived? Finally, ask about foreign language capabilities, and to what extent the tool supports non-English content.

Search/Monitoring:
To make a tool useful you need to tell it what interests you, so it can retrieve and sort content. Usually, this is done using familiar Boolean logic, not unlike with a Google search. Check how sophisticated and granular the search and monitoring capabilities are. Ask about other ways to search, train or query the system – some support so-called natural language searches, for example.

Filter:
My colleague Tangyslice has endured agonies with a commercial social media tool that failed dismally to effectively filter-out spam and duplicate information. A manual attempt to do this couldn't keep pace with the innovation of spammers, who seem to mutate and multiply constantly. Check that a tool can filter effectively.

Measurement:
This is where most people initially focus. Find out what the system can count. Can the system find relationships between data? How does it determine the salience of a source or other data? Can it detect sentiment, and for what? What about the valence of information or sources (in other words, how 'importance' or significance)?

Reporting & Alerting:
How are you going to present the data? What actions do you need to take with the information? How can you automate reporting, or send alerts to the right people in your organization? Its easy (at least for me) to get seduced by snazzy reporting engines that generate gorgeous graphs and charts, but often simple alerts and custom dashboards are a better approach. Think this through.

Connectivity:
Do you need to connect the system to others tools – say, a CRM solution, or another data mining product? Does the system easily support this?

Services:
Some systems are designed as self-service solutions that require the customers to drive and tweak. Others require a layer of professional services to make the tool work and to customize it. Think about what fits your needs.

Cost:
There's a lot of free stuff out there, and its possible to cobble-together a pretty good reporting system using Google, Tweetdeck and a few other tools. Beyond that, there's a layer of product in the $50-$100 a-month range that many tell me aren't very useful. Beyond that, expect to pay $400+ a month for a sophisticated product that covers a wide range of sources, can be customized, and will cover most company's needs (prices are for a single initial user).

Good luck! Let me know what I missed.