Perhaps you have noticed that there really are no successful text analytics systems, which are in general use on people’s desktops. It is fair to ask why this is the case.
It isn’t that people don’t have the need to absorb larger bunches of text. In fact, I might take a guess that the basic approach taken by the makers and vendors that have preceded us isn’t appropriate to what people want to get from text data.
Alternatively, Leximancer is built to analyze big, medium or small; English or Greek or Malay; medical, CPG or high-tech; long or short bodies of unstructured text from just about any source. The idea of what we’re accomplishing is new, and so is our way of making customers and partners successful.
The purpose of this posting is to examine what text offers most people, then compare this with what previous attempts at text analyzing software have tried to do, and failed, as well as arm you with questions to consider when evaluating options.
Text tells the story.
Text tells us the story. A good story lays out the ideas and characters with their attributes. We read the text to set the scene – to explain the situation that we have dropped in on. It is like the first episode of a TV series. After that, we read on to see how the characters and ideas interact. There are changing relationships.
A survey or report or set of online product reviews are no different. We need to see what issues, products or services are front-of-mind for the authors or responders, what attributes they assign to these issues and products, and how they see the relationships. We then move on to start answering questions and fixing problems. This is how we apply the knowledge gained.
In concrete terms:
1. We discover the concepts of the situation from the text.
2. We discover the explanations, or insights, from the text.
3. We can then act on these insights to alter the system.
Step 1 is important and neglected. You cannot understand the situation without understanding the background ideas.
You cannot understand an IT textbook using the concepts from political science. You would struggle to paint a seascape with a palette suitable for a childs cartoon. Unfortunately, this problem is insidious and leads to mistakes that we fail to notice. Why? Because if we naively analyze some data with a set of ideas that we know well, and we fondly expect will apply to the data, we may never see that we are missing a quite different perspective.
Most text analyzing systems will not automatically extract a clear set of the concepts and actors that characterize the text. Systems that come with predefined sets of categories, dictionaries and entity lists are a menace. You cannot risk interpreting your data filtered through an understanding created by someone who is not familiar with your data and your situation, even if the answer looks simple and neat. This leads to
Question 1: Does the system’s set of categories, entities, and concepts reflect a real understanding of my data and my situation?
Some systems use predefined categories that are manually tuned by the vendor during pre-sales. The vendor’s consultants will sift through your data and construct extensive lists of terms, pattern matchers and possibily rules. The analysis will then look okay at that time, but things change. New issues will arise in your business, and the terms and entities will change over time. This leads to Question 2:
Question 2: How much time and effort did the vendor invest in tuning the category dictionaries, rules, and entity lists before go-live? When your data inevitably changes, can you afford to feasibly repeat this process to maintain the fidelity of your analysis?
If the analytics system does not use predefined categories, it may use document or word clustering. Many such systems do not produce clear or validated concepts. Remember that for easy and regular use, the discovered patterns of meaning need to be stable and clear. Don’t be fooled by people who say that this sort of system works because it looks attractive and even compelling. There are ways to check whether discovered term clusters are real measures of meaning, or whether they are wasting your time. Here are some questions for vendors who offer term or document clustering or other concept map solutions:
Question 3: If the product uses document clustering: how does the system scale with vast numbers of documents? If a document contains several different ideas, can it be in two topics at once? If I cut up the same documents into different chunks, would the pattern of clusters be similar? Text content isn’t always organized in predictable ways, so this is an important set of questions.
Question 4: If I take two different documents either by different authors or in different languages, would the discovered patterns of meaning look similar between the two? Multinationals – think about this if you want a consistent, true view of your customer comments.
Step 2 is almost totally ignored. Text information can tell you a story so you can improve business performance—with customers, with marketing. What else would you really want to do with it?
Quantitative, categorical, and numerical data mining is really good for establishing metrics and testing to see if pre-defined metrics change. Great. Do this. It is really good for predicting whether a pre-selected situation is matched, such as customer churn probability.
But don’t forget that analyzing text comments from customers or competitor product reviews on the other hand excels at telling you what is happening. Because text is human communication – that is what it is for. So why waste this extremely valuable and rich source of intelligence?
Think of it this way. If your metrics show your sales are rising, everyone feels great. If your metrics show you your results are falling off a cliff, how do you work out how to fix the system? This is the feedback you need for controlling a system. Your text data will tell you how to turn things around faster and more accurately than almost any other source of management information.
Unfortunately, this is where most text analytics systems fail or don’t even bother. Here are some other questions:
Question 5: Does the system suggest chains of meaning which are well supported by the data, and which I can understand and explain to a manager? In other words, is it an explanatory model?
Question 6: Can I test hypotheses (educated guesses) based on the perspective of the customer?
Question 7: How does a simple list of terms tell me much about the reasons for what is happening, without having to do a whole lot of guessing or having to read large amounts of text after all?
Step 3: Set your bar high and expect an automatic, systematic and scalable system that can enable unstructured textual information to become a real enterprise asset—good for uncovering new customer insights, new product ideas, and business process improvements that were previously unachievable. And now act on what you find!
I hope this helps. People are still doing a whole lot of writing and talking trying to tell you things. I think we need to listen more carefully, understand what they are saying and then act thoughtfully.
By Andrew E. SmithRead Full Post | Make a Comment ( 5 so far )
Leximancer recently was featured on the industry renowned B-eye Network. Mary Jo Nott, executive editor of the B-eye Network interviewed Leximancer CEO Neil Hartley about Leximancer’s growth and technology.
The pair talked about short- and long-term business strategy, current and future trends in the market place and the importance of being able to extract actionable customer insight from social media – including specifics of what Leximancer is doing to address this customer need.
Bottom line is that many text analytics options are both cost-prohibitive and time-prohibitive. According to Hartley:
“First, it needs to be usable by a business person who can pick it up and get to usable set of results extremely quickly. Second, it needs to be able to process masses of data regardless of the language or the source where the data is coming from and without the need for any setup. And it needs to go the extra mile in getting to the root cause of problems – particularly in that customer insight space, just listening or knowing what customer attitudes are is not enough. You need to get to the why your customers are happy or why they are unhappy, so you can make the insight actionable. After all, the link between cause and effect is not always a straight-forward, one-to-one connection.”
To learn more, please join us for a live Web demo. In the meantime, send us your data and we will help you uncover valuable customer insight specific to your business.Read Full Post | Make a Comment ( 1 so far )
Today marked the formal launch of The Customer Insight Portal, and the news was picked up around the Web.
Forbes.com, Marketwatch.com, Morningstar.com, Reuters, Yahoo! Finance and others picked up the story of how The Customer Insight Portal is the “game changer” for companies looking to gain insight into their customers
The full press release:
“Game Changer” Customer Insight Portal Launched
Powerful Leximancer software available as Software as a Service (SaaS)
BOULDER, Colo., Aug 14, 2008 – Leximancer, a Customer Experience Management (CEM) and analytics software development company, today announced the launch of The Customer Insight Portal, a Web-based Software as a Service (SaaS) that delivers insight to not only what customers are saying, but why they’re saying it.
“The Customer Insight Portal is a game changer for the way companies seek to understand their customer attitudes and behaviors,” said Neil Hartley, Leximancer CEO. “It allows virtually any organization or individual user to easily gain insight into the root causes of customer opinion and feedback. Our state-of-the-art market intelligence software has been proven on the desktop world-wide and now we are making it available through The Customer Insight Portal.”
For consumer-focused organizations, The Customer Insight Portal gives marketing professionals, brand managers, competitive intelligence and customer experience managers the ability to make critical decisions based on factual data regarding customers’ thoughts and feelings toward their brand, products or service. The Customer Insight Portal goes multiple steps beyond traditional text analytics by employing a rich variety of scientific methods to analyze call center notes, survey data, e-mails, documents, blogs, social media and Web sites. This enables Leximancer to uniquely provide insight into the root causes of customers’ attitudes and actions, allowing companies to determine not just what people think of them, but also why.
“The Customer Insight Portal lets business people explore and automatically find meaning, and identifies structured relationships between the key ideas or issues that are important to customers – uncovering information that was previously hidden,” said Chris Westfall, Leximancer vice president of business development. “From the people that have already used the portal, they’re saying that it provides previously unknown, actionable insights, which is great validation of what’s possible.”
The Customer Insight Portal provides deep insight without the need for set up, which means that analysis is provided without previous knowledge of the information under investigation. Users of the Customer Insight Portal can upload the data and the analysis is complete in the time it takes to make a pot of coffee. Because there is no selection of terms before getting started the results are unbiased. Companies find out what is there, not just what they think should or might be there.
Leximancer’s patent pending software platform allows customer satisfaction, brand management and competitive intelligence professionals to automatically extract the root causes of customer attitudes from Internet communications such as blogs, Web sites and social media, as well as the vast amount of data currently locked within the enterprise in the form of e-mails, service notes, call center notes, voice transcripts and survey feedback. Through its intuitive keyword discovery, cause-and-effect analysis, thesaurus and search functions, Leximancer is the only solution that delivers deep insight into customer attitudes by objectively identifying “unknown unknowns.” For more information, visit http://www.leximancer.com or http://www.thecustomerinsightportal.com.
# # #Read Full Post | Make a Comment ( None so far )
Leximancer has started its expansion from Europe and Australia into the North American market – and that move is gaining attention as businesses realize the power of the software, the great possibility of partnerships with OEMs and the ease and depth of The Customer Insight Portal.
We were featured in the business section of the Boulder Daily Camera (a regional publication in Colorado). Business writer Alicia Wallace highlights why the Leximancer software and The Customer Insight Portal are such important business tools in today’s marketplace.
The article also talks about the features of Leximancer including language independence, the ability of the technology to deal with massive amounts of data and quotes Chris Westfall, Leximancer vice president for business development:
“The program, which is language-independent, analyzes the words, phrases and context from a broad spectrum of forums and pools that information together in a chart- and web-based form that allows the users to gain meaning from the content.”
Also noted in the article was the unique ability for Leximancer and The Customer Insight Portal to work with no prior set up. You upload the data and the analysis begins, in the time it takes to make a pot of coffee. And, the software delivers the root cause of customer problems or issues, giving insight in the “why” there is an attitude or behavior, not just “what” that attitude or behavior might be.Read Full Post | Make a Comment ( None so far )