Well, it all depends what you mean by ‘read’. Such a small word that implies so much based upon context. If you told me you read this article, it means you understood it. When a machine ‘reads’ a file, it typically means load and scan. When a machine ‘reads’ a survey response, it scans it, and applies predetermined algorithms to the words. It cannot possibly understand the meaning of the text; if it did, it truly would be artificially intelligent             

            A few years ago I led a project to analyse the written interactions between support agents and customers to gain insights into the reason for the call. We set out to analyse a corpus of text-based interactions between customers and service personnel, derive the topics and themes of the discussions, and use them to understand more about the company’s products and why customers need to call regarding their use. Thanks to a great team of analysts and data scientists, we built a prototype and celebrated a 70% success rate. While this may not seem worth celebrating, in the world of text analysis it’s quite good.

             The methodology is complex, but here are the basic steps. The text is cleaned by removing words (stop words) that have less meaning, such as pronouns. Similar words, such as plurals, are merged by stemming (shortening) them. What is left is a dictionary of meaningful words which can then be analysed. Clustering algorithms then scan for words that commonly occur together, and these clusters are surfaced to a SME (Subject Matter Expert) who answers the question ‘ if you see these words together in a piece of text what topic would you think is being discussed?’ Their answer becomes a document tag, and common tags can be counted, and graphically displayed. To test the validity of the output, a sample of the tagged documents are read by a person who compares the tag to the text, and notes whether the tag correctly describes the subject. This is how we assessed our 70% success rate.

            We humans automatically make assumptions when reading text, and one of them is context. We know up front, for instance, whether the conversation is about a disk storage unit or a fridge freezer. Our SME automatically assumes this knowledge when assigning a tag. Machines know nothing about context unless we provide that information.
While the SME is asked to provide the understanding, the machine can apply it methodically to large numbers of conversations very quickly. The next nut to crack is to get the machine, based on past experience, to learn to apply the SME understanding and create the tag. One way this can be done is to record word clusters and associated tags in a database, and use a search algorithm, however it’s important to search by context to get meaningful results. A start has been made by some MiT researchers who have assembled an open database of word associations and topics called ConceptNet that can be looked up by other applications such as Luminoso, which uses ConceptNet to infer topics and themes without the need for human intervention.

            Companies such as Clarabridge and Medallia combine many of these techniques to turn pages of text, such as TripAdvisor comments, into quantifiable terms. Social media tools such as Attensity use similar tools to trawl through tweets and facebook posts to provide insights into what customers are saying on social media. Their products can also determine the sentiment of the conversation by looking for key words and other words they occur with. Bill Inmon’s Forest Rim Technology Textual ETL product uses a relational database to align the results of context, ontology, taxonomy and text processing techniques in a form that can feed directly to a visualisation tool such as Tableau or Qlikview.