12 Applications of Natural Language Processing

One problem I encounter again and again is running natural language processing algorithms on documents corpora or lists of survey responses which are a mixture of American and British spelling, or full of common spelling mistakes. One of the annoying consequences of not normalising spelling is that words like normalising/normalizing do not tend to be picked up as high frequency words if they are split between variants. For that reason we often have to use spelling and grammar normalisation tools.

Natural language processing (NLP) is an interdisciplinary subfield of computer science and linguistics. It is primarily concerned with giving computers the ability to support and manipulate human language. It involves processing natural language datasets, such as text corpora or speech corpora, using either rule-based or probabilistic (i.e. statistical and, most examples of natural language recently, neural network-based) machine learning approaches. The goal is a computer capable of „understanding“ the contents of documents, including the contextual nuances of the language within them. The technology can then accurately extract information and insights contained in the documents as well as categorize and organize the documents themselves.

Final Words on Natural Language Processing

The Cyc KB is a resource of real world knowledge in machine-readable format. While early versions of CycL were described as being a frame language, more recent versions are described as a logic that supports frame-like structures and inferences. Cycorp, started by Douglas Lenat in 1984, has been an ongoing project for more than 35 years and they claim that it is now the longest-lived artificial intelligence project[29]. Other necessary bits of magic include functions for raising quantifiers and negation (NEG) and tense (called “INFL”) to the front of an expression. Raising INFL also assumes that either there were explicit words, such as “not” or “did”, or that the parser creates “fake” words for ones given as a prefix (e.g., un-) or suffix (e.g., -ed) that it puts ahead of the verb. We can take the same approach when FOL is tricky, such as using equality to say that “there exists only one” of something.

The sentences, while longer, are still relatively basic and are likely to contain a lot of mistakes in grammar, pronunciation or word usage. However, the progress is undeniable as more content is added to the speech. You’re not forced to utter words or phrases, much less pronounce them correctly. There are no endless drills on correct usage, no mentions of grammar rules or long lists of vocabulary to memorize.

Text and speech processing

You will notice that the concept of language plays a crucial role in communication and exchange of information. Programming is a highly technical field which is practically gibberish to the average consumer. NLP can help bridge the gap between the programming language and natural language used by humans.