Natural language processing (NLP) is a set of techniques for using computers to detect in human language the kinds of things that humans detect automatically.
NLP is an exciting field of computer science, artificial intelligence, and computational linguistics concerned with the interactions between computers and human (natural) languages. It includes word and sentence tokenization, text classification and sentiment analysis, spelling correction, information extraction, parsing, meaning extraction, and question answering.
In our formative years, we master the basics of spoken and written language. However, the vast majority of us do not progress past some basic processing rules when we learn how to handle text in our applications. Yet unstructured software comprises the majority of the data we see. NLP is the technology for dealing with our all-pervasive product: human language, as it appears in social media, emails, web pages, tweets, product descriptions, newspaper stories, and scientific articles, in thousands of languages and variants.
Many challenges in NLP involve natural language understanding. In other words, computers learn how to determine meaning from human or natural language input, and others involve natural language generation.
Python and R stand toe-to-toe in data science. But in the field of NLP, Python stands very tall. The Natural Language Toolkit (NLTK) for Python is an awesome library and set of corpuses.
However, R offers competent libraries for natural language processing. Many of the techniques such as word and sentence tokenization, n-gram creation, and named entity recognition are easily performed in R.
Start with NLP, offering basic classes and methods. Then take a gander at openNLP which provides an interface to the Apache OpenNLP tools. Speaking of interfaces, there’s also RWeka, an R interface to Weka. But there’s some excellent R packages if you look further afield. We make our recommendations captured in a ratings chart.
Let’s explore the R based NLP tools at hand. For each title we have compiled its own portal page, a full description with an in-depth analysis of its features, together with links to relevant resources.
| R Natural Language Processing Tools | |
|---|---|
| tidytext | Text mining using dplyr, ggplot2, and other tidy tools |
| text2vec | Framework with API for text analysis and natural language processing |
| quanteda | R package for Quantitative Analysis of Textual Data |
| wordcloud | Create attractive word clouds |
| tm | Text Mining Infrastructure in R |
| srtringi | Fast and portable character string processing in R |
| Stringr | String manipulation in R |
| UDPipe | Tokenization, Tagging, Lemmatization and Dependency Parsing |
| Word Vectors | Build and explore embedding models |
| spacyr | R wrapper around the Python spaCy package |
 Read our complete collection of recommended free and open source software. Our curated compilation covers all categories of software. The software collection forms part of our series of informative articles for Linux enthusiasts. There are hundreds of in-depth reviews, open source alternatives to proprietary software from large corporations like Google, Microsoft, Apple, Adobe, IBM, Cisco, Oracle, and Autodesk. There are also fun things to try, hardware, free programming books and tutorials, and much more. |
Want to learn to program in R? Try our compilation of excellent free R books.