What are the Natural Language Processing Challenges, and How to Fix?

Unsolved Problems in Natural Language Understanding Datasets by Julia Turc

In response to this, we organized a number of research gatherings in collaboration with colleagues around the world, which led to establishment of a SIG (SIGBIOMED) at ACL. The first workshop took place in 2002, collocated with the ACL conference (Workshop 2002). It has been expanding rapidly and has become one of the most active SIGs in NLP applications. The research field of application of structure-based NLP to text-mining is broadening to cover clinical/medical domains (Xu et al. 2012; Sohrab et al. 2020), chemistry, and material science domains (Kuniyoshi et al. 2019). I soon realized, however, that the research would involve a whole range of difficult research topics in artificial intelligence, such as representation of common sense, human ways of reasoning, and so on.

At the same time, considering NLP as an engineering field, I took it to be essential to have a clear definition of knowledge or information with which language is to be related. I would like to avoid too much vagueness of research into commonsense knowledge and reasoning and to restrict our research focus to the relationship between language and knowledge. As a research strategy, I chose to focus on the biomedicine as the application domain.

Learn

Supertags were derived from the original HPSG grammar and a set of supertags were attached to a word in its lexicon. A suppertagger would choose the most probable sequence of supertags for the given sequence of words. The task was a sequence labeling task, which could be carried out in a very efficient manner (Zhang, Matsuzaki, and Tsujii 2009). This means that the surface local context (i.e., local sequences of supertags) was used for disambiguation, without constructing actual DAGs of features.

That is, translation would be constructed in a bottom up manner, from smaller units of translation to larger units. Another view shared by the community was an abstraction hierarchy of representation, called the triangle of translation. For example, Figure 3(a),4 shows the hierarchy of representation used in the Eurotra project, with their definition of each level (Figure 3(b)). Both of the lower disciplines are concerned with processing language, that is, how language is processed in our minds or our brains, and how computer systems should be designed to process language efficiently and effectively. We sat down with David Talby, CTO at John Snow Labs, to discuss the importance of NLP in healthcare and other industries, some state-of-the-art NLP use cases in healthcare as well as challenges when building NLP models. With the rising popularity of NFTs, artists show great interest in learning how to create an NFT art to earn money.

Approaches: Symbolic, statistical, neural networks

Informal phrases, expressions, idioms, and culture-specific lingo present a number of problems for NLP – especially for models intended for broad use. Because as formal language, colloquialisms may have no “dictionary definition” at all, and these expressions may even have different meanings in different geographic areas. Furthermore, cultural slang is constantly morphing and expanding, so new words pop up every day. Natural Language Processing (NLP) research at Google focuses on algorithms that apply at scale, across languages, and across domains. Our systems are used in numerous ways across Google, impacting user experience in search, mobile, apps, ads, translate and more. The evaluation criteria (EvaRQ4) of the proposals are indicated in the fifth column, which emphasizes the preference of such proposals for the Area Under the Curve-Receiver Operating Characteristic (AUC-ROC).

Differently, we have identified studies for temporal handling and explanation as the two main research trends in this area. Temporal handling is a compulsory requirement for the health domain and the inexistence of such ability is a barrier for the use of the transformer technology in real applications. Similarly, explainability is also becoming a compulsory requirement for deep learning problems in nlp models, according to the upcoming AI regulations. Indeed, the explainability for transformers models and their results are in the initial stage, and this area requires strategies beyond the simple analysis of attention weights. These open questions are opportunities for research directions, which must mainly consider replicable forms to compare and justify their designs.

This is where contextual embedding comes into play and is used to learn sequence-level semantics by taking into consideration the sequence of all words in the documents. This technique can help overcome challenges within NLP and give the model a better understanding of polysemous words. Yes, words make up text data, however, words and phrases have different meanings depending on the context of a sentence.

Rospocher et al. [112] purposed a novel modular system for cross-lingual event extraction for English, Dutch, and Italian Texts by using different pipelines for different languages. The pipeline integrates modules for basic NLP processing as well as more advanced tasks such as cross-lingual named entity linking, semantic role labeling and time normalization. Thus, the cross-lingual framework allows for the interpretation of events, participants, locations, and time, as well as the relations between them. Output of these individual pipelines is intended to be used as input for a system that obtains event centric knowledge graphs. All modules take standard input, to do some annotation, and produce standard output which in turn becomes the input for the next module pipelines. Their pipelines are built as a data centric architecture so that modules can be adapted and replaced.

Top Natural Language Processing (NLP) Techniques

Moreover, the topics had to deal with uncertainty and peculiarities of individual humans. Knowledge or the world models that individual humans have may differ from one person to another. In the extreme view, the top of the hierarchy was taken as the language-independent representation of meaning. Proponents of the interlingual approach claimed that, if the analysis phase reached this level, then no transfer phase would be required. Rather, translation would consist only of the two monolingual phases (i.e., the analysis and generation phases).

Noah Chomsky, one of the first linguists of twelfth century that started syntactic theories, marked a unique position in the field of theoretical linguistics because he revolutionized the area of syntax (Chomsky, 1965) [23]. Further, Natural Language Generation (NLG) is the process of producing phrases, sentences and paragraphs that are meaningful from an internal representation. The first objective of this paper is to give insights of the various important terminologies of NLP and NLG. It is a known issue that while there are tons of data for popular languages, such as English or Chinese, there are thousands of languages that are spoken but few people and consequently receive far less attention. There are 1,250–2,100 languages in Africa alone, but the data for these languages are scarce.

The data characterization (EvaRQ1) column shows that most of the datasets present a high number of samples. This high number is essential for approaches that require a pretraining stage. We also included the data dimension (dim) used for each approach between square brackets. For example, the work of Li et al. (2020) has only one dimension represented by diagnosis codes. Approaches that do not require this stage and rely on mobile data (Shome 2021; Dong et al. 2021), for example, present fewer samples.

On the other hand, direct application of CL theories to NLP did not work, since this would result in extremely slow processing. We had to transform them into more processing-oriented formats, which required significant efforts and time on the NLP side. For example, we had to transform the original HPSG grammar into processing-oriented forms, such as supertags, CFG skeletons, and so on.

Natural language processing

The most interesting example is the study in Fouladvand et al. (2021), with authors from the Biomedical Informatics, Computer Science, Internal Medicine, Pharmaceutical Sciences, Biostatistics, Psychiatry, Family and Community Medicine departments. In this case, the involvement of interdisciplinary teams is related to the research complexity that involves aspects of opioid use disorder. In the quest for highest accuracy, non-English languages are less frequently being trained.

• Rationalist approach or symbolic approach assumes that a crucial part of the knowledge in the human mind is not derived by the senses but is firm in advance, probably by genetic inheritance.
• Semantic analysis focuses on literal meaning of the words, but pragmatic analysis focuses on the inferred meaning that the readers perceive based on their background knowledge.
• We next discuss some of the commonly used terminologies in different levels of NLP.
• Creating and maintaining natural language features is a lot of work, and having to do that over and over again, with new sets of native speakers to help, is an intimidating task.
• According to experimental results (Culurciello 2018), RNNs are good for remembering sentences in the order of hundreds but not thousands of timesteps.
• Compared with language in medical records, language in published papers is not so restricted and intertwined with rules of general language.

The third column represents the longitudinal unit, which aggregates the data assessed at each timestep (InpRQ2). Many studies use the idea of visits as longitudinal unit, and they are aperiodic (Li et al. 2020; Darabi et al. 2020). For aperiodic units, for example, the work in Boursalie et al. (2021) proposes concatenating the time between assessments (elapsed time) in the unit encode. Proposals based on sensors (Shome 2021; Dong et al. 2021) have each data capture cycle as their longitudinal unit, which is mostly periodic. Models uncover patterns in the data, so when the data is broken, they develop broken behavior. This is why researchers allocate significant resources towards curating datasets.