NLP Pipeline: An Overview

Guest lecture in IT576: Natural Language Processing (NLP) Techniques, Marymount University, USA

• Researcher at National Research Council, Canada
• Past experiences: Senior Data Scientist in two Canadian companies
• Before that: Assistant Professor at Iowa State University, USA - taught NLP, programming and data science courses
• Co-authored a book recently: “Practical Natural Language Processing - A Comprehensive Guide to Building Real-World NLP Systems”

(Today's session is based on material drawn from Chapter 2 of the book)

• Building NLP Systems: NLP Pipeline
• A real world NLP Pipeline: case study/discussion
  
• A pipeline from personal experience

[Q&A]

Goal: To show a quick preview of some aspects of building NLP systems we don't (typically) learn about in a classroom.

• We all have to live with a background of toddler noises for the rest of this session.
• I will ask questions a few times in the middle, anyone can unmute themselves and answer.
• You can ask questions towards the end (chat or unmute are okay)
• All code used here is at: https://github.com/practical-nlp/practical-nlp/

• Some basic linguistic ideas (e.g., words, morphology, part of speech, syntax etc. )
• Many algorithms (e.g., naive bayes, HMMs, LSTMs, Transformers etc)
• Different NLP tasks (e.g., POS tagging, parsing, coreference resolution, information extraction etc)
• Some important NLP applications (e.g., Machine Translation, Chatbots, Question Answering etc.)

[
    {
    "detectedLanguage": {
          "language": "en",
          "score": 1.0
    },
    "translations": [
          {
               "text": "Wie gut ist maschinelle Übersetzung?",
               "to": "de"
          }
    ]
    }
]

Advantage: You don't have to worry about setting stuff up, hiring a large NLP team, maintaining the NLP system etc.

Disadvantages:

• This only works if you have problem that exactly meets the specifications of such an available API
• No possibility of customization/modification
  
• Depending on how much you use, costs may escalate

Note: You still have to think whether this approach is a long term solution for your problem.

Assuming you already have some training data, let AutoML do the job for you.
e.g., https://cloud.google.com/automl

Advantages:

• We don't need an expert machine learning or NLP team
• We don't have to worry about training/tuning the model well
• We can get by with writing minimum “machine learning” related code ourselves.

Disadvantages:

• This works only if we have a large amount of training data already in place.
• It is still hard to customize if we have some custom pre/post processing, feature engineering etc.

Note: We still have take care of deployment, maintenance etc.

Here are the main components of a generic pipeline for modern-day, data-driven NLP system development

• Where do we get our data from?

• Let us say you are working in a software company, and they asked you to develop a customer ticket routing system that looks at a ticket text, and classifies it into three categories: technical, sales, other. Where will your data come from?

• In an ideal scenario, we have some historical data of customer tickets along with this routing information.

• But what if they were just routing to the right team, but not storing that information anywhere in the past? We don't have the training data we want!

• use a public NLP dataset (e.g., at https://datasets.quantumstat.com/)
• scrape the data from the web (e.g., customer support forums of other products, if they are available and are tagged with categories)
• work together with your customer support team and gradually collect the data you want.

• using pattern matching and other such methods to create some data, and use “data augmentation” methods to create large enough data to train your own models. (e.g., Snorkel, NLPAug etc)

• and so on.

What, according to you, is the format of data you see in NLP?

Data can come in all forms: PDF, Docs, HTML files, scanned png files, tables etc.

Text extraction and cleanup refers to the process of extracting raw text from the input data by removing all the other non-textual information.

Text extraction may not involve NLP per se, but it defines the rest of your NLP pipeline. Bad text extraction = Bad NLP system.

• PDF to text conversion is hard and imperfect. Not all pdfs can be efficiently parsed.

• When we are extracting text from images, We may see some characters not rendered properly, or some words extracted with spelling mistakes etc

• If all of the “dataset” is a large collection of pdf documents with scanned text along with some tables: it is like the worst of all NLP worlds :-)

We don't have a single solution that works for all. There are tools like Amazon Textract, but they are not perfect.

• Sentence segmentation and word tokenization.
• Stop word removal, stemming and lemmatization, removing digits/punctuation, lowercasing, etc.
• Normalization, language detection, code mixing, transliteration, etc.
• POS tagging, parsing, coreference resolution, etc.

(you probably are already familiar with some of these.)

• The goal of feature engineering is to capture the characteristics of the text into a numeric vector that can be understood by the ML algorithms.

• There are primarily two ways of feature extraction in NLP:

  • hand crafted features (e.g., number of words/sentences in a document, number of spelling errors/sentence etc.)
  • automatically extracted features (e.g., TF-IDF, which you already know)

• Heuristics based systems (e.g., regular expressions, rule based matching etc)
• Learning from data: Machine learning (e.g., logistic regression, support vector machines etc.)
• Model ensemble (combining predictions from multiple models)
• Model cascade (using one model's prediction as input to another model)
• …….

• Intrinsic evaluation focuses on intermediary objectives, while extrinsic focuses on evaluating performance on the final objective.

• Consider a email spam classification system:

  • Intrinsic evaluation will focus on measuring the system performance using precision and recall.
  • Extrinsic evaluation will focus on measuring the time a user wasted because a spam email went to their inbox or a genuine email went to their spam folder.

• Once we have a good model, we have to deploy it in the context of a larger system (e.g., spam classification is a part of an email software)

• A common approach: deploy the NLP system as a micro service/web service

• Model monitoring: e.g., using a performance dashboard showing the model parameters and key performance indicators

• Model updating: Model has to stay current, with changing data. So, we should have some way of regularly updating, evaluating and deploying a model.

source

source

What is the difference between traditional pipeline and deep learning pipeline?

What are some advantages and disadvantages of deep learning?

What are some advantages and disadvantages of transfer learning?

If you already know some ML/deep learning usage, how many steps of this pipeline did you learn/think about so far?

It is a tool used within Uber to help agents do better customer support, by supporting quick and efficient issue resolution for a majority of Uber's support tickets.

(i.e., instead of asking the customer to answer several questions related to the issue, automate that process so that agents can react more quickly.)

Task: identify the issue type, and find out the right resolution based on ticket text, and other info such as trip data.

“COTA can reduce ticket resolution time by over 10 percent while delivering service with similar or higher levels of customer satisfaction”

“deep learning can improve the solution's top-1 prediction accuracy by 16 percent (from 49 percent to 65 percent) for the Contact Type model, and 8 percent (from 47 percent to 55 percent) for the Reply model compared to COTA v1, which can directly improve the customer support experience.”

• What is one important thing you learnt from this quick look at COTA?
• For me, it is:
  • start with a relatively straight forward approach, and build incrementally.
  • We don't have to start with deep learning.
    
  • Have your extrinsic evaluation measures in place (here: speed of resolution or improved customer support experience etc) along with intrinsic ones

• What according to you is the most important part of the NLP Pipeline?

• Should our data to train an NLP system come from only a single source?

• When are rule based “models” relevant?

• What can be a rule based approach to developing a sentiment analyzer?

Different approaches to setup a Named Entity Recognition model

Scenario: NER with legal documents (e.g., agreements etc) which are PDFs.

• deploy using an off the shelf solution (most likely approach)
• use an off the shelf solution, tune it to your domain, and then deploy
• build your own NER model, using rules
• build your own NER using machine learning/deep learning

import spacy

nlp = spacy.load("en_core_web_sm")
doc = nlp("Apple is looking at buying U.K. startup for $1 billion")

for ent in doc.ents:
    print(ent.text, ent.start_char, ent.end_char, ent.label_)

Output:

Apple 0 5 ORG
U.K. 27 31 GPE
$1 billion 44 54 MONEY

• a lookup table with a large collection of names of people/organizations/etc relevant for your organization +
• a bunch of hand crafted rules (e.g., a proper noun followed by “was born” may indicate a person) etc.

Spacy's entity ruler is a useful tool for this kind of an approach.

• using feature engineering + machine learning
  https://github.com/practical-nlp/practical-nlp/blob/master/Ch5/02_NERTraining.ipynb

• using spacy's training pipeline (deep learning)
  https://github.com/practical-nlp/practical-nlp/blob/master/Ch5/04_NER_using_spaCy%20-%20CoNLL.ipynb

• using transfer learning
  https://github.com/practical-nlp/practical-nlp/blob/master/Ch5/05_BERT_CONLL_NER.ipynb

• Explored various NLP libraries and cloud providers. We figured they don't do so well for our use case.
• Trained our own NER with a standard dataset (CONLL-03), and with simple features, as a base NER model.
• Used prodi.gy for domain specific annotation, and added these examples to the existing model to update it (and tune it for legal docs)
• Added some rules/heuristics on top of a machine learning model
• Explored a model ensemble with 2-3 models
• Looked at training with noisy text (i.e., output of pdf to text conversion + sentence/word segmentation) …. ….

• off the shelf NER is not perfect, but it is a good starting point.
• If you have only a small amount of annotated data for your domain, you can explore transfer learning, and gradually collect more domain specific data.
• A reliable approach afterwards: rules + (feature engineering + model)
• If you want to use deep learning, check if it is better than the above process first (for any NLP problem)
• Remember: your implementation should also be deployable, not just accurate.

• There is no single model. We have to set up a model monitoring/updating pipeline using some evaluation criteria.
• There is no single training set or test set. They will also evolve with time.
• Try doing NER on a PDF document - it will change your perspective on what is the most important part of a NLP pipeline :-)

• NLP tools we use in our pipeline are not perfect. Even a simple thing as text extraction or tokenization can have many unresolved issues. While our models are all very valuable effort, these steps are, too.

• No model can solve the problem of data quality. So, focus on getting good quality data to solve your problem first.

• Build a solution incrementally. Don't jump into the most complex solution first. Eventually, you want your stuff to be reliable, and not too expensive to maintain in short/long term.

contact: vbsowmya @ gmail

Questions?