Overall

• Bridge quantitative and qualitative research
  • Provide examples that cross paradigms
  • What can we learn from each paradigm?

Today

• Would some of these topics be relevant across paradigms?
• How could they be presented in an engaging way?
  • Real examples

“Natural Language Processing (NLP)” is broadly defined as the automatic manipulation of natural language, like speech or text, by software.” (https://machinelearningmastery.com/natural-language-processing/)

• How can NLP assist researchers in extracting meaning from large volumes of text?
  • Journal articles
  • Open-ended survey responses
  • Social media posts
  • News articles
  • Other

• Cuddapah, J. L. (2002). The Teachers College New Teacher Institute: Supporting New Teachers through Mentoring Relationships.
• Cuddapah, J. L., & Clayton, C. D. (2011). Using Wenger’s communities of practice to explore a new teacher cohort. Journal of Teacher Education, 62(1), 62-75.
• Cuddapah, J. L., & Stanford, B. H. (2015). Career-changers’ ideal teacher images and grounded classroom perspectives. Teaching and teacher education, 51, 27-37.
• Gifford, J., Snyder, M. G., & Cuddapah, J. L. (2013). Novice career changers weather the classroom weather. Phi Delta Kappan, 94(6), 50-54.
• Masci, F. J., Cuddapah, J. L., & Pajak, E. F. (2008). Becoming and agent of stability: Keeping your school in balance in the perfect storm. American Secondary Education 36(2), 57-68.

• Import the Text
• Clean the Text
• Perform Statistical Analysis
• Create Visualizations

• Text Corpus
  • Can store text from multiple documents
  • Holds additional metadata (e.g. title, author, date, page numbers)

##   author       : Jennifer Locraft Cuddapah
##   datetimestamp: 2106-02-07 01:28:15
##   description  : Teaching and Teacher Education, 51 (2015) 27-37. doi:10.1016/j.tate.2015.05.004
##   heading      : Career-changers’ ideal teacher images and grounded classroom perspectives
##   id           : IdealTeacherImages.pdf
##   language     : en
##   origin       : Elsevier

##  [1] "Teaching and Teacher Education 51 (2015) 27e37"                         
##  [2] ""                                                                       
##  [3] "Contents lists available at ScienceDirect"                              
##  [4] ""                                                                       
##  [5] "Teaching and Teacher Education journal homepage:"                       
##  [6] "www.elsevier.com/locate/tate"                                           
##  [7] ""                                                                       
##  [8] "Career-changers’ ideal teacher images and grounded classroom"           
##  [9] "perspectives Jennifer Locraft Cuddapah a, *, Beverly Hardcastle"        
## [10] "Stanford b, 1 a Hood College, 401 Rosemont Avenue, Frederick, MD 21701,"

• Remove numbers, punctuation, symbols, etc.
  • Remove irrelevant clutter
• Convert all terms to lower case
  • e.g. Should “Principal” be treated as the same word as “principal”?
• Remove stopwords
  • e.g. the, a, and, if, is, are
• Stemming / Lemmatization
  • e.g. Should “train”, “trained”, “training”, etc., be treated in common?

Original text (from https://monkeylearn.com/blog/text-cleaning/)

• “Hey Amazon - my package never arrived https://www.amazon.com/gp/css/order-history?ref_=nav_orders_first PLEASE FIX ASAP! @amazonhelp”

Step 1: Remove numbers, punctuation, symbols, etc.

• “Hey Amazon my package never arrived PLEASE FIX ASAP”

Original text (from https://monkeylearn.com/blog/text-cleaning/)

• “Hey Amazon - my package never arrived https://www.amazon.com/gp/css/order-history?ref_=nav_orders_first PLEASE FIX ASAP! @amazonhelp”

Step 2: Convert to lower case

• “hey amazon my package never arrived please fix asap”

Original text (from https://monkeylearn.com/blog/text-cleaning/)

• “Hey Amazon - my package never arrived https://www.amazon.com/gp/css/order-history?ref_=nav_orders_first PLEASE FIX ASAP! @amazonhelp”

Step 3: Remove stopwords

• “amazon package never arrived fix asap”

Original text (from https://monkeylearn.com/blog/text-cleaning/)

• “Hey Amazon - my package never arrived https://www.amazon.com/gp/css/order-history?ref_=nav_orders_first PLEASE FIX ASAP! @amazonhelp”

Step 4: Stemming

• “amazon package never arrive fix asap”

At what level should the text be analyzed?

• Terms
  • Single words (unigrams)
  • N-grams (bigrams, trigrams, etc.)
  • Sentences
  • Paragraphs
  • Pages

• Term Frequency \[ tf_w^i = \frac{n_w^i}{n^i} \]

• Inverse Document Frequency

\[ idf_w = log(\frac{D}{j | t_w \in d_j}) = log(\frac{1}{df_w}) \]

• Inverse Document Frequency

• Tf-Idf \[ TfIdf_w^i = tf_w^i * idf_w \]

• Top Words per Document by TF

• Top Words per Document by TF-IDF

• n-Grams

• Word Clouds

• Term Correlation Graphs

• Latent Dirichlet Allocation (LDA)

• Latent Dirichlet Allocation (LDA) Topic by Document

• Sentiment Analysis

• Sentiment Analysis