Meaning is essential as it is elusive. While potentially irksome when confronting the banalities of everyday existence, such ambiguity is consequential, if not detrimental, in a national security environment. Textual analysis enables one to realize the content, structure, and function of words and phrases contained in documents. Such understandings are particularly relevant in Industrial Engineering (IE) as textual analysis allows one to rigorously interrogate potential negotiated meanings resident in often complex systems of texts and explore implications for action. To illustrate we generate analytically-based insights of the historical versions of the United States Air Force’s (USAF’s) “little blue book” document. We leverage word clouds to reveal high-frequency terms, n-gram analyses to illustrate how key words and phrases are employed, and sentiment analysis to ascertain potential contextual polarities in the text. Collectively these insights help to delimit the elusive meaning(s) of a text and provide one with an analytic inroad for understanding. In the process, one is able to establish better the clarity of intent required to effectively execute one’s mission.
The following documentation along with this repository accompanies the research paper “Unpacking the United States Air Force’s Little Blue Book(s)” submitted for inclusion in the Proceedings of the 2016 Industrial and Systems Engineering Research Conference. Due to the constrained page limit for submissions this year the methodology section for the submitted paper was greatly truncated. This file expands the methodology section to provide a more robust understanding of the approach applied. Furthermore, this repository contains the code scripts employed in the analysis and the text analyzed.
Computational textual analysis of the four historical versions of the little blue book was executed via R programming software. To accomodate this, the historical versions of the Air Force “Little Blue Book” documents were converted to text (.txt) documents. The published and text-converted files are contained in this repository. The converted .txt files that our textual analysis effectuates are titled 1983r.txt, 1997r.txt, 2012r.txt, & 2015r.txt.1
The four documents exhibit substantially different format styles. One version might contain an opening note, table of contents, or reference section, while other versions do not. To standardize the page count across the different format types, a decision was made to focus on the number of pages associated with the “body text” of the document. In short, the page count reflects the number of pages on which “body text” is resident, and not the total number of pages contained in the document. While this complicates slightly the determination of page count, it is considered more reflective of the underlying differences in content among the texts. These “body text” sections were the basis of the word count and content analyses as well.
Total word count and page density (words per page) for each document were calculated in the descriptive_statistics.R script. These descriptives are calculated by importing the text files, splitting the text by individual words, eliminating extra white space and then aggregating.
To assess the content of the documents we begin by assessing frequently used terms. This is performed through a comparative word cloud for which the code can be assessed in the comparative_wordcloud.R script. The comparison word cloud algorithm assesses both inter and intra word frequency such that \(p_{i,j}\) is the rate at which word \(i\) occurs in document \(j\), and \(p_j\) is the average across \(n\) documents (\(\sum_{i}p_{i,j}/n\)). The size of each word is mapped to its maximum deviation ( \(max_i(p_{i,j}-p_j)\) ), and its quadrant position is determined by the document where that maximum occurs.2
To aid the assessment of the comparison word cloud, further evaluation of single term frequency was performed. We assessed the frequency of single terms within each document, compared the frequency of terms found in all four documents, and then compared words that were not commonly found between each document using pairwise assessments. These assessments can be found in the single_term_frequency.R script.
To advance our content analysis we moved from single word analysis to assessment of phrases through \(n\)-gram analysis. An \(n\)-gram is simply a contiguous sequence of words and assessment of these word sequences allows us to understand more behind the emphasized phrases in each document.
Our focus is to understand the substantive phrases commonly used; consequently, we remove common stop words3 so that we can focus on the principal elements of the grammar 4 rather than conjunctions and coordinations which primarily serve as linkages within sentences.5
To perform our \(n\)-gram analysis we use RWeka to interface with Weka and tokenize the document text into bi-grams, tri-grams, and quad-grams and develop our term document matrix. We then visually assessed the frequent bi-, tri-, and quad-grams for each little blue book text. These assessments can be found in the n-gram_XXXX files where XXXX is represented by the year of the document (i.e. 1983, 1997, etc.)
Lastly, we transition to sentiment analysis. The first part of our sentiment analysis is to classify the polarity (e.g. positiveness versus negativeness) of the four texts using a naïve Bayes classifier trained on Janyce Wiebe’s subjectivity lexicon6\(^,\)7. The dataset containing the list of positive and negative subjective words can be viewed in the sentiment/data/subjectivity.csv file.
The Naïve Bayes model involves a simplifying conditional independence assumption. That is, given a particular class (positive versus negative in our case), the words are conditionally independent of each other. This assumption has shown to not significantly influence the accuracy in text classification.8\(^,\)9 The maximum likelihood probability of a word belonging to a particular class is simply the the count of a particular word (\(x_i\)) of positive or negative class (\(c_i\)) in a given document divided by the total number of words of class \(i\) in the document:10
\[P(x_i | c) = \frac{\sum [x_i \in c_i]}{\sum \forall x_i \in c_i}\]
According to the Bayes Rule, the probability of a particular document belonging to a class \(c_i\) is given by,
\[P(c_i | d) = \frac{P(d | c_i) * P(c_i)}{P(d)}\]
Using the simplifying conditional independence assumption, that given a class (positive or negative), the words are conditionally independent of each other, the model is considered “naïve”.
\[P(c_i | d) = \frac{\prod P(x_i | c_j) * P(c_j)}{P(d)}\]
Consequently, the classifier outputs the class with the maximum posterior probability. To implement we use the sentiment package and the code can be found in the files titled Polarity Assessment XXXX.R in which XXXX is represented by the document year of interest. The analysis provides four outputs for each text assessed:
Lastly, as a result of prior analyses we suspected a possible shift in directive versus inclusive language across the blue book versions. To assess we compared the use of “you” (directive) to “our” (inclusive) across the documents. We normalized by dividing the total count of occurrences for each of these two words in each document by the total number of words in the respective document. The code for this can be found in the directive_inclusive.R file.
The published documents contained in the AF Little Blue Book Documents folder and are titled AF Blue Book - XXXX where XXXX is represented by the year published (1983, 1997, etc.). An online version of the most recent (2015) document can be found here.↩
Ian Fellows (2014). wordcloud: Word Clouds. R package version 2.5. https://CRAN.R-project.org/package=wordcloud.↩
Stop words refer to the most common words in a language. English stop words generally include words such as the, is, at, which, etc.↩
I.e. Nouns, verbs/actions, descriptive adjectives, etc.↩
Williams, J. M. & Bizup, J. (2014). Style: lessons in clarity and grace. Pearson.↩
Riloff and Wiebe (2003). Learning extraction patterns for subjective expressions. EMNLP-2003. http://www.cs.pitt.edu/mpqa/#subj_lexicon↩
Janyce Wiebe and Rada Mihalcea (2006). Word Sense and Subjectivity. Joint conference of the International Committee on Computational Linguistics and the Association for Computational Linguistics. (COLING-ACL 2006).↩
Maas, A. L., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y., & Potts, C. (2011, June). Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1 (pp. 142-150). Association for Computational Linguistics.↩
Narayanan, V., Arora, I., & Bhatia, A. (2013). Fast and accurate sentiment classification using an enhanced Naive Bayes model. In Intelligent Data Engineering and Automated Learning–IDEAL 2013 (pp. 194-201). Springer Berlin Heidelberg.↩
This notation uses the “Iverson bracket notation” that denotes a number that is 1 if the condition in square brackets is satisfied, and 0 otherwise.↩