Tokenize words in Document

Given a character sequence and a defined document unit, tokenization is the task of chopping it up into pieces, called tokens , perhaps at the same time throwing away certain characters, such as punctuation.

These tokens are often loosely referred to as terms or words, but it is sometimes important to make a type/token distinction. A token is an instance of a sequence of characters in some particular document that are grouped together as a useful semantic unit for processing.

A type is the class of all tokens containing the same character sequence. A term is a (perhaps normalized) type that is included in the Information Retrieval(IR) system’s dictionary.

For example, if the document to be indexed is to sleep perchance to dream, then there are 5 tokens, but only 4 types (since there are 2 instances of to). However, if to is omitted from the index (as a stop word), then there will be only 3 terms: sleep, perchance, and dream.

metadata <- read_csv(sprintf("%s/%s", base_url, "metadata.csv"))
## Parsed with column specification:
## cols(
##   president = col_character(),
##   year = col_integer(),
##   party = col_character(),
##   sotu_type = col_character()
## )
metadata
## # A tibble: 236 x 4
##            president  year       party sotu_type
##                <chr> <int>       <chr>     <chr>
##  1 George Washington  1790 Nonpartisan    speech
##  2 George Washington  1790 Nonpartisan    speech
##  3 George Washington  1791 Nonpartisan    speech
##  4 George Washington  1792 Nonpartisan    speech
##  5 George Washington  1793 Nonpartisan    speech
##  6 George Washington  1794 Nonpartisan    speech
##  7 George Washington  1795 Nonpartisan    speech
##  8 George Washington  1796 Nonpartisan    speech
##  9        John Adams  1797  Federalist    speech
## 10        John Adams  1798  Federalist    speech
## # ... with 226 more rows
tab <- filter(tab, frequency < 0.002)
result <- c(metadata$president[236], metadata$year[236], tab$word[1:5])
paste(result, collapse = "; ")
## [1] "Barack Obama; 2016; laughter; voices; allies; harder; qaida"
result <- c(metadata$president[23], metadata$year[23], tab$word[1:5])
paste(result, collapse = "; ")
## [1] "James Madison; 1811; laughter; voices; allies; harder; qaida"
files <- sprintf("%s/sotu_text/%03d.txt", base_url, 1:236)
text <- c()
for (f in files) {
  text <- c(text, paste(readLines(f), collapse = "\n"))
}

# Exploratory 
words <- tokenize_words(text)
sapply(words, length)
##   [1]  1091  1404  2305  2100  1969  2922  1988  2879  2062  2220  1507
##  [12]  1374  3228  2203  2270  2100  2930  2863  2389  2676  1832  2448
##  [23]  2273  3248  3259  2115  3145  3368  4423  4378  4709  3447  5831
##  [34]  4733  6383  8416  9027  7745  6996  7329 10547 15109  7200  7889
##  [45]  7918 13473 10840 12386 11471 11521 13463  9011  8254  8436  8049
##  [56]  9331 16151 18251 16447 21368  7637  8340 13273  9951  9613 10152
##  [67] 11626 10512 13685 16396 12378 14085  6998  8410  6132  5975  9258
##  [78]  7155 12032  9886  7720  8772  6480 10131 10058  9844 12238  6816
##  [89] 10755  7909 11670 13405 13382 10315  8414  8956 19828 15196  5305
## [100] 13276 13038 11559 16357 13718 12360 15972 14710 15568 12157 20301
## [111] 22907 19228 19708  9825 15017 17517 25147 23680 27519 19515 13947
## [122] 27696 23838 25275  3566  4550  7735  2125  3931  5482  4765  2714
## [133]  5621  5774  6715  6978 10871 10333  8800  8083 11049  4560  5705
## [144]  4232  2240  3545  3849  2745  4736  3815  3244  3354  3555  4649
## [155]  3876  3162  8202 27942  6090  5130  3418  5154  4000  5387  6997
## [166]  9737  6027  7295  1090  8339  4166  4954  4995  5693  5300  6260
## [177]  6637  5429  3233  4453  5582  7221  4941  4137  4482  4532  3997
## [188] 17383  5199 22419  4153  4999  4747  4580 12155  3273 21668  3475
## [199] 33643  4470 33921  5182  5582  4968  4235  3493  3843  4851  4824
## [210]  3788  3964  5117  7033  7440  9216  6366  6787  7335  7535  9155
## [221]  4386  3845  5396  5201  5075  5342  5575  5729  6092  7263  6909
## [232]  7066  6872  7087  6819  6113
qplot(metadata$year, sapply(words, length))

qplot(metadata$year, sapply(words, length),
      color = metadata$sotu_type)

Stylometric Analysis

Stylometry is the application of the study of linguistic style, usually to written language, but it has successfully been applied to music and to fine-art paintings as well.

sentences <- tokenize_sentences(text)
sentence_words <- sapply(sentences, tokenize_words)
sentence_length <- list()
for (i in 1:nrow(metadata)) {
  sentence_length[[i]] <- sapply(sentence_words[[i]], length)
}

sentence_length_median <- sapply(sentence_length, median)
qplot(metadata$year, sentence_length_median)

qplot(metadata$year, sentence_length_median) +
  geom_smooth()
## `geom_smooth()` using method = 'loess'

Document Summarization

The topics can be retrieved from the frequency count of words in each speech of each president.

description <- c()
for (i in 1:length(words)) {
  tab <- table(words[[i]])
  tab <- data_frame(word = names(tab), count = as.numeric(tab))
  tab <- arrange(tab, desc(count))
  tab <- inner_join(tab, wf)
  tab <- filter(tab, frequency < 0.002)

  result <- c(metadata$president[i], metadata$year[i], tab$word[1:5])
  description <- c(description, paste(result, collapse = "; "))
}
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
## Joining, by = "word"
cat(description, sep = "\n")
## George Washington; 1790; ought; render; afford; blessings; derive
## George Washington; 1790; militia; requisite; abundant; belongs; consuls
## George Washington; 1791; indians; treaties; equally; gentlemen; naturally
## George Washington; 1792; tribes; frontier; hitherto; hostility; legislature
## George Washington; 1793; ought; militia; commissioners; dignity; fulfillment
## George Washington; 1794; militia; insurrection; inspector; ought; gentlemen
## George Washington; 1795; indians; tranquillity; gentlemen; treaty; blessings
## George Washington; 1796; treaty; commissioners; ought; majesty; proportion
## John Adams; 1797; treaty; commissioners; vessels; gentlemen; sums
## John Adams; 1798; commissioners; gentlemen; treaty; croix; demarcation
## John Adams; 1799; gentlemen; commissioners; satisfactory; treaty; amity
## John Adams; 1800; gentlemen; happiness; considerable; ought; treaty
## Thomas Jefferson; 1801; legislature; requisite; respecting; vessels; expedient
## Thomas Jefferson; 1802; vessels; discharge; legislature; naval; extraordinary
## Thomas Jefferson; 1803; friendship; treasury; vessels; neutral; ours
## Thomas Jefferson; 1804; discharge; dispositions; establishing; friendship; intercourse
## Thomas Jefferson; 1805; vessels; belligerent; competent; desirable; admit
## Thomas Jefferson; 1806; negotiations; clarke; inhabitants; mediterranean; messrs
## Thomas Jefferson; 1807; harbors; intercourse; legislature; militia; threatened
## Thomas Jefferson; 1808; belligerent; decrees; embargo; manifested; militia
## James Madison; 1809; arrangement; favorable; engagements; ratification; treasury
## James Madison; 1810; neutral; blockades; decrees; contemplated; danish
## James Madison; 1811; repeal; communicated; councils; decrees; friendship
## James Madison; 1812; savages; militia; favorable; armistice; considerable
## James Madison; 1813; prisoners; honorable; militia; savages; commander
## James Madison; 1814; militia; warfare; civilized; commanded; frontier
## James Madison; 1815; treasury; tribes; afford; attained; branches
## James Madison; 1816; treasury; favorable; intercourse; militia; prosperity
## James Monroe; 1817; colonies; tribes; arrangement; treasury; treaty
## James Monroe; 1818; savages; tribes; adventurers; indians; respecting
## James Monroe; 1819; treaty; ratified; majesty; ratification; explanations
## James Monroe; 1820; considerable; vessels; colonies; blessings; favorable
## James Monroe; 1821; vessels; manufactures; treaty; colonies; provinces
## James Monroe; 1822; treasury; likewise; vessels; respecting; communicated
## James Monroe; 1823; appropriated; expenditures; treasury; treaty; vessels
## James Monroe; 1824; treaty; tribes; negotiation; likewise; appropriation
## John Quincy Adams; 1825; treasury; accomplished; commissioners; deliberations; naval
## John Quincy Adams; 1826; colonies; vessels; discriminating; intercourse; receipts
## John Quincy Adams; 1827; intercourse; receipts; treaty; negotiation; vessels
## John Quincy Adams; 1828; treaties; commenced; intercourse; colonies; enumeration
## Andrew Jackson; 1829; treasury; discharge; disposition; injurious; feelings
## Andrew Jackson; 1830; vessels; treaty; appropriations; treasury; indians
## Andrew Jackson; 1831; treaty; indemnity; majesty; tribes; assurances
## Andrew Jackson; 1832; treasury; intercourse; heretofore; hoped; indians
## Andrew Jackson; 1833; treasury; treaty; chambers; intercourse; vessels
## Andrew Jackson; 1834; treaty; appropriation; treasury; chambers; appropriations
## Andrew Jackson; 1835; treaty; chambers; constitutional; intercourse; treasury
## Andrew Jackson; 1836; surplus; treasury; appropriations; ration; treaty
## Martin Van Buren; 1837; treasury; disposition; satisfactory; vessels; indians
## Martin Van Buren; 1838; treasury; indians; tribes; moneys; appropriations
## Martin Van Buren; 1839; treasury; specie; prosperity; afford; necessity
## Martin Van Buren; 1840; expenditures; indians; treasury; treaty; appropriations
## John Tyler; 1841; treasury; circulation; discharge; necessity; distant
## John Tyler; 1842; treasury; circulation; exchequer; treaty; specie
## John Tyler; 1843; treasury; regarded; greatly; heretofore; consequence
## John Tyler; 1844; treaty; annexation; treasury; happiness; negotiation
## James K. Polk; 1845; treasury; treaty; vessels; imposed; negotiation
## James K. Polk; 1846; treaty; paredes; redress; annexation; grande
## James K. Polk; 1847; treaty; treasury; indemnity; steamers; cession
## James K. Polk; 1848; treasury; expenditures; constitutional; veto; tariff
## Zachary Taylor; 1849; treaty; treasury; vessels; correspondence; intercourse
## Millard Fillmore; 1850; expenditures; treasury; hoped; respectfully; treaty
## Millard Fillmore; 1851; treasury; appropriations; expenditures; exports; expedition
## Millard Fillmore; 1852; indians; prosperity; blessings; naval; treasury
## Franklin Pierce; 1853; intercourse; treasury; regarded; appropriations; arrangement
## Franklin Pierce; 1854; treaty; naval; treasury; proposition; favorable
## Franklin Pierce; 1855; treaty; constitutional; expenditures; nicaragua; pretensions
## Franklin Pierce; 1856; treaty; constitutional; panama; repeal; territories
## James Buchanan; 1857; treaty; slavery; honduras; circulation; whilst
## James Buchanan; 1858; treaty; treasury; whilst; nicaragua; ought
## James Buchanan; 1859; treasury; treaty; slaves; expenditures; ought
## James Buchanan; 1860; constitutional; treaty; slave; slavery; sovereign
## Abraham Lincoln; 1861; insurrection; insurgents; hired; judges; appropriation
## Abraham Lincoln; 1862; emancipation; slavery; slave; colored; treasury
## Abraham Lincoln; 1863; naval; proclamation; emancipation; receipts; disbursements
## Abraham Lincoln; 1864; treasury; naval; pensioners; receipts; aggregate
## Andrew Johnson; 1865; preservation; freedmen; treason; circulation; electors
## Andrew Johnson; 1866; expenditures; constitutional; declared; emperor; expedition
## Andrew Johnson; 1867; treasury; circulation; inclusive; coin; expenditures
## Andrew Johnson; 1868; expenditures; treasury; treaty; circulation; republics
## Ulysses S. Grant; 1869; expenditures; treaty; receipts; treasury; largely
## Ulysses S. Grant; 1870; treaty; vessels; domingo; disposed; appropriation
## Ulysses S. Grant; 1871; treaty; desirable; appropriation; herewith; hoped
## Ulysses S. Grant; 1872; treaty; appropriation; majesty; steamship; disposed
## Ulysses S. Grant; 1873; correspondence; herewith; specie; expenditures; treasury
## Ulysses S. Grant; 1874; prosperity; herewith; hoped; naturalization; specie
## Ulysses S. Grant; 1875; shores; treasury; satisfactory; treaty; herewith
## Ulysses S. Grant; 1876; treaty; appropriations; naturalization; diplomatic; expenditures
## Rutherford B. Hayes; 1877; coinage; coin; tender; treaty; expenditures
## Rutherford B. Hayes; 1878; indians; expenditures; appropriations; commissioners; receipts
## Rutherford B. Hayes; 1879; appropriation; appropriations; examinations; expenditures; treasury
## Rutherford B. Hayes; 1880; appropriations; appropriation; treasury; necessity; treaty
## Chester A. Arthur; 1881; imports; indians; exports; treasury; territories
## Chester A. Arthur; 1882; expenditures; appointments; treasury; appropriations; lieutenant
## Chester A. Arthur; 1883; treaty; circulation; customs; treasury; vessels
## Chester A. Arthur; 1884; vessels; treaty; intercourse; treaties; expenditures
## Grover Cleveland; 1885; treaty; indians; coinage; vessels; citizenship
## Grover Cleveland; 1886; treaty; surplus; treasury; pensions; worthy
## Grover Cleveland; 1887; tariff; treasury; taxation; wool; surplus
## Grover Cleveland; 1888; expenditures; treaty; pensions; surplus; citizenship
## Benjamin Harrison; 1889; treasury; circulation; indians; surplus; appropriation
## Benjamin Harrison; 1890; tariff; largely; receipts; treaty; expenditures
## Benjamin Harrison; 1891; naval; electors; treasury; appropriation; tariff
## Benjamin Harrison; 1892; tons; wages; vessels; imports; exports
## Grover Cleveland; 1893; preceding; vessels; treaty; amounted; tariff
## Grover Cleveland; 1894; amounted; circulation; treasury; vessels; expenditures
## Grover Cleveland; 1895; treasury; circulation; arbitration; coinage; consular
## Grover Cleveland; 1896; preceding; expenditures; amounted; treasury; vessels
## William McKinley; 1897; cuban; tribes; civilized; negotiations; indians
## William McKinley; 1898; naval; santiago; admiral; treasury; vessels
## William McKinley; 1899; treaty; manila; treasury; exposition; archipelago
## William McKinley; 1900; treaty; imperial; philippine; peking; expenditures
## Theodore Roosevelt; 1901; merely; cannot; arid; irrigation; streams
## Theodore Roosevelt; 1902; tariff; corporations; evils; prosperity; canal
## Theodore Roosevelt; 1903; treaty; panama; isthmus; canal; colombia
## Theodore Roosevelt; 1904; naturalization; corporations; interstate; civilized; citizenship
## Theodore Roosevelt; 1905; cannot; interstate; railroad; corporations; supervision
## Theodore Roosevelt; 1906; merely; colored; interstate; seals; corporations
## Theodore Roosevelt; 1907; interstate; corporations; railroads; supervision; tariff
## Theodore Roosevelt; 1908; corporations; judges; interstate; forests; fisheries
## William Howard Taft; 1909; tariff; deficit; naval; expenditures; treaty
## William Howard Taft; 1910; canal; tariff; naval; treasury; ought
## William Howard Taft; 1911; statute; canal; wool; tariff; corporations
## William Howard Taft; 1912; canal; panama; ought; tariff; naval
## Woodrow Wilson; 1913; ought; constitutional; nominees; administer; beg
## Woodrow Wilson; 1914; ought; profitable; constructive; friendship; jealous
## Woodrow Wilson; 1915; cruisers; submarines; treasury; fifteen; cannot
## Woodrow Wilson; 1916; interstate; railroads; acted; arbitration; concerted
## Woodrow Wilson; 1917; peoples; enemies; wrongs; alien; everywhere
## Woodrow Wilson; 1918; railways; railroads; billions; armies; empires
## Woodrow Wilson; 1919; unrest; exports; interstate; remedy; disputes
## Woodrow Wilson; 1920; treasury; expenditures; appropriations; floating; necessity
## Warren G. Harding; 1921; tariff; ought; necessity; reclamation; imports
## Warren G. Harding; 1922; railway; tile; ought; tribunal; freight
## Calvin Coolidge; 1923; ought; coal; burden; greatly; humanity
## Calvin Coolidge; 1924; nitrogen; prosperity; railways; taxation; burden
## Calvin Coolidge; 1925; ought; expenditures; judges; prosperity; tile
## Calvin Coolidge; 1926; ought; tariff; imports; cooperative; greatly
## Calvin Coolidge; 1927; farmer; surplus; treasury; appropriations; considerable
## Calvin Coolidge; 1928; prosperity; expenditures; surplus; encouragement; irrigation
## Herbert Hoover; 1929; tariff; appropriations; necessity; annum; desirable
## Herbert Hoover; 1930; distress; commodities; reorganization; unemployment; expenditure
## Herbert Hoover; 1931; unemployment; emergencies; deposits; expenditures; naval
## Herbert Hoover; 1932; reconstruction; expenditures; greatly; failures; accomplished
## Franklin D. Roosevelt; 1934; cannot; civilization; exploitation; mere; readjustment
## Franklin D. Roosevelt; 1935; cannot; undertaken; unemployed; livelihood; definite
## Franklin D. Roosevelt; 1936; autocracy; neighbor; peaceful; rulers; americas
## Franklin D. Roosevelt; 1937; speculation; unemployment; civilization; constitutional; continuously
## Franklin D. Roosevelt; 1938; abuses; expenditures; wages; cannot; balanced
## Franklin D. Roosevelt; 1939; aggression; eighty; idle; cannot; sixty
## Franklin D. Roosevelt; 1940; cannot; everywhere; centuries; civilization; compelled
## Franklin D. Roosevelt; 1941; everywhere; cannot; dictators; hemisphere; unity
## Franklin D. Roosevelt; 1942; conquest; hitler; enemies; planes; nazis
## Franklin D. Roosevelt; 1943; cannot; planes; conquest; criticism; decent
## Franklin D. Roosevelt; 1944; allies; cannot; enemies; sailors; selfish
## Franklin D. Roosevelt; 1945; peoples; allies; nurses; liberated; cannot
## Franklin D. Roosevelt; 1945; nurses; allies; cannot; enemies; peoples
## Harry S Truman; 1946; expenditures; appropriations; receipts; authorizations; peacetime
## Harry S Truman; 1947; disputes; strikes; bargaining; atomic; jurisdictional
## Harry S Truman; 1948; inflation; decade; incomes; restore; afford
## Harry S Truman; 1949; prosperity; cannot; authorize; afford; boom
## Harry S Truman; 1950; peoples; expenditures; ideals; cannot; mankind
## Harry S Truman; 1951; soviet; aggression; defend; ideals; allies
## Harry S Truman; 1952; communist; aggression; allies; cannot; soviet
## Dwight D. Eisenhower; 1953; inflation; deficits; promptly; burden; communist
## Harry S Truman; 1953; soviet; communist; atomic; rulers; cannot
## Dwight D. Eisenhower; 1954; allies; communist; propose; reductions; mobilization
## Dwight D. Eisenhower; 1955; vigorous; strengthen; urge; civilian; communist
## Dwight D. Eisenhower; 1956; expanding; allies; disarmament; intend; peacetime
## Dwight D. Eisenhower; 1956; prosperity; strengthen; atomic; communist; accomplished
## Dwight D. Eisenhower; 1957; prosperity; peoples; cannot; productive; atomic
## Dwight D. Eisenhower; 1958; missiles; missile; soviet; soviets; ballistic
## Dwight D. Eisenhower; 1959; expenditures; peoples; cannot; inflation; missile
## Dwight D. Eisenhower; 1960; peoples; cannot; expenditures; inflation; soviet
## John F. Kennedy; 1961; deficit; allies; domination; hopes; soviet
## Dwight D. Eisenhower; 1961; communist; strengthened; greatly; strengthening; unemployment
## John F. Kennedy; 1962; cannot; urge; allies; farms; compete
## John F. Kennedy; 1963; cannot; communist; afford; strengthen; communism
## Lyndon B. Johnson; 1964; hopes; cannot; unemployment; decent; afford
## Lyndon B. Johnson; 1965; propose; unity; communist; soviet; enrich
## Lyndon B. Johnson; 1966; propose; aggression; pursuit; sacrifice; strengthen
## Lyndon B. Johnson; 1967; soviet; succeed; allies; cannot; communist
## Lyndon B. Johnson; 1968; abundance; expenditures; propose; treaty; appropriations
## Lyndon B. Johnson; 1969; commitments; prosperity; treaty; presidents; surtax
## Richard M. Nixon; 1970; decade; seventies; propose; balanced; blame
## Richard M. Nixon; 1971; propose; localities; inflation; prosperity; peacetime
## Richard M. Nixon; 1972; burden; decades; bipartisan; commitments; compete
## Richard M. Nixon; 1972; manpower; inflation; propose; cannot; prosperity
## Richard M. Nixon; 1974; prosperity; afford; cannot; colleagues; lasting
## Richard M. Nixon; 1974; urge; inflation; bicentennial; prosperity; unemployment
## Gerald R. Ford; 1975; propose; coal; barrels; crude; inflation
## Gerald R. Ford; 1976; cannot; propose; inflation; generations; adversaries
## Gerald R. Ford; 1977; productive; branches; cannot; deter; recession
## Jimmy Carter; 1978; inflation; cannot; unemployment; canal; deficit
## Jimmy Carter; 1978; strengthen; propose; passage; inflation; allies
## Jimmy Carter; 1979; inflation; cannot; soviet; allies; congressional
## Jimmy Carter; 1979; inflation; propose; fy; congressional; exports
## Jimmy Carter; 1980; soviet; preserve; aggression; inflation; persian
## Jimmy Carter; 1980; soviet; inflation; urge; strengthen; propose
## Ronald Reagan; 1981; inflation; reductions; subsidies; taxpayers; expenditures
## Jimmy Carter; 1981; soviet; coal; allies; urge; assure
## Ronald Reagan; 1982; inflation; deficits; unemployment; needy; soviet
## Ronald Reagan; 1983; deficits; inflation; soviet; allies; bipartisan
## Ronald Reagan; 1984; bipartisan; inflation; courage; eighties; restore
## Ronald Reagan; 1985; stronger; barriers; propose; spreading; abortion
## Ronald Reagan; 1986; cannot; soviet; courage; decade; deficit
## Ronald Reagan; 1987; soviet; competitiveness; hemisphere; honorable; nicaragua
## Ronald Reagan; 1988; laughter; deficits; nicaragua; prosperity; reforms
## George Bush; 1989; propose; cannot; deficit; priorities; freeze
## George Bush; 1990; panama; soviet; anchor; announcing; bless
## George Bush; 1991; struggle; aggression; soviet; blessings; persian
## George Bush; 1992; laughter; missiles; burden; cannot; gains
## William J. Clinton; 1993; deficit; propose; incomes; invest; decade
## William J. Clinton; 1994; deficit; renew; ought; brady; cannot
## William J. Clinton; 1995; ought; covenant; deficit; bureaucracy; voted
## William J. Clinton; 1996; bipartisan; gangs; medicare; deficit; harder
## William J. Clinton; 1997; bipartisan; cannot; balanced; nato; immigrants
## William J. Clinton; 1998; bipartisan; deficit; propose; bosnia; millennium
## William J. Clinton; 1999; medicare; propose; surplus; balanced; bipartisan
## William J. Clinton; 2000; propose; laughter; medicare; bipartisan; prosperity
## George W. Bush; 2001; medicare; courage; surplus; josefina; laughter
## George W. Bush; 2002; terrorist; terrorists; allies; camps; homeland
## George W. Bush; 2003; hussein; saddam; inspectors; qaida; terrorists
## George W. Bush; 2004; terrorists; propose; medicare; seniors; killers
## George W. Bush; 2005; terrorists; iraqis; reforms; decades; generations
## George W. Bush; 2006; hopeful; offensive; retreat; terrorists; terrorist
## George W. Bush; 2007; terrorists; qaida; extremists; struggle; baghdad
## George W. Bush; 2008; terrorists; empower; qaida; extremists; deny
## Barack Obama; 2009; deficit; afford; cannot; lending; invest
## Barack Obama; 2010; deficit; laughter; afford; decade; decades
## Barack Obama; 2011; deficit; republicans; democrats; laughter; afghan
## Barack Obama; 2012; afford; deficit; tuition; cannot; doubling
## Barack Obama; 2013; deficit; deserve; stronger; bipartisan; medicare
## Barack Obama; 2014; cory; laughter; decades; diplomacy; invest
## Barack Obama; 2015; laughter; childcare; democrats; rebekah; republicans
## Barack Obama; 2016; laughter; voices; allies; harder; qaida