rm(list=ls())
setwd("C:\\Users\\huiwu\\OneDrive\\Desktop\\CAU\\Spring2025\\R_DataScience\\text_documents")
library("DMwR2")
## Registered S3 method overwritten by 'quantmod':
## method from
## as.zoo.data.frame zoo
library(tm)
## Loading required package: NLP
library(tidyverse)
## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr 1.1.4 ✔ readr 2.1.5
## ✔ forcats 1.0.0 ✔ stringr 1.5.1
## ✔ ggplot2 3.5.2 ✔ tibble 3.3.0
## ✔ lubridate 1.9.4 ✔ tidyr 1.3.1
## ✔ purrr 1.0.4
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ ggplot2::annotate() masks NLP::annotate()
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag() masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
docs <- Corpus(DirSource("Alice"))
docs
## <<SimpleCorpus>>
## Metadata: corpus specific: 1, document level (indexed): 0
## Content: documents: 1
docs[1]
## <<SimpleCorpus>>
## Metadata: corpus specific: 1, document level (indexed): 0
## Content: documents: 1
content(docs[[1]])[1:3]
## [1] "Alice’s Adventures in Wonderland\n\nby Lewis Carroll\n\nTHE MILLENNIUM FULCRUM EDITION 3.0\n\nContents\n\n CHAPTER I. Down the Rabbit-Hole\n CHAPTER II. The Pool of Tears\n CHAPTER III. A Caucus-Race and a Long Tale\n CHAPTER IV. The Rabbit Sends in a Little Bill\n CHAPTER V. Advice from a Caterpillar\n CHAPTER VI. Pig and Pepper\n CHAPTER VII. A Mad Tea-Party\n CHAPTER VIII. The Queen’s Croquet-Ground\n CHAPTER IX. The Mock Turtle’s Story\n CHAPTER X. The Lobster Quadrille\n CHAPTER XI. Who Stole the Tarts?\n CHAPTER XII. Alice’s Evidence\n\n\n\n\nCHAPTER I.\nDown the Rabbit-Hole\n\n\nAlice was beginning to get very tired of sitting by her sister on the\nbank, and of having nothing to do: once or twice she had peeped into\nthe book her sister was reading, but it had no pictures or\nconversations in it, “and what is the use of a book,” thought Alice\n“without pictures or conversations?”\n\nSo she was considering in her own mind (as well as she could, for the\nhot day made her feel very sleepy and stupid), whether the pleasure of\nmaking a daisy-chain would be worth the trouble of getting up and\npicking the daisies, when suddenly a White Rabbit with pink eyes ran\nclose by her.\n\nThere was nothing so _very_ remarkable in that; nor did Alice think it\nso _very_ much out of the way to hear the Rabbit say to itself, “Oh\ndear! Oh dear! I shall be late!” (when she thought it over afterwards,\nit occurred to her that she ought to have wondered at this, but at the\ntime it all seemed quite natural); but when the Rabbit actually _took a\nwatch out of its waistcoat-pocket_, and looked at it, and then hurried\non, Alice started to her feet, for it flashed across her mind that she\nhad never before seen a rabbit with either a waistcoat-pocket, or a\nwatch to take out of it, and burning with curiosity, she ran across the\nfield after it, and fortunately was just in time to see it pop down a\nlarge rabbit-hole under the hedge.\n\nIn another moment down went Alice after it, never once considering how\nin the world she was to get out again.\n\nThe rabbit-hole went straight on like a tunnel for some way, and then\ndipped suddenly down, so suddenly that Alice had not a moment to think\nabout stopping herself before she found herself falling down a very\ndeep well.\n\nEither the well was very deep, or she fell very slowly, for she had\nplenty of time as she went down to look about her and to wonder what\nwas going to happen next. First, she tried to look down and make out\nwhat she was coming to, but it was too dark to see anything; then she\nlooked at the sides of the well, and noticed that they were filled with\ncupboards and book-shelves; here and there she saw maps and pictures\nhung upon pegs. She took down a jar from one of the shelves as she\npassed; it was labelled “ORANGE MARMALADE”, but to her great\ndisappointment it was empty: she did not like to drop the jar for fear\nof killing somebody underneath, so managed to put it into one of the\ncupboards as she fell past it.\n\n“Well!” thought Alice to herself, “after such a fall as this, I shall\nthink nothing of tumbling down stairs! How brave they’ll all think me\nat home! Why, I wouldn’t say anything about it, even if I fell off the\ntop of the house!” (Which was very likely true.)\n\nDown, down, down. Would the fall _never_ come to an end? “I wonder how\nmany miles I’ve fallen by this time?” she said aloud. “I must be\ngetting somewhere near the centre of the earth. Let me see: that would\nbe four thousand miles down, I think—” (for, you see, Alice had learnt\nseveral things of this sort in her lessons in the schoolroom, and\nthough this was not a _very_ good opportunity for showing off her\nknowledge, as there was no one to listen to her, still it was good\npractice to say it over) “—yes, that’s about the right distance—but\nthen I wonder what Latitude or Longitude I’ve got to?” (Alice had no\nidea what Latitude was, or Longitude either, but thought they were nice\ngrand words to say.)\n\nPresently she began again. “I wonder if I shall fall right _through_\nthe earth! How funny it’ll seem to come out among the people that walk\nwith their heads downward! The Antipathies, I think—” (she was rather\nglad there _was_ no one listening, this time, as it didn’t sound at all\nthe right word) “—but I shall have to ask them what the name of the\ncountry is, you know. Please, Ma’am, is this New Zealand or Australia?”\n(and she tried to curtsey as she spoke—fancy _curtseying_ as you’re\nfalling through the air! Do you think you could manage it?) “And what\nan ignorant little girl she’ll think me for asking! No, it’ll never do\nto ask: perhaps I shall see it written up somewhere.”\n\nDown, down, down. There was nothing else to do, so Alice soon began\ntalking again. “Dinah’ll miss me very much to-night, I should think!”\n(Dinah was the cat.) “I hope they’ll remember her saucer of milk at\ntea-time. Dinah my dear! I wish you were down here with me! There are\nno mice in the air, I’m afraid, but you might catch a bat, and that’s\nvery like a mouse, you know. But do cats eat bats, I wonder?” And here\nAlice began to get rather sleepy, and went on saying to herself, in a\ndreamy sort of way, “Do cats eat bats? Do cats eat bats?” and\nsometimes, “Do bats eat cats?” for, you see, as she couldn’t answer\neither question, it didn’t much matter which way she put it. She felt\nthat she was dozing off, and had just begun to dream that she was\nwalking hand in hand with Dinah, and saying to her very earnestly,\n“Now, Dinah, tell me the truth: did you ever eat a bat?” when suddenly,\nthump! thump! down she came upon a heap of sticks and dry leaves, and\nthe fall was over.\n\nAlice was not a bit hurt, and she jumped up on to her feet in a moment:\nshe looked up, but it was all dark overhead; before her was another\nlong passage, and the White Rabbit was still in sight, hurrying down\nit. There was not a moment to be lost: away went Alice like the wind,\nand was just in time to hear it say, as it turned a corner, “Oh my ears\nand whiskers, how late it’s getting!” She was close behind it when she\nturned the corner, but the Rabbit was no longer to be seen: she found\nherself in a long, low hall, which was lit up by a row of lamps hanging\nfrom the roof.\n\nThere were doors all round the hall, but they were all locked; and when\nAlice had been all the way down one side and up the other, trying every\ndoor, she walked sadly down the middle, wondering how she was ever to\nget out again.\n\nSuddenly she came upon a little three-legged table, all made of solid\nglass; there was nothing on it except a tiny golden key, and Alice’s\nfirst thought was that it might belong to one of the doors of the hall;\nbut, alas! either the locks were too large, or the key was too small,\nbut at any rate it would not open any of them. However, on the second\ntime round, she came upon a low curtain she had not noticed before, and\nbehind it was a little door about fifteen inches high: she tried the\nlittle golden key in the lock, and to her great delight it fitted!\n\nAlice opened the door and found that it led into a small passage, not\nmuch larger than a rat-hole: she knelt down and looked along the\npassage into the loveliest garden you ever saw. How she longed to get\nout of that dark hall, and wander about among those beds of bright\nflowers and those cool fountains, but she could not even get her head\nthrough the doorway; “and even if my head would go through,” thought\npoor Alice, “it would be of very little use without my shoulders. Oh,\nhow I wish I could shut up like a telescope! I think I could, if I only\nknew how to begin.” For, you see, so many out-of-the-way things had\nhappened lately, that Alice had begun to think that very few things\nindeed were really impossible.\n\nThere seemed to be no use in waiting by the little door, so she went\nback to the table, half hoping she might find another key on it, or at\nany rate a book of rules for shutting people up like telescopes: this\ntime she found a little bottle on it, (“which certainly was not here\nbefore,” said Alice,) and round the neck of the bottle was a paper\nlabel, with the words “DRINK ME,” beautifully printed on it in large\nletters.\n\nIt was all very well to say “Drink me,” but the wise little Alice was\nnot going to do _that_ in a hurry. “No, I’ll look first,” she said,\n“and see whether it’s marked ‘_poison_’ or not”; for she had read\nseveral nice little histories about children who had got burnt, and\neaten up by wild beasts and other unpleasant things, all because they\n_would_ not remember the simple rules their friends had taught them:\nsuch as, that a red-hot poker will burn you if you hold it too long;\nand that if you cut your finger _very_ deeply with a knife, it usually\nbleeds; and she had never forgotten that, if you drink much from a\nbottle marked “poison,” it is almost certain to disagree with you,\nsooner or later.\n\nHowever, this bottle was _not_ marked “poison,” so Alice ventured to\ntaste it, and finding it very nice, (it had, in fact, a sort of mixed\nflavour of cherry-tart, custard, pine-apple, roast turkey, toffee, and\nhot buttered toast,) she very soon finished it off."
## [2] NA
## [3] NA
docs <- docs %>%
tm_map(removePunctuation) %>%
tm_map(content_transformer(tolower)) %>%
tm_map(removeNumbers) %>%
tm_map(removeWords, stopwords("en")) %>%
tm_map(stripWhitespace) %>%
tm_map(stemDocument)
data <- DocumentTermMatrix(docs, control=list(weighting=weightTfIdf))
## Warning in TermDocumentMatrix.SimpleCorpus(x, control): custom functions are
## ignored
data
## <<DocumentTermMatrix (documents: 1, terms: 442)>>
## Non-/sparse entries: 0/442
## Sparsity : 100%
## Maximal term length: 15
## Weighting : term frequency - inverse document frequency (normalized) (tf-idf)
findFreqTerms(data,0.7)
## character(0)
findAssocs(data,"race",0.5)
## $race
## numeric(0)
newData <- removeSparseTerms(data,0.7)
newData
## <<DocumentTermMatrix (documents: 1, terms: 0)>>
## Non-/sparse entries: 0/0
## Sparsity : 100%
## Maximal term length: 0
## Weighting : term frequency - inverse document frequency (normalized) (tf-idf)
library(tibble)
a =as_tibble(as.matrix(newData))
##View(a)