Removing backslashes

# remove backslash
removebackslash <- function(x) gsub('b"|b\'|\\\\|\\"', "", x)
myCorpus <- tm_map(myCorpus,content_transformer(removebackslash))

Removing URLs

#remove URLS
removeURL <- function(x) gsub("http[^[:space:]]*", "", x)
myCorpus <- tm_map(myCorpus, content_transformer(removeURL))
myCorpus[["50"]][["content"]]

## [1] "The Other Man On The Podium' Dutch non-profit reproductive health organization sails ship around the world, anchoring in international waters to provide abortions to women in countries where abortion is banned' Grand Ayatollah Issues Decree Calling Muslims to Defend Iraq's Christians Marx Das Kapital sales soar among young Germans ' A man in England killed his wife because she changed her Facebook relationship status to single' Georgia Used Cluster Bombs in August War, Too' Arctic Temperatures Break All-Time Recorded High' Reddit, please send help, (UK) politicians insane. Apparently not monitoring mobile and web records would be giving a licence to terrorists to kill people' Wow. Secret coded messages are being embedded into child pornographic images, and paedophile websites are being exploited as a secure way of passing information between terrorists.' England to run out of honey before Christmas after catastrophic honeybee decline '  Iran to stop executing youths' China watches over internet caf customers in web crackdown ' China's media freedoms are reduced. New measures include cameras in internet cafes and pictures taken of all users' Bali bombing - new suspects' Hindu American Foundation petitions the NY Times to focus as much on the activities of Christian missionaries in India as it does on anti-Christian violence.' A quick overview of how Islamic terror organizations get their funding.' Last Titantic Survivor to Auction Off Mementos to Pay for Nursing Home - We can do better!' Hungary Gets $6.7B Loan to Avert Meltdown ' Sao Paolo : Hundreds of black-clad military police fired teargas, stun grenades and rubber bullets at striking civilian officers seeking a 15 percent pay raise' Austrailian historian arrested for Holocaust denial' Defense Secretary Gates said he is prepared for reconciliation with Taliban as part of the political outcome in Afghanistan' Is Switzerland the next Iceland?  Switzerland was forced to take emergency measures yesterday to shore up its two biggest lenders to prevent a collapse in confidence in the country's banking system.' Police battle police in Sao Paulo' 25 more civilians killed in NATO air strike in Afghanistan: villagers' The West Is at a Loss in Afghanistan'"

Removing Numbers

#remove numbers 
myCorpus <- tm_map(myCorpus,content_transformer(removeNumbers))
myCorpus[["194"]][["content"]]

## [1] " Unsung hero : The only reason we know anything about all those claims for light bulbs and moat cleaning is that campaigning journalist Heather Brooke has spent the last five years fighting tooth and nail for MPs to come clean about their expenses ...' Noticed an unusual, strong burst of pro-israel comments? Here's why... The world elite meets again for Bilderberg: Nothing. Nothing is happening here. Nothing to report. There's none. Not a single member of the mainstream press.'  Boycott Israel campaign starts to bite' If you are watching this message, it is because I was assassinated by President Alvaro Colom...' Muslim famous for predicting a totally Islamified Britain, has resigned in disgrace after being revealed as the biggest swindler of them all, ripping off taxpayers more than any other single member of parliament.' Australian immigration authorities told a detainee they were taking his daughter shopping before they sent her back to Iran ' Guatemala arrests Twitter user for inciting financial panic' Our man at Bilderberg: Close, but still no cabal' Our man at Bilderberg: in pursuit of the world's most powerful cabal<br /> 'Slumdog' (Salim) child star's home torn down Author Dan Brown feels the wrath of Catholics' Sweden wants to force ISPs to save user data' German government has predicted the economy will shrink % this year' Bishop of Baghdad worried that Christianity would become extinct in the Middle East' Piratebay DDo$ attack site ( has been reported to the police' Thousands flee Pakistan fighting' Polica Nacional del Per: Gay Sex Tarnishes the Badge' Shadowy Bilderberg group meet in Greece--and here's their address Vietnam veteran on spiritual quest lands Aung San Suu Kyi in prison ' Avigdor Lieberman - To ban Israeli Arabs from marking the anniversary of the Nakba' Paul Craig Roberts on the Global Anti-Semitism Review Act' The rich, shadowy Bilderberg group' The World's Best Illusion: The Secret of the Curve Ball Defining 'Success' Down"

Removing punctuations

# remove punctuations
removeNumPunct <- function(x) gsub("([<>])|[[:punct:]]", " ", x)
myCorpus <- tm_map(myCorpus, content_transformer(removeNumPunct))
myCorpus[["50"]][["content"]]

## [1] "The Other Man On The Podium  Dutch non profit reproductive health organization sails ship around the world  anchoring in international waters to provide abortions to women in countries where abortion is banned  Grand Ayatollah Issues Decree Calling Muslims to Defend Iraq s Christians Marx Das Kapital sales soar among young Germans   A man in England killed his wife because she changed her Facebook relationship status to single  Georgia Used Cluster Bombs in August War  Too  Arctic Temperatures Break All Time Recorded High  Reddit  please send help   UK  politicians insane  Apparently not monitoring mobile and web records would be giving a licence to terrorists to kill people  Wow  Secret coded messages are being embedded into child pornographic images  and paedophile websites are being exploited as a secure way of passing information between terrorists   England to run out of honey before Christmas after catastrophic honeybee decline    Iran to stop executing youths  China watches over internet caf customers in web crackdown   China s media freedoms are reduced  New measures include cameras in internet cafes and pictures taken of all users  Bali bombing   new suspects  Hindu American Foundation petitions the NY Times to focus as much on the activities of Christian missionaries in India as it does on anti Christian violence   A quick overview of how Islamic terror organizations get their funding   Last Titantic Survivor to Auction Off Mementos to Pay for Nursing Home   We can do better   Hungary Gets   B Loan to Avert Meltdown   Sao Paolo   Hundreds of black clad military police fired teargas  stun grenades and rubber bullets at striking civilian officers seeking a  percent pay raise  Austrailian historian arrested for Holocaust denial  Defense Secretary Gates said he is prepared for reconciliation with Taliban as part of the political outcome in Afghanistan  Is Switzerland the next Iceland   Switzerland was forced to take emergency measures yesterday to shore up its two biggest lenders to prevent a collapse in confidence in the country s banking system   Police battle police in Sao Paulo   more civilians killed in NATO air strike in Afghanistan  villagers  The West Is at a Loss in Afghanistan "

Converting to lower case

myCorpus <- tm_map(myCorpus, content_transformer(tolower))
myCorpus[["50"]][["content"]]

## [1] "the other man on the podium  dutch non profit reproductive health organization sails ship around the world  anchoring in international waters to provide abortions to women in countries where abortion is banned  grand ayatollah issues decree calling muslims to defend iraq s christians marx das kapital sales soar among young germans   a man in england killed his wife because she changed her facebook relationship status to single  georgia used cluster bombs in august war  too  arctic temperatures break all time recorded high  reddit  please send help   uk  politicians insane  apparently not monitoring mobile and web records would be giving a licence to terrorists to kill people  wow  secret coded messages are being embedded into child pornographic images  and paedophile websites are being exploited as a secure way of passing information between terrorists   england to run out of honey before christmas after catastrophic honeybee decline    iran to stop executing youths  china watches over internet caf customers in web crackdown   china s media freedoms are reduced  new measures include cameras in internet cafes and pictures taken of all users  bali bombing   new suspects  hindu american foundation petitions the ny times to focus as much on the activities of christian missionaries in india as it does on anti christian violence   a quick overview of how islamic terror organizations get their funding   last titantic survivor to auction off mementos to pay for nursing home   we can do better   hungary gets   b loan to avert meltdown   sao paolo   hundreds of black clad military police fired teargas  stun grenades and rubber bullets at striking civilian officers seeking a  percent pay raise  austrailian historian arrested for holocaust denial  defense secretary gates said he is prepared for reconciliation with taliban as part of the political outcome in afghanistan  is switzerland the next iceland   switzerland was forced to take emergency measures yesterday to shore up its two biggest lenders to prevent a collapse in confidence in the country s banking system   police battle police in sao paulo   more civilians killed in nato air strike in afghanistan  villagers  the west is at a loss in afghanistan "

Removing stopwords

# smart stopword list - http://www.lextek.com/manuals/onix/stopwords2.html
myStopwords <- c(stopwords(kind = 'SMART'))
myCorpus <- tm_map(myCorpus, removeWords, myStopwords)
myCorpus[["50"]][["content"]]

## [1] "  man   podium  dutch  profit reproductive health organization sails ship   world  anchoring  international waters  provide abortions  women  countries  abortion  banned  grand ayatollah issues decree calling muslims  defend iraq  christians marx das kapital sales soar  young germans    man  england killed  wife   changed  facebook relationship status  single  georgia  cluster bombs  august war    arctic temperatures break  time recorded high  reddit   send    uk  politicians insane  apparently  monitoring mobile  web records   giving  licence  terrorists  kill people  wow  secret coded messages   embedded  child pornographic images   paedophile websites   exploited   secure   passing information  terrorists   england  run   honey  christmas  catastrophic honeybee decline    iran  stop executing youths  china watches  internet caf customers  web crackdown   china  media freedoms  reduced   measures include cameras  internet cafes  pictures    users  bali bombing    suspects  hindu american foundation petitions  ny times  focus     activities  christian missionaries  india     anti christian violence    quick overview   islamic terror organizations   funding    titantic survivor  auction  mementos  pay  nursing home         hungary     loan  avert meltdown   sao paolo   hundreds  black clad military police fired teargas  stun grenades  rubber bullets  striking civilian officers seeking   percent pay raise  austrailian historian arrested  holocaust denial  defense secretary gates    prepared  reconciliation  taliban  part   political outcome  afghanistan   switzerland   iceland   switzerland  forced   emergency measures yesterday  shore    biggest lenders  prevent  collapse  confidence   country  banking system   police battle police  sao paulo    civilians killed  nato air strike  afghanistan  villagers   west    loss  afghanistan "

Removing extra whitespaces

# remove extra whitespaces
myCorpus <- tm_map(myCorpus, stripWhitespace)
myCorpus[["50"]][["content"]]

## [1] " man podium dutch profit reproductive health organization sails ship world anchoring international waters provide abortions women countries abortion banned grand ayatollah issues decree calling muslims defend iraq christians marx das kapital sales soar young germans man england killed wife changed facebook relationship status single georgia cluster bombs august war arctic temperatures break time recorded high reddit send uk politicians insane apparently monitoring mobile web records giving licence terrorists kill people wow secret coded messages embedded child pornographic images paedophile websites exploited secure passing information terrorists england run honey christmas catastrophic honeybee decline iran stop executing youths china watches internet caf customers web crackdown china media freedoms reduced measures include cameras internet cafes pictures users bali bombing suspects hindu american foundation petitions ny times focus activities christian missionaries india anti christian violence quick overview islamic terror organizations funding titantic survivor auction mementos pay nursing home hungary loan avert meltdown sao paolo hundreds black clad military police fired teargas stun grenades rubber bullets striking civilian officers seeking percent pay raise austrailian historian arrested holocaust denial defense secretary gates prepared reconciliation taliban part political outcome afghanistan switzerland iceland switzerland forced emergency measures yesterday shore biggest lenders prevent collapse confidence country banking system police battle police sao paulo civilians killed nato air strike afghanistan villagers west loss afghanistan "

Convert vectors to character for tokenisation and stemming

# onvert vectors to character for tokenisation and stemming
char_corpus <- convert.tm.to.character(myCorpus)

Writing Data into Excel Format

# write_xlsx(data.frame(char_corpus),"Reddit News headlines - Cleaned Corpus.xlsx")