Text Mining
Genaro Rodríguez Alcántara - A00833172
2024-02-26
Teoría
La minería de texto (TM) es el proceso de extraer información útil, patrones o conocimiento de textos no estructurados.
Consta de 3 etapas: 1. Obtener datos: El reconocimeinto óptico de caracteres (OCR) es una tecnología que permite convertir imágenes de texto en el texto editable. También es conocido como extracción de texto de imágenes. 2. Explorar datos: Representación gráfica o visual de los datos para su interpretación. Los métodos más comunes son el Análisis de Sentimientos, la Nube de Palabras y el Topic Modeling. 3. Análisis Predictivo: Son las técnicas y modelos estadísticos para predecir resultados futuros. Los modelos más usados son el Random Forest, Redes Neuronales y Regresiones.
Instalar paquetes y llamar librerías
#install.packages("tidyverse") # Data wrangling
library(tidyverse)## Warning: package 'ggplot2' was built under R version 4.3.1## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr 1.1.2 ✔ readr 2.1.4
## ✔ forcats 1.0.0 ✔ stringr 1.5.0
## ✔ ggplot2 3.4.4 ✔ tibble 3.2.1
## ✔ lubridate 1.9.2 ✔ tidyr 1.3.0
## ✔ purrr 1.0.2
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag() masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors#install.packages("tesseract") # OCR
library(tesseract)## Warning: package 'tesseract' was built under R version 4.3.1#install.packages("magick") # PNG
library(magick)## Warning: package 'magick' was built under R version 4.3.1## Linking to ImageMagick 6.9.12.93
## Enabled features: cairo, fontconfig, freetype, heic, lcms, pango, raw, rsvg, webp
## Disabled features: fftw, ghostscript, x11#install.packages("officer") # Office (word)
library(officer)## Warning: package 'officer' was built under R version 4.3.1#install.packages("pdftools") # PDF
library(pdftools)## Warning: package 'pdftools' was built under R version 4.3.1## Using poppler version 23.04.0#install.packages("purrr") # Para la función map
library(purrr)
#install.packages("tm") # Text Mining
library(tm)## Loading required package: NLP
##
## Attaching package: 'NLP'
##
## The following object is masked from 'package:ggplot2':
##
## annotate#install.packages("RColorBrewer") # Colores
library(RColorBrewer)
#install.packages("wordcloud") # Nubre de palabras
library(wordcloud)
#install.packages("topicmodels") # Modelos de temas
library(topicmodels)## Warning: package 'topicmodels' was built under R version 4.3.1library(ggplot2)1. Obtener Datos mediante OCR
imagen1 <- image_read("/Users/genarorodriguezalcantara/Desktop/Tec/AI - Concentración/Módulo 2 - Machine Learning/BD/imagen1.PNG")
texto1 <- ocr(imagen1)
texto1## [1] "Linear regression with one variable x is also known as univariate linear regression\nor simple linear regression. Simple linear regression is used to predict a single\noutput from a single input. This is an example of supervised learning, which means\nthat the data is labeled, i.e., the output values are known in the training data. Let us\nfit a line through the data using simple linear regression as shown in Fig. 4.1.\n"doc1 <- read_docx() # Crea un documento en blanco
doc1 <- doc1 %>% body_add_par(texto1, style = "Normal")
print(doc1, target = "texto1.docx")Imagen a WORD
imagen2 <- image_read("/Users/genarorodriguezalcantara/Desktop/Tec/AI - Concentración/Módulo 2 - Machine Learning/BD/imagen2.PNG")
tesseract_download("spa")## [1] "/Users/genarorodriguezalcantara/Library/Application Support/tesseract5/tessdata/spa.traineddata"texto2 <- ocr(imagen2, engine = tesseract("spa"))
texto2## [1] "Un importante, y quizá controversial, asunto político es el que se refiere al efecto del salario mínimo sobre\nlas tasas de desempleo en diversos grupos de trabajadores. Aunque este problema puede ser estudiado con\ndiversos tipos de datos (corte transversal, series de tiempo o datos de panel), suelen usarse las series de\ntiempo para observar los efectos agregados. En la tabla 1.3 se presenta un ejemplo de una base de datos\nde series de tiempo sobre tasas de desempleo y salarios mínimos.\n"doc2 <- read_docx() # Crea un documento de word en blanco
doc2 <- doc2 %>% body_add_par(texto2, style = "Normal")
#print(doc2, "texto2.docx")De PDF a Texto en WORD
pdf1 <- pdf_convert ("/Users/genarorodriguezalcantara/Desktop/Tec/AI - Concentración/Módulo 2 - Machine Learning/BD/pdf1.pdf", dpi = 600) %>% map(ocr)## Converting page 1 to pdf1_1.png... done!
## Converting page 2 to pdf1_2.png... done!
## Converting page 3 to pdf1_3.png... done!
## Converting page 4 to pdf1_4.png... done!
## Converting page 5 to pdf1_5.png... done!
## Converting page 6 to pdf1_6.png... done!
## Converting page 7 to pdf1_7.png... done!
## Converting page 8 to pdf1_8.png... done!Actividad 1. Novela “IT”
pdf2 <- pdf_convert("/Users/genarorodriguezalcantara/Desktop/Tec/AI - Concentración/Módulo 2 - Machine Learning/BD/eso3.pdf", dpi = 600) %>% map(ocr)## Converting page 1 to eso3_1.png... done!
## Converting page 2 to eso3_2.png... done!
## Converting page 3 to eso3_3.png... done!# Agregar el texto extraído del primer archivo PNG
eso1 <- image_read("/Users/genarorodriguezalcantara/Desktop/Tec/AI - Concentración/Módulo 2 - Machine Learning/Code/eso3_1.png")
eso2 <- image_read("/Users/genarorodriguezalcantara/Desktop/Tec/AI - Concentración/Módulo 2 - Machine Learning/Code/eso3_2.png")
eso3 <- image_read("/Users/genarorodriguezalcantara/Desktop/Tec/AI - Concentración/Módulo 2 - Machine Learning/Code/eso3_3.png")
tesseract_download("spa")## [1] "/Users/genarorodriguezalcantara/Library/Application Support/tesseract5/tessdata/spa.traineddata"eso_1 <- ocr(eso1, engine = tesseract("spa"))
eso_2 <- ocr(eso2, engine = tesseract("spa"))
eso_3 <- ocr(eso3, engine = tesseract("spa"))
doc_it <- read_docx()
doc_it <- doc_it %>% body_add_par(eso_1, style = "Normal") %>% body_add_par(eso_2, style = "Normal") %>% body_add_par(eso_3, style = "Normal")
# Guardar el documento de Word
print(doc_it, target = "textos_extraidos.docx")2. Exploración de Datos
Análisis de Frecuencias
text <- readLines("http://www.sthda.com/sthda/RDoc/example-files/martin-luther-king-i-have-a-dream-speech.txt")
corpus <- Corpus(VectorSource(text))
corpus <- tm_map(corpus, content_transformer(tolower))## Warning in tm_map.SimpleCorpus(corpus, content_transformer(tolower)):
## transformation drops documentscorpus <- tm_map(corpus, removePunctuation)## Warning in tm_map.SimpleCorpus(corpus, removePunctuation): transformation drops
## documentscorpus <- tm_map(corpus, removeNumbers)## Warning in tm_map.SimpleCorpus(corpus, removeNumbers): transformation drops
## documentscorpus <- tm_map(corpus, removeWords, stopwords("en"))## Warning in tm_map.SimpleCorpus(corpus, removeWords, stopwords("en")):
## transformation drops documents# corpus <- tm_map(corpus, removeWords, c("dream","will")) # Elimina palabras puntuales
tdm <- TermDocumentMatrix(corpus)
m <- as.matrix(tdm) # Cuentas las veces que aparece cada palabra por renglón
frecuencia <- sort(rowSums(m), decreasing = TRUE) # Cuenta la frecuencia de cada palabra en el texto completo
frecuencia_df <- data.frame(word=names(frecuencia), freq=frecuencia) # Convierte la frecuencia a data frame.
ggplot(head(frecuencia_df, 20), aes(x = word, y = freq)) +
geom_bar(stat = "identity") +
labs(title = "TOP 10 palabras más frecuenctes", x="Palabra", y="Frecuencia", subtitle = "Discurso 'I Have a Dream' & M. L. King") +
ylim(0,20)
inspect(corpus)## <<SimpleCorpus>>
## Metadata: corpus specific: 1, document level (indexed): 0
## Content: documents: 46
##
## [1]
## [2] even though face difficulties today tomorrow still dream dream deeply rooted american dream
## [3]
## [4] dream one day nation will rise live true meaning creed
## [5]
## [6] hold truths selfevident men created equal
## [7]
## [8] dream one day red hills georgia sons former slaves sons former slave owners will able sit together table brotherhood
## [9]
## [10] dream one day even state mississippi state sweltering heat injustice sweltering heat oppression will transformed oasis freedom justice
## [11]
## [12] dream four little children will one day live nation will judged color skin content character
## [13]
## [14] dream today
## [15]
## [16] dream one day alabama vicious racists governor lips dripping words interposition nullification one day right alabama little black boys black girls will able join hands little white boys white girls sisters brothers
## [17]
## [18] dream today
## [19]
## [20] dream one day every valley shall exalted every hill mountain shall made low rough places will made plain crooked places will made straight glory lord shall revealed flesh shall see together
## [21]
## [22] hope faith go back south
## [23]
## [24] faith will able hew mountain despair stone hope faith will able transform jangling discords nation beautiful symphony brotherhood faith will able work together pray together struggle together go jail together stand freedom together knowing will free one day
## [25]
## [26] will day will day god s children will able sing new meaning
## [27]
## [28] country tis thee sweet land liberty thee sing
## [29] land fathers died land pilgrim s pride
## [30] every mountainside let freedom ring
## [31] america great nation must become true
## [32] let freedom ring prodigious hilltops new hampshire
## [33] let freedom ring mighty mountains new york
## [34] let freedom ring heightening alleghenies pennsylvania
## [35] let freedom ring snowcapped rockies colorado
## [36] let freedom ring curvaceous slopes california
## [37]
## [38]
## [39] let freedom ring stone mountain georgia
## [40] let freedom ring lookout mountain tennessee
## [41] let freedom ring every hill molehill mississippi
## [42] every mountainside let freedom ring
## [43] happens allow freedom ring let ring every village every hamlet every state every city will able speed day god s children black men white men jews gentiles protestants catholics will able join hands sing words old negro spiritual
## [44] free last free last
## [45]
## [46] thank god almighty free lastNube de Palabras
# El procesamiento de datos antes de la nube de palabras es igual que en el Análisis de Frecuencias, desde importar el texto hasta frecuencia_df
set.seed(123)
wordcloud(words = frecuencia_df$word, freq = frecuencia_df$freq, min.freq = 1, random.order = FALSE, colors = brewer.pal(8, "RdPu"))
Ejercicio 2. Novela IT
text2 <- readLines("/Users/genarorodriguezalcantara/Desktop/Tec/AI - Concentración/Módulo 2 - Machine Learning/Code/textos_extraidos.txt", encoding = "UTF-8")
# Convertir el texto a UTF-8 si es necesario
text2 <- iconv(text2, to = "UTF-8", sub = "byte")
# Crear un corpus con el texto
corpus <- Corpus(VectorSource(text2))
# Pre-procesamiento del texto
corpus <- tm_map(corpus, content_transformer(tolower))## Warning in tm_map.SimpleCorpus(corpus, content_transformer(tolower)):
## transformation drops documentscorpus <- tm_map(corpus, removePunctuation)## Warning in tm_map.SimpleCorpus(corpus, removePunctuation): transformation drops
## documentscorpus <- tm_map(corpus, removeNumbers)## Warning in tm_map.SimpleCorpus(corpus, removeNumbers): transformation drops
## documentscorpus <- tm_map(corpus, removeWords, stopwords("spanish"))## Warning in tm_map.SimpleCorpus(corpus, removeWords, stopwords("spanish")):
## transformation drops documents# Crear una matriz de términos por documento
tdm <- TermDocumentMatrix(corpus)
# Convertir a matriz y calcular frecuencias
m <- as.matrix(tdm)
frecuencia2 <- sort(rowSums(m), decreasing = TRUE)
frecuencia_df2 <- data.frame(word = names(frecuencia2), freq = frecuencia2)ggplot(head(frecuencia_df2, 10), aes(x = word, y = freq)) +
geom_bar(stat = "identity") +
labs(title = "TOP 10 palabras más frecuentes", x = "Palabra", y = "Frecuencia", subtitle = "Análisis del archivo textos_extraidos") +
ylim(0, max(frecuencia_df2$freq[1:10]) + 10) # Ajustar el límite superior para mejor visualización
LS0tCnRpdGxlOiAiVGV4dCBNaW5pbmciCmF1dGhvcjogIkdlbmFybyBSb2Ryw61ndWV6IEFsY8OhbnRhcmEgLSBBMDA4MzMxNzIiCmRhdGU6ICIyMDI0LTAyLTI2IgpvdXRwdXQ6IAogIGh0bWxfZG9jdW1lbnQ6CiAgICB0b2M6IFRSVUUKICAgIHRvY19mbG9hdDogVFJVRQogICAgY29kZV9kb3dubG9hZDogVFJVRQotLS0KCiFbXSgvVXNlcnMvZ2VuYXJvcm9kcmlndWV6YWxjYW50YXJhL0Rlc2t0b3AvVGVjL0FJIC0gQ29uY2VudHJhY2lvzIFuL01vzIFkdWxvIDIgLSBNYWNoaW5lIExlYXJuaW5nL0JEL3RleHQtZGF0YS1taW5pbmcucG5nKQoKIyA8c3BhbiBzdHlsZT0iY29sb3I6IHJlZDsgIj5UZW9yw61hPC9zcGFuPgoKTGEgKiptaW5lcsOtYSBkZSB0ZXh0byoqIChUTSkgZXMgZWwgcHJvY2VzbyBkZSBleHRyYWVyIGluZm9ybWFjacOzbiDDunRpbCwgcGF0cm9uZXMgbyBjb25vY2ltaWVudG8gZGUgdGV4dG9zIG5vIGVzdHJ1Y3R1cmFkb3MuCgpDb25zdGEgZGUgMyBldGFwYXM6CjEuIE9idGVuZXIgZGF0b3M6IEVsICoqcmVjb25vY2ltZWludG8gw7NwdGljbyBkZSBjYXJhY3RlcmVzIChPQ1IpKiogZXMgdW5hIHRlY25vbG9nw61hIHF1ZSBwZXJtaXRlIGNvbnZlcnRpciBpbcOhZ2VuZXMgZGUgdGV4dG8gZW4gZWwgdGV4dG8gZWRpdGFibGUuIFRhbWJpw6luIGVzIGNvbm9jaWRvIGNvbW8gKipleHRyYWNjacOzbiBkZSB0ZXh0byBkZSBpbcOhZ2VuZXMqKi4KMi4gRXhwbG9yYXIgZGF0b3M6IFJlcHJlc2VudGFjacOzbiBncsOhZmljYSBvIHZpc3VhbCBkZSBsb3MgZGF0b3MgcGFyYSBzdSBpbnRlcnByZXRhY2nDs24uIExvcyBtw6l0b2RvcyBtw6FzIGNvbXVuZXMgc29uIGVsIEFuw6FsaXNpcyBkZSBTZW50aW1pZW50b3MsIGxhIE51YmUgZGUgUGFsYWJyYXMgeSBlbCBUb3BpYyBNb2RlbGluZy4KMy4gQW7DoWxpc2lzIFByZWRpY3Rpdm86IFNvbiBsYXMgdMOpY25pY2FzIHkgbW9kZWxvcyBlc3RhZMOtc3RpY29zIHBhcmEgcHJlZGVjaXIgcmVzdWx0YWRvcyBmdXR1cm9zLiBMb3MgbW9kZWxvcyBtw6FzIHVzYWRvcyBzb24gZWwgUmFuZG9tIEZvcmVzdCwgUmVkZXMgTmV1cm9uYWxlcyB5IFJlZ3Jlc2lvbmVzLgoKIyA8c3BhbiBzdHlsZT0iY29sb3I6IHJlZDsgIj5JbnN0YWxhciBwYXF1ZXRlcyB5IGxsYW1hciBsaWJyZXLDrWFzPC9zcGFuPgpgYGB7cn0KI2luc3RhbGwucGFja2FnZXMoInRpZHl2ZXJzZSIpICMgRGF0YSB3cmFuZ2xpbmcKbGlicmFyeSh0aWR5dmVyc2UpCiNpbnN0YWxsLnBhY2thZ2VzKCJ0ZXNzZXJhY3QiKSAjIE9DUgpsaWJyYXJ5KHRlc3NlcmFjdCkKI2luc3RhbGwucGFja2FnZXMoIm1hZ2ljayIpICMgUE5HCmxpYnJhcnkobWFnaWNrKQojaW5zdGFsbC5wYWNrYWdlcygib2ZmaWNlciIpICMgT2ZmaWNlICh3b3JkKQpsaWJyYXJ5KG9mZmljZXIpCiNpbnN0YWxsLnBhY2thZ2VzKCJwZGZ0b29scyIpICMgUERGCmxpYnJhcnkocGRmdG9vbHMpCiNpbnN0YWxsLnBhY2thZ2VzKCJwdXJyciIpICMgUGFyYSBsYSBmdW5jacOzbiBtYXAKbGlicmFyeShwdXJycikKI2luc3RhbGwucGFja2FnZXMoInRtIikgIyBUZXh0IE1pbmluZwpsaWJyYXJ5KHRtKQojaW5zdGFsbC5wYWNrYWdlcygiUkNvbG9yQnJld2VyIikgIyBDb2xvcmVzCmxpYnJhcnkoUkNvbG9yQnJld2VyKQojaW5zdGFsbC5wYWNrYWdlcygid29yZGNsb3VkIikgIyBOdWJyZSBkZSBwYWxhYnJhcwpsaWJyYXJ5KHdvcmRjbG91ZCkKI2luc3RhbGwucGFja2FnZXMoInRvcGljbW9kZWxzIikgIyBNb2RlbG9zIGRlIHRlbWFzCmxpYnJhcnkodG9waWNtb2RlbHMpCmxpYnJhcnkoZ2dwbG90MikKYGBgCgoKIyA8c3BhbiBzdHlsZT0iY29sb3I6IHJlZDsgIj4xLiBPYnRlbmVyIERhdG9zIG1lZGlhbnRlIE9DUjwvc3Bhbj4KYGBge3J9CmltYWdlbjEgPC0gaW1hZ2VfcmVhZCgiL1VzZXJzL2dlbmFyb3JvZHJpZ3VlemFsY2FudGFyYS9EZXNrdG9wL1RlYy9BSSAtIENvbmNlbnRyYWNpb8yBbi9Nb8yBZHVsbyAyIC0gTWFjaGluZSBMZWFybmluZy9CRC9pbWFnZW4xLlBORyIpCnRleHRvMSA8LSBvY3IoaW1hZ2VuMSkKdGV4dG8xCmRvYzEgPC0gcmVhZF9kb2N4KCkgIyBDcmVhIHVuIGRvY3VtZW50byBlbiBibGFuY28KZG9jMSA8LSBkb2MxICU+JSBib2R5X2FkZF9wYXIodGV4dG8xLCBzdHlsZSA9ICJOb3JtYWwiKQpwcmludChkb2MxLCB0YXJnZXQgPSAidGV4dG8xLmRvY3giKQoKYGBgCgoKIyA8c3BhbiBzdHlsZT0iY29sb3I6IHJlZDsgIj5JbWFnZW4gYSBXT1JEPC9zcGFuPgpbQ29uc3VsdGFyIGlkaW9tYXMgZGlzcG9uaWJsZXNdKGh0dHBzOi8vdGVzc2VyYWN0LW9jci5naXRodWIuaW8vdGVzc2RvYy90ZXNzMy9EYXRhLUZpbGVzLmh0bWwpCmBgYHtyfQppbWFnZW4yIDwtIGltYWdlX3JlYWQoIi9Vc2Vycy9nZW5hcm9yb2RyaWd1ZXphbGNhbnRhcmEvRGVza3RvcC9UZWMvQUkgLSBDb25jZW50cmFjaW/MgW4vTW/MgWR1bG8gMiAtIE1hY2hpbmUgTGVhcm5pbmcvQkQvaW1hZ2VuMi5QTkciKQp0ZXNzZXJhY3RfZG93bmxvYWQoInNwYSIpCnRleHRvMiA8LSBvY3IoaW1hZ2VuMiwgZW5naW5lID0gdGVzc2VyYWN0KCJzcGEiKSkKdGV4dG8yCmRvYzIgPC0gcmVhZF9kb2N4KCkgIyBDcmVhIHVuIGRvY3VtZW50byBkZSB3b3JkIGVuIGJsYW5jbwpkb2MyIDwtIGRvYzIgJT4lIGJvZHlfYWRkX3Bhcih0ZXh0bzIsIHN0eWxlID0gIk5vcm1hbCIpCiNwcmludChkb2MyLCAidGV4dG8yLmRvY3giKQpgYGAKCiMgPHNwYW4gc3R5bGU9ImNvbG9yOiByZWQ7ICI+RGUgUERGIGEgVGV4dG8gZW4gV09SRDwvc3Bhbj4KYGBge3J9CnBkZjEgPC0gcGRmX2NvbnZlcnQgKCIvVXNlcnMvZ2VuYXJvcm9kcmlndWV6YWxjYW50YXJhL0Rlc2t0b3AvVGVjL0FJIC0gQ29uY2VudHJhY2lvzIFuL01vzIFkdWxvIDIgLSBNYWNoaW5lIExlYXJuaW5nL0JEL3BkZjEucGRmIiwgZHBpID0gNjAwKSAlPiUgbWFwKG9jcikKYGBgCgojIyA8c3BhbiBzdHlsZT0iY29sb3I6IHJlZDsgIj5BY3RpdmlkYWQgMS4gTm92ZWxhICJJVCI8L3NwYW4+CmBgYHtyfQpwZGYyIDwtIHBkZl9jb252ZXJ0KCIvVXNlcnMvZ2VuYXJvcm9kcmlndWV6YWxjYW50YXJhL0Rlc2t0b3AvVGVjL0FJIC0gQ29uY2VudHJhY2lvzIFuL01vzIFkdWxvIDIgLSBNYWNoaW5lIExlYXJuaW5nL0JEL2VzbzMucGRmIiwgZHBpID0gNjAwKSAlPiUgbWFwKG9jcikKYGBgCmBgYHtyfQoKCiMgQWdyZWdhciBlbCB0ZXh0byBleHRyYcOtZG8gZGVsIHByaW1lciBhcmNoaXZvIFBORwplc28xIDwtIGltYWdlX3JlYWQoIi9Vc2Vycy9nZW5hcm9yb2RyaWd1ZXphbGNhbnRhcmEvRGVza3RvcC9UZWMvQUkgLSBDb25jZW50cmFjaW/MgW4vTW/MgWR1bG8gMiAtIE1hY2hpbmUgTGVhcm5pbmcvQ29kZS9lc28zXzEucG5nIikKZXNvMiA8LSBpbWFnZV9yZWFkKCIvVXNlcnMvZ2VuYXJvcm9kcmlndWV6YWxjYW50YXJhL0Rlc2t0b3AvVGVjL0FJIC0gQ29uY2VudHJhY2lvzIFuL01vzIFkdWxvIDIgLSBNYWNoaW5lIExlYXJuaW5nL0NvZGUvZXNvM18yLnBuZyIpCmVzbzMgPC0gaW1hZ2VfcmVhZCgiL1VzZXJzL2dlbmFyb3JvZHJpZ3VlemFsY2FudGFyYS9EZXNrdG9wL1RlYy9BSSAtIENvbmNlbnRyYWNpb8yBbi9Nb8yBZHVsbyAyIC0gTWFjaGluZSBMZWFybmluZy9Db2RlL2VzbzNfMy5wbmciKQp0ZXNzZXJhY3RfZG93bmxvYWQoInNwYSIpCmVzb18xIDwtIG9jcihlc28xLCBlbmdpbmUgPSB0ZXNzZXJhY3QoInNwYSIpKQplc29fMiA8LSBvY3IoZXNvMiwgZW5naW5lID0gdGVzc2VyYWN0KCJzcGEiKSkKZXNvXzMgPC0gb2NyKGVzbzMsIGVuZ2luZSA9IHRlc3NlcmFjdCgic3BhIikpCgpkb2NfaXQgPC0gcmVhZF9kb2N4KCkKZG9jX2l0IDwtIGRvY19pdCAlPiUgYm9keV9hZGRfcGFyKGVzb18xLCBzdHlsZSA9ICJOb3JtYWwiKSAlPiUgYm9keV9hZGRfcGFyKGVzb18yLCBzdHlsZSA9ICJOb3JtYWwiKSAlPiUgYm9keV9hZGRfcGFyKGVzb18zLCBzdHlsZSA9ICJOb3JtYWwiKQoKIyBHdWFyZGFyIGVsIGRvY3VtZW50byBkZSBXb3JkCnByaW50KGRvY19pdCwgdGFyZ2V0ID0gInRleHRvc19leHRyYWlkb3MuZG9jeCIpCmBgYAoKCiMgPHNwYW4gc3R5bGU9ImNvbG9yOiByZWQ7ICI+Mi4gRXhwbG9yYWNpw7NuIGRlIERhdG9zPC9zcGFuPgoKIyMgPHNwYW4gc3R5bGU9ImNvbG9yOiByZWQ7ICI+QW7DoWxpc2lzIGRlIEZyZWN1ZW5jaWFzPC9zcGFuPgpgYGB7cn0KdGV4dCA8LSByZWFkTGluZXMoImh0dHA6Ly93d3cuc3RoZGEuY29tL3N0aGRhL1JEb2MvZXhhbXBsZS1maWxlcy9tYXJ0aW4tbHV0aGVyLWtpbmctaS1oYXZlLWEtZHJlYW0tc3BlZWNoLnR4dCIpCgpjb3JwdXMgPC0gQ29ycHVzKFZlY3RvclNvdXJjZSh0ZXh0KSkKY29ycHVzIDwtIHRtX21hcChjb3JwdXMsIGNvbnRlbnRfdHJhbnNmb3JtZXIodG9sb3dlcikpCmNvcnB1cyA8LSB0bV9tYXAoY29ycHVzLCByZW1vdmVQdW5jdHVhdGlvbikKY29ycHVzIDwtIHRtX21hcChjb3JwdXMsIHJlbW92ZU51bWJlcnMpCmNvcnB1cyA8LSB0bV9tYXAoY29ycHVzLCByZW1vdmVXb3Jkcywgc3RvcHdvcmRzKCJlbiIpKQojIGNvcnB1cyA8LSB0bV9tYXAoY29ycHVzLCByZW1vdmVXb3JkcywgYygiZHJlYW0iLCJ3aWxsIikpICMgRWxpbWluYSBwYWxhYnJhcyBwdW50dWFsZXMKdGRtIDwtIFRlcm1Eb2N1bWVudE1hdHJpeChjb3JwdXMpCm0gPC0gYXMubWF0cml4KHRkbSkgIyBDdWVudGFzIGxhcyB2ZWNlcyBxdWUgYXBhcmVjZSBjYWRhIHBhbGFicmEgcG9yIHJlbmdsw7NuCmZyZWN1ZW5jaWEgPC0gc29ydChyb3dTdW1zKG0pLCBkZWNyZWFzaW5nID0gVFJVRSkgIyBDdWVudGEgbGEgZnJlY3VlbmNpYSBkZSBjYWRhIHBhbGFicmEgZW4gZWwgdGV4dG8gY29tcGxldG8KZnJlY3VlbmNpYV9kZiA8LSBkYXRhLmZyYW1lKHdvcmQ9bmFtZXMoZnJlY3VlbmNpYSksIGZyZXE9ZnJlY3VlbmNpYSkgIyBDb252aWVydGUgbGEgZnJlY3VlbmNpYSBhIGRhdGEgZnJhbWUuCmdncGxvdChoZWFkKGZyZWN1ZW5jaWFfZGYsIDIwKSwgYWVzKHggPSB3b3JkLCB5ID0gZnJlcSkpICsKICBnZW9tX2JhcihzdGF0ID0gImlkZW50aXR5IikgKwogIGxhYnModGl0bGUgPSAiVE9QIDEwIHBhbGFicmFzIG3DoXMgZnJlY3VlbmN0ZXMiLCB4PSJQYWxhYnJhIiwgeT0iRnJlY3VlbmNpYSIsIHN1YnRpdGxlID0gIkRpc2N1cnNvICdJIEhhdmUgYSBEcmVhbScgJiBNLiBMLiBLaW5nIikgKwogIHlsaW0oMCwyMCkKaW5zcGVjdChjb3JwdXMpCmBgYAoKIyMgPHNwYW4gc3R5bGU9ImNvbG9yOiByZWQ7ICI+TnViZSBkZSBQYWxhYnJhczwvc3Bhbj4KYGBge3J9CiMgRWwgcHJvY2VzYW1pZW50byBkZSBkYXRvcyBhbnRlcyBkZSBsYSBudWJlIGRlIHBhbGFicmFzIGVzIGlndWFsIHF1ZSBlbiBlbCBBbsOhbGlzaXMgZGUgRnJlY3VlbmNpYXMsIGRlc2RlIGltcG9ydGFyIGVsIHRleHRvIGhhc3RhIGZyZWN1ZW5jaWFfZGYKc2V0LnNlZWQoMTIzKQp3b3JkY2xvdWQod29yZHMgPSBmcmVjdWVuY2lhX2RmJHdvcmQsIGZyZXEgPSBmcmVjdWVuY2lhX2RmJGZyZXEsIG1pbi5mcmVxID0gMSwgcmFuZG9tLm9yZGVyID0gRkFMU0UsIGNvbG9ycyA9IGJyZXdlci5wYWwoOCwgIlJkUHUiKSkKCgpgYGAKCiMjIDxzcGFuIHN0eWxlPSJjb2xvcjogcmVkOyAiPkVqZXJjaWNpbyAyLiBOb3ZlbGEgSVQ8L3NwYW4+CmBgYHtyfQp0ZXh0MiA8LSByZWFkTGluZXMoIi9Vc2Vycy9nZW5hcm9yb2RyaWd1ZXphbGNhbnRhcmEvRGVza3RvcC9UZWMvQUkgLSBDb25jZW50cmFjaW/MgW4vTW/MgWR1bG8gMiAtIE1hY2hpbmUgTGVhcm5pbmcvQ29kZS90ZXh0b3NfZXh0cmFpZG9zLnR4dCIsIGVuY29kaW5nID0gIlVURi04IikKCiMgQ29udmVydGlyIGVsIHRleHRvIGEgVVRGLTggc2kgZXMgbmVjZXNhcmlvCnRleHQyIDwtIGljb252KHRleHQyLCB0byA9ICJVVEYtOCIsIHN1YiA9ICJieXRlIikKCiMgQ3JlYXIgdW4gY29ycHVzIGNvbiBlbCB0ZXh0bwpjb3JwdXMgPC0gQ29ycHVzKFZlY3RvclNvdXJjZSh0ZXh0MikpCgojIFByZS1wcm9jZXNhbWllbnRvIGRlbCB0ZXh0bwpjb3JwdXMgPC0gdG1fbWFwKGNvcnB1cywgY29udGVudF90cmFuc2Zvcm1lcih0b2xvd2VyKSkKY29ycHVzIDwtIHRtX21hcChjb3JwdXMsIHJlbW92ZVB1bmN0dWF0aW9uKQpjb3JwdXMgPC0gdG1fbWFwKGNvcnB1cywgcmVtb3ZlTnVtYmVycykKY29ycHVzIDwtIHRtX21hcChjb3JwdXMsIHJlbW92ZVdvcmRzLCBzdG9wd29yZHMoInNwYW5pc2giKSkKCiMgQ3JlYXIgdW5hIG1hdHJpeiBkZSB0w6lybWlub3MgcG9yIGRvY3VtZW50bwp0ZG0gPC0gVGVybURvY3VtZW50TWF0cml4KGNvcnB1cykKCiMgQ29udmVydGlyIGEgbWF0cml6IHkgY2FsY3VsYXIgZnJlY3VlbmNpYXMKbSA8LSBhcy5tYXRyaXgodGRtKQpmcmVjdWVuY2lhMiA8LSBzb3J0KHJvd1N1bXMobSksIGRlY3JlYXNpbmcgPSBUUlVFKQpmcmVjdWVuY2lhX2RmMiA8LSBkYXRhLmZyYW1lKHdvcmQgPSBuYW1lcyhmcmVjdWVuY2lhMiksIGZyZXEgPSBmcmVjdWVuY2lhMikKCgpgYGAKCmBgYHtyfQpnZ3Bsb3QoaGVhZChmcmVjdWVuY2lhX2RmMiwgMTApLCBhZXMoeCA9IHdvcmQsIHkgPSBmcmVxKSkgKwogIGdlb21fYmFyKHN0YXQgPSAiaWRlbnRpdHkiKSArCiAgbGFicyh0aXRsZSA9ICJUT1AgMTAgcGFsYWJyYXMgbcOhcyBmcmVjdWVudGVzIiwgeCA9ICJQYWxhYnJhIiwgeSA9ICJGcmVjdWVuY2lhIiwgc3VidGl0bGUgPSAiQW7DoWxpc2lzIGRlbCBhcmNoaXZvIHRleHRvc19leHRyYWlkb3MiKSArCiAgeWxpbSgwLCBtYXgoZnJlY3VlbmNpYV9kZjIkZnJlcVsxOjEwXSkgKyAxMCkgIyBBanVzdGFyIGVsIGzDrW1pdGUgc3VwZXJpb3IgcGFyYSBtZWpvciB2aXN1YWxpemFjacOzbgoKYGBgCgo=