Mining Supplementary PDFs

Using pdftools to extract tabular information hidden in supplemtary pdfs

A lot of the information is always burried in the supplementary section of any paper. The best possible scenarios is to have raw information avaialble, for example any information in tabular format should have been made available through a csv/tsv file. However for various reasons, this information is buried deep in doc/pdfs.

My aim here was to draw information from one of such supplementary pdfs from a 2011 paper Kishore et al.

rOpenSci’s pdftools package comes in very handy. I was able to extract info from Supplementary table 1. The input here is a trimmed version of the original pdf, with only Supplementary table 1.

I used pdfshuffler, but there are lot of tools that can do this.

library(pdftools)
library(stringr)
txt <- pdf_text('../datasets/kishore_quantitative_2011/kishore_targets.pdf')
all_lines <- unlist(strsplit(txt, split="\r\n",fixed = TRUE))
splitted_lines <- unlist(strsplit(all_lines, split="\n"))
## Do away with the footer
splitted_lines <- as.character(splitted_lines[!grepl('Nature Methods*|Supplementary*', splitted_lines)])
## Replace multiple spaces by single
splitted_lines <- lapply(splitted_lines, function(x) gsub("\\s+", " ", str_trim(x)))
unlist(splitted_lines)[1:7]

[1] "TranscriptID BeginSite EndSite CoverageCLIP CoverageMRNASeq Sequence Sample"                        "NM_018638 3267 3306 256934.0444 0.390896169 TAATACTCTTAATTTTTTTTTTTTTTTTTTTTTTTTTTTT HuR_CLIP_A"   
[3] "NM_020177 5604 5643 12482.26704 0.25452461 TCATCATCATTTTCTTTTTTTCTTTTTCTTTTTTTTTTTT HuR_CLIP_A"     "NM_032682 3030 3069 8616.659701 0.25384843 CCTTTACTCCTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT HuR_CLIP_A"    
[5] "NM_001025105 2650 2689 42182.42275 1.788372875 CATTCTTTCATTTTTTTCTTTTTTTTTTTTTTTTTTTATG HuR_CLIP_A" "NM_001164579 691 730 2775.808962 0.1277939 ACTCTTATTATTTTTTATTTTATTTTATTTTTTTATTTTT HuR_CLIP_A"    
[7] "NM_032226 2192 2231 3040.574399 0.183886964 TCATCAATATTTTTCAACTTTTTTTTTTTTTTTTTTTACT HuR_CLIP_A"   

Next step is simple, convert the list of strings to a dataframe.

df <- data.frame( do.call( rbind, strsplit( as.character(splitted_lines), ' ' ) ) ) 
## Set column names
colnames(df) <- as.character(unlist(df[1,]))
df <- df[-1,]
df

write.table(df, file='../datasets/kishore_quantitative_2011/kishore_clip_targets.csv', row.names = F, sep="\t", quote = F)