Converting Bismark files to methpipe format.

setwd("~/Documents/RobertsLab/methpipe/")
files <- list.files(pattern = "*.bam")
files

 [1] "EPI-103_S27_L005_R1_001_val_1_bismark_bt2_pe.bam"   "EPI-104_S28_L005_R1_001_val_1_bismark_bt2_pe.bam"  
 [3] "EPI-111_S29_L005_R1_001_val_1_bismark_bt2_pe.bam"   "EPI-113_S30_L005_R1_001_val_1_bismark_bt2_pe.bam"  
 [5] "EPI-119_S31_L005_R1_001_val_1_bismark_bt2_pe.bam"   "EPI-120_S32_L005_R1_001_val_1_bismark_bt2_pe.bam"  
 [7] "EPI-127_S33_L005_R1_001_val_1_bismark_bt2_pe.bam"   "EPI-128_S34_L005_R1_001_val_1_bismark_bt2_pe.bam"  
 [9] "EPI-135_S35_L005_R1_001_val_1_bismark_bt2_pe.bam"   "EPI-135WG_S42_L005_R1_001_val_1_bismark_bt2_pe.bam"
[11] "EPI-136_S36_L005_R1_001_val_1_bismark_bt2_pe.bam"   "EPI-143_S37_L005_R1_001_val_1_bismark_bt2_pe.bam"  
[13] "EPI-145_S38_L005_R1_001_val_1_bismark_bt2_pe.bam"   "EPI-151_S2_L002_R1_001_val_1_bismark_bt2_pe.bam"   
[15] "EPI-152_S3_L002_R1_001_val_1_bismark_bt2_pe.bam"    "EPI-153_S4_L002_R1_001_val_1_bismark_bt2_pe.bam"   
[17] "EPI-154_S5_L002_R1_001_val_1_bismark_bt2_pe.bam"    "EPI-159_S6_L002_R1_001_val_1_bismark_bt2_pe.bam"   
[19] "EPI-160_S7_L002_R1_001_val_1_bismark_bt2_pe.bam"    "EPI-161_S8_L002_R1_001_val_1_bismark_bt2_pe.bam"   
[21] "EPI-162_S9_L002_R1_001_val_1_bismark_bt2_pe.bam"    "EPI-167_S10_L002_R1_001_val_1_bismark_bt2_pe.bam"  
[23] "EPI-168_S11_L002_R1_001_val_1_bismark_bt2_pe.bam"   "EPI-169_S12_L002_R1_001_val_1_bismark_bt2_pe.bam"  
[25] "EPI-170_S13_L002_R1_001_val_1_bismark_bt2_pe.bam"   "EPI-175_S14_L003_R1_001_val_1_bismark_bt2_pe.bam"  
[27] "EPI-176_S15_L003_R1_001_val_1_bismark_bt2_pe.bam"   "EPI-181_S16_L003_R1_001_val_1_bismark_bt2_pe.bam"  
[29] "EPI-182_S17_L003_R1_001_val_1_bismark_bt2_pe.bam"   "EPI-184_S18_L003_R1_001_val_1_bismark_bt2_pe.bam"  
[31] "EPI-185_S19_L003_R1_001_val_1_bismark_bt2_pe.bam"   "EPI-187_S20_L003_R1_001_val_1_bismark_bt2_pe.bam"  
[33] "EPI-188_S21_L003_R1_001_val_1_bismark_bt2_pe.bam"   "EPI-193_S22_L003_R1_001_val_1_bismark_bt2_pe.bam"  
[35] "EPI-194_S23_L003_R1_001_val_1_bismark_bt2_pe.bam"   "EPI-199_S24_L003_R1_001_val_1_bismark_bt2_pe.bam"  
[37] "EPI-200_S25_L003_R1_001_val_1_bismark_bt2_pe.bam"   "EPI-205_S26_L004_R1_001_val_1_bismark_bt2_pe.bam"  
[39] "EPI-206_S27_L004_R1_001_val_1_bismark_bt2_pe.bam"   "EPI-208_S28_L004_R1_001_val_1_bismark_bt2_pe.bam"  
[41] "EPI-209_S29_L004_R1_001_val_1_bismark_bt2_pe.bam"   "EPI-214_S30_L004_R1_001_val_1_bismark_bt2_pe.bam"  
[43] "EPI-215_S31_L004_R1_001_val_1_bismark_bt2_pe.bam"   "EPI-220_S32_L004_R1_001_val_1_bismark_bt2_pe.bam"  
[45] "EPI-221_S33_L004_R1_001_val_1_bismark_bt2_pe.bam"   "EPI-226_S34_L004_R1_001_val_1_bismark_bt2_pe.bam"  
[47] "EPI-227_S35_L004_R1_001_val_1_bismark_bt2_pe.bam"   "EPI-229_S38_L004_R1_001_val_1_bismark_bt2_pe.bam"  
[49] "EPI-230_S37_L004_R1_001_val_1_bismark_bt2_pe.bam"   "EPI-41_S38_L005_R1_001_val_1_bismark_bt2_pe.bam"   
[51] "EPI-42_S39_L005_R1_001_val_1_bismark_bt2_pe.bam"   

for(i in 1:length(files))
  
  system(paste0("to-mr -m bismark -v -o ", substr(files[i], 1, 7), ".mr ", files[i]))

Input file: EPI-103_S27_L005_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-103.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Done.
Input file: EPI-104_S28_L005_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-104.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Processed 5000000 records
Processed 6000000 records
Done.
Input file: EPI-111_S29_L005_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-111.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Processed 5000000 records
Done.
Input file: EPI-113_S30_L005_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-113.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Processed 5000000 records
Done.
Input file: EPI-119_S31_L005_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-119.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Processed 5000000 records
Done.
Input file: EPI-120_S32_L005_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-120.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Done.
Input file: EPI-127_S33_L005_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-127.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Done.
Input file: EPI-128_S34_L005_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-128.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Done.
Input file: EPI-135_S35_L005_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-135.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Processed 5000000 records
Done.
Input file: EPI-135WG_S42_L005_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-135.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Processed 5000000 records
Processed 6000000 records
Processed 7000000 records
Processed 8000000 records
Processed 9000000 records
Processed 10000000 records
Processed 11000000 records
Processed 12000000 records
Processed 13000000 records
Processed 14000000 records
Processed 15000000 records
Processed 16000000 records
Processed 17000000 records
Processed 18000000 records
Processed 19000000 records
Processed 20000000 records
Processed 21000000 records
Processed 22000000 records
Processed 23000000 records
Processed 24000000 records
Processed 25000000 records
Processed 26000000 records
Processed 27000000 records
Processed 28000000 records
Processed 29000000 records
Processed 30000000 records
Processed 31000000 records
Processed 32000000 records
Done.
Input file: EPI-136_S36_L005_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-136.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Processed 5000000 records
Done.
Input file: EPI-143_S37_L005_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-143.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Done.
Input file: EPI-145_S38_L005_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-145.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Processed 5000000 records
Done.
Input file: EPI-151_S2_L002_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-151.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Done.
Input file: EPI-152_S3_L002_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-152.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Processed 5000000 records
Done.
Input file: EPI-153_S4_L002_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-153.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Done.
Input file: EPI-154_S5_L002_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-154.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Done.
Input file: EPI-159_S6_L002_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-159.mr
Processed 1000000 records
Processed 2000000 records
Done.
Input file: EPI-160_S7_L002_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-160.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Processed 5000000 records
Done.
Input file: EPI-161_S8_L002_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-161.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Done.
Input file: EPI-162_S9_L002_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-162.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Done.
Input file: EPI-167_S10_L002_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-167.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Done.
Input file: EPI-168_S11_L002_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-168.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Done.
Input file: EPI-169_S12_L002_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-169.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Done.
Input file: EPI-170_S13_L002_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-170.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Done.
Input file: EPI-175_S14_L003_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-175.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Done.
Input file: EPI-176_S15_L003_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-176.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Processed 5000000 records
Processed 6000000 records
Done.
Input file: EPI-181_S16_L003_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-181.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Done.
Input file: EPI-182_S17_L003_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-182.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Processed 5000000 records
Done.
Input file: EPI-184_S18_L003_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-184.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Done.
Input file: EPI-185_S19_L003_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-185.mr
Processed 1000000 records
Done.
Input file: EPI-187_S20_L003_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-187.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Done.
Input file: EPI-188_S21_L003_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-188.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Done.
Input file: EPI-193_S22_L003_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-193.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Done.
Input file: EPI-194_S23_L003_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-194.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Processed 5000000 records
Done.
Input file: EPI-199_S24_L003_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-199.mr
Processed 1000000 records
Processed 2000000 records
Done.
Input file: EPI-200_S25_L003_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-200.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Done.
Input file: EPI-205_S26_L004_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-205.mr
Processed 1000000 records
Processed 2000000 records
Done.
Input file: EPI-206_S27_L004_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-206.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Done.
Input file: EPI-208_S28_L004_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-208.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Processed 5000000 records
Done.
Input file: EPI-209_S29_L004_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-209.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Processed 5000000 records
Done.
Input file: EPI-214_S30_L004_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-214.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Processed 5000000 records
Done.
Input file: EPI-215_S31_L004_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-215.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Done.
Input file: EPI-220_S32_L004_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-220.mr
Processed 1000000 records
Processed 2000000 records
Done.
Input file: EPI-221_S33_L004_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-221.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Done.
Input file: EPI-226_S34_L004_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-226.mr
Processed 1000000 records
Processed 2000000 records
Done.
Input file: EPI-227_S35_L004_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-227.mr
Processed 1000000 records
Processed 2000000 records
Done.
Input file: EPI-229_S38_L004_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-229.mr
Done.
Input file: EPI-230_S37_L004_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-230.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Processed 5000000 records
Done.
Input file: EPI-41_S38_L005_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-41_.mr
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Processed 4000000 records
Done.
Input file: EPI-42_S39_L005_R1_001_val_1_bismark_bt2_pe.bam
Output file: EPI-42_.mr
[bam_header_read] EOF marker is absent. The input is probably truncated.
Processed 1000000 records
Processed 2000000 records
Processed 3000000 records
Done.

the instructions next suggest running duplication remover. So I’ll give that a whirl

mr.files <- list.files(pattern = "*.mr")
for(i in 2:length(mr.files))   {
  
  system(paste0("sort -k 1,1 -k 2,2n -k 3,3n -k 6,6 -o ", substr(mr.files[i], 1, 7), ".mr.sorted_start ", mr.files[i]))
  
}

sorted.files <- list.files(pattern = "*.sorted_start")
for(i in 1: length(sorted.files))   {
  
  
  system(paste0("duplicate-remover -S ", substr(sorted.files[i], 1, 7) , "_stats.txt -o ", substr(sorted.files[i], 1, 7), ".mr.duprem ", sorted.files[i]))
  
  
}

error reading file: EPI-229.mr.sorted_start

Looks like i had a bac copy of the EPI-229 file, redownloaded it from Owl, ran it in terminal, and all seems well. On to methcounts. I ran the 103 sample by hand in terminal just to make sure of the syntax, the rest will be run in a loop

methcounts.file <- list.files(pattern = "*.duprem")
for(i in 2:length(methcounts.file))   {
  
  system(paste0("methcounts -c Panopea_generosa-Scaff-10k.fa -o ", substr(methcounts.file[i], 1, 7), ".meth ", methcounts.file[i]))
  
  
}

Add a new chunk by clicking the Insert Chunk button on the toolbar or by pressing Cmd+Option+I.

When you save the notebook, an HTML file containing the code and output will be saved alongside it (click the Preview button or press Cmd+Shift+K to preview the HTML file).