Fix geoduck files on Nightingales

First, lets test that the files I’ve been operating on on Emu are correct. There are the original 24 samples that are not in the second drop on Emu, so hopefully the set difference will only have those 24 files, otherwise I’ll be profoundly sad.

Lets look.

library(readr)
hd_md5s <- read_delim("~/Documents/fixNight/md5sum.txt", 
     " ", escape_double = FALSE, col_names = FALSE, 
     trim_ws = TRUE)

Parsed with column specification:
cols(
  X1 = col_character(),
  X2 = col_character()
)

emu_md5s <- read_delim("~/Documents/fixNight/emu_md5s.txt", 
     " ", escape_double = FALSE, col_names = FALSE, 
     trim_ws = TRUE)

Parsed with column specification:
cols(
  X1 = col_character(),
  X2 = col_character()
)

bad.emu_md5s.2 <- emu_md5s[!emu_md5s$X2 %in% hd_md5s$X2,]
print(bad.emu_md5s.2)

Huzzah. Those are the 24 original drop files, so I didn’t waste tons of time on bad files. Hooray.

Next lets look at the RRBS2 directory under p_generosa on nightingales. I originally copied files here because A) I didn’t trust my script totally, and B) I had some permission issues on Owl. There will only be the second drop files here, so there’s hopefully nothing in the badrrbs2 data frame after I’m done

rrbs2_md5s <- read_delim("~/Documents/fixNight/rrbs2_md5s.txt", 
     ",", escape_double = FALSE, col_names = FALSE, 
     trim_ws = TRUE)

Parsed with column specification:
cols(
  X1 = col_character(),
  X2 = col_character()
)

badrrbs2_md5s <- rrbs2_md5s[!rrbs2_md5s$X1 %in% hd_md5s$X1,]
print(badrrbs2_md5s)

Aaaaand not so much. 4 bad files there.

How about in p_geneorsa itself? I’ll shorten the data frame to only include those drop 2 files initially, just for ease.

p_generosa_md5s <- read_delim("~/Documents/fixNight/p_generosa_md5s.txt", 
     " ", escape_double = FALSE, col_names = FALSE, 
     trim_ws = TRUE)

Parsed with column specification:
cols(
  X1 = col_character(),
  X2 = col_character()
)

p_generosa_md5s.2 <- p_generosa_md5s[p_generosa_md5s$X2 %in% hd_md5s$X2,]
p_generosa_md5s.3 <- p_generosa_md5s.2[!p_generosa_md5s.2$X1 %in% hd_md5s$X1,]
print(p_generosa_md5s.3)

Looks like there’s 7 to fix, from my files on Emu. Once that’s done and they check out, everything should match the files on the Genewiz supplied hard drive.

files.to.fix <- p_generosa_md5s.3$X2

lets check the new files just copied over

Woo, looks like all of the files match what was on the Genewiz HDD.

Just for fun, I’ll run the MD5s on the Nightingales/P_generosa files once more. Just because.

system("md5sum ~/Documents/owl/nightingales/P_generosa/EPI*.gz >> ~/Documents/fixNight/secondnightMD5s.txt")

Perform the same operations as the P_generosa files previously, where I subset them to get just the 82 files from the hard drive, and then look for MD5s that aren’t in the hard drive MD5 list.

Ok! Everything on nightingales now matches the files delivered by Genewiz.