This lesson introduces the concept of invariant columns and why they should be removed. It also provides a function to remove them.
All of this material will appear on the exam. Take notes on the workflow, functions, and concepts.
By the end of this lesson you will
SNPs are Single Nucleotide Polymorphisms. This means that they are by definition polymorphic - more than one nucleotide occurs in that position (locus) in different individual in a species. For example, most individuals in a population may be G|G, but some are G|A or even A|A.
The SNP concept applies at the level of a species. Within separate populations of a species (or individuals genotyped for a study) only one allele of the SNP may be present. This means that though there is genetic variation at that locus in the level of species, for the the population or sample for a study there is no variation.
If all individuals in a population are homozygous for either the major allele (e.g. all are MM), or all are homozygous for the minor allele (e.g. all are mm) we say that the allele is fixed in that population. They other allele is present in other populations, but not all of them. When an allele is fixed in a population it is actually very useful for distinguishing that population genetically from others.
Similarly, for a sample of individuals in a single study, all individuals genotyped may turn out to be the same type of homozygote for a SNP - all are mm or MM. A different sample may have yield heterozygotes or a mix of both types of homozygotes (mm and MM), but for the data at hand, only one type of homozygote is present. While the allele is not necessarily fixed in any population, it is invariant in the individuals that happened to be gentoyped.
SNP data is converted to numeric genotype scores for analysis with machine learning methods such as PCA. If a SNP is invariant in all of the individuals genotyped in a sample for a study, the entire column (aka feature) in the dataframe will be either 0 (all individuals have the major allele) or 2 (all individuals have the minor allele). Similarly, if all individuals are heterozygotes, the entire column will be 1.
Completely invariant alleles in a dataset present 2 issues:
When a column in a dataset is invariant, whether a SNP or anything else, it doesn’t provide you any useful information. For example, if I want to predict whether taking Advanced Placement (AP) Biology predicts success in my intro biology course, and everyone in the class has taken AP Bio, then I can’t learn or conclude anything.
Invariant features, however, still have to be handled by the computer. They therefore eat up computational resources. Since SNP data can have tens or ever hundreds of thousands of features, this can be a major waste of computational time. It is therefore advisable to remove invariant features to speed up the your analyses.
A feature that is invariant will have a standard deviation of 0. For example:
x <- c(2,2,2,2,2,2)
mean(x)## [1] 2sd(x)## [1] 0While we can calculate the mean and standard deviation of an invariant feature, we can’t scale it because this requires division by 0, which is not defined: there is no answer to, for example, x/0.
More generally, machine learning algorithms can run into problems with invariant features and fail to work. It is therefore advisable to remove invariant features before an analysis.
There are several ways to identify if a column in a dataframe is invariant, but probably the easiest is to calculate the standard deviation. The standard deviation is a measure of variation in the data, and if everything in a column is the same, there is no variation and the sd = 0. You could similarly calculate the range; if all the data in a column is 2, then the minimum is 2, the max is 2, and the range is 2 - 2 = 0.
Here’s some example data that shows that this works for all 3 possible forms of invariant SNP genotypic scores data:
First, some fake data. Add c() to make the vectors.
# add the c() function to make all of the vectors
# All samples are homozygous for major allele
SNP_fixed_0 <- c(0, 0, 0, 0, 0, 0) # TODO
# All samples are homozygous for minor allele
SNP_fixed_2 <- c(2, 2, 2, 2, 2, 2) # TODO
# All samples are heterozygous, e.g "A|G"
SNP_all_hetero <- c(1, 1, 1, 1, 1, 1) # TODONow the standard deviations with sd():
# add sd() to calculate the SDs
sd(SNP_fixed_0) # TODO## [1] 0sd(SNP_fixed_2)## [1] 0sd(SNP_all_hetero)## [1] 0Now let’s make a larger dataframe with some features that aren’t invariant.
First, some invariant features:
# Mixture of the 2 homozygotes
SNP_mix_vs1 <- c(0, 0, 0, 2, 2, 2)
# Mixture of the 2 homozygotes
## and heterozygotes
SNP_mix_vs2 <- c(0, 0, 1, 1, 2, 2)We put these into a dataframe:
# add data.frame() and assign to an object
## called df to look at the data
df <- data.frame(SNP_fixed_0, # TODO
SNP_fixed_2,
SNP_mix_vs1,
SNP_mix_vs2)
df## SNP_fixed_0 SNP_fixed_2 SNP_mix_vs1 SNP_mix_vs2
## 1 0 2 0 0
## 2 0 2 0 0
## 3 0 2 0 1
## 4 0 2 2 1
## 5 0 2 2 2
## 6 0 2 2 2To identify which columns are invariant we first calculate all the SDs.
# add sd() to calculate the SDs
# add [, 1] etc to each line
## lines to calculate the sd on each column
## The first one is shown as an exame
#add [, 1]
sd_col01 <- sd(df[,1]) # EXAMPLE
#add [, 2]
sd_col02 <- sd(df[,1]) # TODO
#add [, 3]
sd_col03 <- sd(df[,1]) # TODO
#add [, 4]
sd_col04 <- sd(df[,1]) # TODOWe can put this all into a vector that contains the SDs:
# add c() to make into a vector
## assign it to a vector called sd_vector
sd_vector <- c(sd_col01, # TODO
sd_col02,
sd_col03,
sd_col04)Now use which() to see which parts of the vector are
equal to 0.
# add which() and == 0
which(sd_vector == 0 ) # TODO## [1] 1 2 3 4We’ll save this to a vector to hold the indices of the invariant columns.
# assign to a vector called i_invariant
i_invariant <- which(sd_vector == 0) # TODOThis vector of indices can be used to look at which columns are invariant in our dataframe:
# add [,i_invariant] to see
## the invariant columns
df # TODO## SNP_fixed_0 SNP_fixed_2 SNP_mix_vs1 SNP_mix_vs2
## 1 0 2 0 0
## 2 0 2 0 0
## 3 0 2 0 1
## 4 0 2 2 1
## 5 0 2 2 2
## 6 0 2 2 2We can then use this to show use which columns DO vary by using negative indexing to remove the invariant columns.
# add -i_invariant to REMOVE the invariant columns
df[,-i_invariant] # TODO## data frame with 0 columns and 6 rowsWe can then make a new dataframe of just the variant columns
# assign to a dataframe called df03
df03 <- df[,-i_invariant] #TODOLook at the results:
df03## data frame with 0 columns and 6 rowsThe function below will remove invariant columns from a dataframe, as
long as you add return(x) to the end.
# run this function
## add return(x_no_invar)
invar_omit <- function(x){
cat("Dataframe of dim",dim(x), "processed...\n")
sds <- apply(x, 2, sd, na.rm = TRUE)
i_var0 <- which(sds == 0)
cat(length(i_var0),"columns removed\n")
if(length(i_var0) > 0){
x <- x[, -i_var0]
}
## add return() with x in it
return(x) #TODO
}We can test it on our data above.
# run invar_omit() on df
invar_omit(df) # TODO## Dataframe of dim 6 4 processed...
## 2 columns removed## SNP_mix_vs1 SNP_mix_vs2
## 1 0 0
## 2 0 0
## 3 0 1
## 4 2 1
## 5 2 2
## 6 2 2It should only change data dataframe when columns are invariant. Its always good to test a function to make sure it does what you want. If we run it once, save the output, and run it again on the output we just made, it shouldn’t remove anything the second time.
# run invar_omit() on df
df_no_invar <- invar_omit(df) # TODO## Dataframe of dim 6 4 processed...
## 2 columns removed# look out output
df_no_invar## SNP_mix_vs1 SNP_mix_vs2
## 1 0 0
## 2 0 0
## 3 0 1
## 4 2 1
## 5 2 2
## 6 2 2# run invar_omit() on df_no_invar
invar_omit(df_no_invar) # TODO## Dataframe of dim 6 2 processed...
## 0 columns removed## SNP_mix_vs1 SNP_mix_vs2
## 1 0 0
## 2 0 0
## 3 0 1
## 4 2 1
## 5 2 2
## 6 2 2Let’s work through an example with a real VCF file. The steps we need to follow are
vcfR package..vcffile is present with
list.files()vcfR::read.vcfR()vcfR::extract.gt()t()invar_omit()First, load up the vcfR package with library() and check
the working directory with getwd().
# call library() on vcfR
library(vcfR) #TODO##
## ***** *** vcfR *** *****
## This is vcfR 1.13.0
## browseVignettes('vcfR') # Documentation
## citation('vcfR') # Citation
## ***** ***** ***** *****# check your working directory with getwd()
getwd() #TODO## [1] "/Users/danyajung/Downloads"Second, set the working directory to the location of this file if it isn’t already. Do this via
Session->Set Working Directory -> To source file location
Third, make sure the file all_loci.vcf is present with
list.files() and/or
list.files(pattern = "vcf").
# call list.files()
list.files() #TODO## [1] "_tmp__lease_documents_20220805123038.pdf"
## [2] "_tmp__lease_documents_20220805123547.pdf"
## [3] "%EC%A3%BC%EC%9D%98.html"
## [4] "01CA9274-D435-4A5B-998D-9319E0058B21.JPEG"
## [5] "11_Clicker_Questions.ppt"
## [6] "11.21443531-21683531.ALL.chr11_GRCh38.genotypes.20170504.vcf"
## [7] "13moretrigidentities.pdf"
## [8] "189581 copy.png"
## [9] "1953-nature-papers-watson-crick-wilkins-franklin-2.pdf"
## [10] "1953-nature-papers-watson-crick-wilkins-franklin-3.pdf"
## [11] "1953-nature-papers-watson-crick-wilkins-franklin-4-2.pdf"
## [12] "1953-nature-papers-watson-crick-wilkins-franklin-4.pdf"
## [13] "1953-nature-papers-watson-crick-wilkins-franklin.en.ko.pdf"
## [14] "1953-nature-papers-watson-crick-wilkins-franklin.pdf"
## [15] "19ed36e85a7e8a52ee67488ede3e7085.jpg"
## [16] "2020-10-07-141516-2.jpg"
## [17] "2020-10-07-141516-3 (2).jpg"
## [18] "2022-01-29_022914.pdf"
## [19] "2022-02-22_143744.pdf"
## [20] "2022-04-20_014652.pdf"
## [21] "2022-04-24_222325.pdf"
## [22] "2022-04-25_163137.pdf"
## [23] "2022-05-17_204604 2.pdf"
## [24] "2022-05-17_204604.pdf"
## [25] "2022.pdf"
## [26] "ACFrOgCd7XetGn7-UqMhbTaNAb01yjrBvXh7yYYS4HoyU37K_mdYa1a9A0-LLNNRcQBa0VERlfs8urpjW9Yo2S-WavUGYzVdm_m9E-a2WqRkP0Z1je3D6Qtvto2wDsQ=.pdf"
## [27] "ACSExamReview_Spring2018-1 (2).pdf"
## [28] "ACSExamReview_Spring2018-1.pdf"
## [29] "Adventure Game Deliverable 3 CS 0007.pdf"
## [30] "Algebra"
## [31] "all_loci-1.vcf"
## [32] "all_loci.vcf"
## [33] "ALL.chr11_GRCh38.genotypes.20170504.vcf"
## [34] "Animal Development.docx"
## [35] "Annuity and Amortization Comprehensive Problem #1 S22.docx"
## [36] "Annuity and Amortization Comprehensive Problem #2 S22.docx"
## [37] "Anth 1557 Exam 1 Review Guide.pdf"
## [38] "Anth 1557 Exam 2 Review Guide.pdf"
## [39] "Anthony2009.pdf"
## [40] "Applications of angle measure worksheet S22.docx"
## [41] "Argumentative essay-2.pdf"
## [42] "Argumentative essay-3.pdf"
## [43] "Argumentative essay-4.pdf"
## [44] "Argumentative essay.docx"
## [45] "Argumentative essay.pdf"
## [46] "Argumentative peer review.docx"
## [47] "Assessment for Enzymes_Revised.docx"
## [48] "assisted suicide outline-2.pdf"
## [49] "assisted suicide outline.pdf"
## [50] "bfd817c2-72b4-4b40-a01a-a9304086a580.pdf"
## [51] "Bio 2 Final Review 2022.pdf"
## [52] "Bio Chap 10-2.pdf"
## [53] "Bio Chap 10.pdf"
## [54] "BIO CHAP 2.pdf"
## [55] "Bio Chap 35.pdf"
## [56] "Bio final essay-2.pdf"
## [57] "Bio final essay-3.pdf"
## [58] "Bio final essay-Final paper.docx"
## [59] "Bio final essay.docx"
## [60] "Bio final essay.pdf"
## [61] "BIO LAB inquary Qs.pdf"
## [62] "BIO LAB Reflection Q.1.pdf"
## [63] "Bio lab report 1 data - Sheet1.pdf"
## [64] "Bio Lab Report 1-2.pdf"
## [65] "Bio Lab Report 1.pdf"
## [66] "bio2 chapter 14.pdf"
## [67] "Bio2 Midterm Review"
## [68] "Bioinformatics_Lab10.docx"
## [69] "Biology 1 Laboratory Syllabus F'21.docx"
## [70] "Blank_Excel.xlsx - LA Docs.html"
## [71] "Breast Cancer Worksheet 1.docx"
## [72] "Breast Cancer Worksheet 1.docx.pdf"
## [73] "Buffer worksheet.pdf"
## [74] "Bullet points.docx"
## [75] "Bullet points.pdf"
## [76] "BusinessCard-2021-09-19-153209.pdf"
## [77] "By_Steppe_Desert_and_Ocean_The_Birth_of_Eurasia_----_(1_The_Land_and_the_People_).pdf"
## [78] "By_Steppe_Desert_and_Ocean_The_Birth_of_Eurasia_----_(1_The_Land_and_the_People_).pdf.zip"
## [79] "By_Steppe_Desert_and_Ocean_The_Birth_of_Eurasia_----_(1_The_Land_and_the_People_)%5B01-33%5D.pdf"
## [80] "By_Steppe_Desert_and_Ocean_The_Birth_of_Eurasia_----_(1_The_Land_and_the_People_)%5B01-34%5D.pdf"
## [81] "calorimetry lab report.pdf"
## [82] "Calorimetry_Data_Results.xlsx"
## [83] "Campus UPG Relocation Request Form.pdf"
## [84] "cat.jpg"
## [85] "Cell Division Notes"
## [86] "Cell Division Notes-2"
## [87] "Cell Division Notes.docx"
## [88] "CH_110_ACS_EXAM_STUDY_GUIDE (1).pdf"
## [89] "CH_110_ACS_EXAM_STUDY_GUIDE_ANSWER_SHEET (1).pdf"
## [90] "chap 10 notes.pdf"
## [91] "Chap. 49.pdf"
## [92] "Chap.44.pdf"
## [93] "chap.48.docx"
## [94] "Chapter 1.doc"
## [95] "Chapter 11 Problems-2.pdf"
## [96] "Chapter 11 Problems.pdf"
## [97] "Chapter 11: 2221 BIOSC 0170 SEC3030 FOUNDATION OF BIOLOGY 1.html"
## [98] "Chapter 12 Problems edit.pdf"
## [99] "Chapter 12 Review.docx"
## [100] "Chapter 13 Problems.pdf"
## [101] "Chapter 13 Review.docx"
## [102] "Chapter 14 Review.docx"
## [103] "Chapter 15 Review.docx"
## [104] "Chapter 16 Problems.pdf"
## [105] "Chapter 16 Review.docx"
## [106] "Chapter 17 Problems.pdf"
## [107] "Chapter 17 Review.docx"
## [108] "Chapter 18 Review.docx"
## [109] "Chapter 19 Review.docx"
## [110] "Chapter 2 Activity-2.pdf"
## [111] "Chapter 2 Activity.pdf"
## [112] "Chapter 2 Problems.pdf"
## [113] "Chapter 2.doc"
## [114] "Chapter 20 Review.docx"
## [115] "Chapter 21 Review.docx"
## [116] "Chapter 22 Review.docx"
## [117] "Chapter 23 Review.docx"
## [118] "Chapter 26 Review.docx"
## [119] "Chapter 27 Review.docx"
## [120] "Chapter 3 Problems.pdf"
## [121] "Chapter 3 Review-1.pdf"
## [122] "Chapter 3 review.doc"
## [123] "Chapter 4 Problems Answer Key.pdf"
## [124] "Chapter 4 Problems.pdf"
## [125] "Chapter 4 review.doc"
## [126] "Chapter 40_Study Guide.docx"
## [127] "Chapter 5 Problems.pdf"
## [128] "Chapter 5_review.doc"
## [129] "Chapter 51 Review.docx"
## [130] "Chapter 52 Review.docx"
## [131] "Chapter 7 Molecular Shapes Activity.pdf"
## [132] "Chapter 9 Problems - answer key.pdf"
## [133] "Chapter 9 Problems.pdf"
## [134] "character.java"
## [135] "cheatsheet_exam5.HEIC"
## [136] "Chem chap 4 w:s.pdf"
## [137] "CHEM CHAP.1-2.pdf"
## [138] "CHEM CHAP.1.pdf"
## [139] "CHEM chap.5 problems.pdf"
## [140] "chem chap7.pdf"
## [141] "CHEM LAB 2_ post-lab.pdf"
## [142] "Chem lab final review.pdf"
## [143] "Chem Lab Report 1.pdf"
## [144] "CHEM LAB_ post-lab.pdf"
## [145] "chem post lab 6.pdf"
## [146] "Chemistry2e.pdf"
## [147] "Chemistry2e.pdf-2.download"
## [148] "Chemistry2e.pdf.download"
## [149] "Citation_ Assisted Suicide.pdf"
## [150] "code_checkpoint_vcfR.html"
## [151] "code_checkpoint_vcfR.Rmd"
## [152] "CODE_CHECKPOINT-first_rstudio_script.R"
## [153] "Comp1Essay1peerreview-1.doc"
## [154] "Comp1Essay1peerreview-1.pages"
## [155] "comp2rubric-2.docx"
## [156] "Conerstone Reflection #1.pdf"
## [157] "Covid Vaccine Card 2021.jpeg"
## [158] "Criminal Justice.docx"
## [159] "critical response.pdf"
## [160] "CS 0007 Syllabus.pdf"
## [161] "Cunliffe intro.pdf"
## [162] "Curriculum-fulfillment-in-Engage-Fall-18.pdf"
## [163] "D81A8FDF-E93C-4AEB-8AAF-49AB1E5AF378.JPEG"
## [164] "Danya's Resume-2.pdf"
## [165] "Danya's Resume.pdf"
## [166] "Danya's Resume.pdf"
## [167] "Dawoon Jung.mp4"
## [168] "Dawoon-2.docx"
## [169] "Dawoon-3.docx"
## [170] "Dawoon-4.docx"
## [171] "Dawoon.docx"
## [172] "DisclosureStatement.pdf"
## [173] "Document_2021-12-10_145124.pdf"
## [174] "Document_2022-01-18_032436.pdf"
## [175] "Document_2022-03-16_002213.pdf"
## [176] "Document_2022-03-22_032209.pdf"
## [177] "Document_2022-03-22_042937.pdf"
## [178] "Document_2022-04-12_001504.pdf"
## [179] "Document_2022-04-12_171636.pdf"
## [180] "Document_2022-04-18_062809.pdf"
## [181] "Document_2022-04-18_064828.pdf"
## [182] "Document-2021-09-20-230931.pdf"
## [183] "Dog Tree(1).png"
## [184] "Dog Tree(2).png"
## [185] "download-2.html"
## [186] "download-3.html"
## [187] "download-4.html"
## [188] "download-5.html"
## [189] "download.html"
## [190] "Downloads.Rproj"
## [191] "eclipse-inst-jre-mac64.dmg"
## [192] "ENG ILP.pdf"
## [193] "Eng_ essay 1 draft-2.pdf"
## [194] "Eng_ essay 1 draft.pdf"
## [195] "Eng_ essay 1.pdf"
## [196] "Eng_ essay 2-2.pdf"
## [197] "Eng_ essay 2-3.pdf"
## [198] "Eng_ essay 2-4.pdf"
## [199] "Eng_ essay 2.pdf"
## [200] "Eng_ essay 3-2.pdf"
## [201] "Eng_ essay 3-3.pdf"
## [202] "Eng_ essay 3.pdf"
## [203] "EquilibriumBackgroundELN2020.pdf"
## [204] "essay 3 draft.docx"
## [205] "Essay 3 RD.docx"
## [206] "Essay 4 RD.pdf"
## [207] "Essay 4.docx"
## [208] "Essay 4.pdf"
## [209] "Essay1 Peer Review.pdf"
## [210] "Exam 2 practice Fall 2022 updated.docx"
## [211] "Family Feud Module 4.docx"
## [212] "feature_engineering_intro_2_functions-part2.Rmd"
## [213] "fildede.pdf"
## [214] "file.pdf"
## [215] "Final essay PLR 1 (1).docx"
## [216] "Final Exam Review-1.pdf"
## [217] "Final Exam review-GenChem2-Rowland.pptx"
## [218] "Final for PSC S2022.docx"
## [219] "Final for PSC S2022.pdf"
## [220] "final paper - Google Docs.pdf"
## [221] "Final paper presentation.pdf"
## [222] "final paper-2.pdf"
## [223] "final paper.pdf"
## [224] "final project.docx"
## [225] "Final Review Sheet F'21.docx"
## [226] "Final Review.docx"
## [227] "final soucres-1.pdf"
## [228] "final soucres.pdf"
## [229] "Final_Review_CHEM_0120.pdf"
## [230] "Formal Report Writing.docx"
## [231] "Formulas for Applications of Angle Measurement.docx"
## [232] "Foundations 2 Syllabus.pdf"
## [233] "GB Ch42 outline_Part 1.docx"
## [234] "GB_ Ch48_Study Guids.docx"
## [235] "GB_Ch1_PP_F21.pptx"
## [236] "GB_Ch10_Outline.docx"
## [237] "GB_Ch10_Outline.pages"
## [238] "GB_Ch10_PPs_F21.pptx"
## [239] "GB_Ch11_Outline_F21.docx"
## [240] "GB_Ch11_Outline_F21.pages"
## [241] "GB_Ch11_PPs_F21.pptx"
## [242] "GB_CH11_StudyGuide-F21.docx"
## [243] "GB_Ch2_Outline_F21.docx"
## [244] "Gb_Ch40_Outline.docx"
## [245] "GB_Ch41_Outline.docx"
## [246] "GB_Ch41_Study Guide.docx"
## [247] "GB_Ch42_Part1_Study Guide.doc"
## [248] "GB_Ch42_part2_Outline.docx"
## [249] "GB_Ch42_part2_Study Guide.docx"
## [250] "GB_Ch48_Outline.docx"
## [251] "GB_Ch49_Outline.docx"
## [252] "Gb_Ch49_Study Guide.docx"
## [253] "GB_Ch6_StudyGuide.docx"
## [254] "GB_Ch7_StudyGuide.docx"
## [255] "GB_Ch8_Review Sheet.doc"
## [256] "Gb_Chapter42ptIITerms_GasExch.docx"
## [257] "GB_Module 1_Ch1_outline_F21.docx"
## [258] "Gen Chem 0110 Exam 1 Chapter 1 and 2 Study Guide.pptx"
## [259] "Gen Chem 0110 Exam 3 chapters 9 and 5 edit.pdf"
## [260] "Gen Chem 2 Chapter 10 Problems-2.pdf"
## [261] "Gen Chem 2 Chapter 10 Problems.pdf"
## [262] "Gen Chem II ICE table handout - solutions.pdf"
## [263] "Gen Chem II ICE table handout.pdf"
## [264] "Genetics PLR1 shared doc.pdf"
## [265] "Goldberg_Genetics_7e_CH01_SMSG_pdf.pdf"
## [266] "Goldberg_Genetics_7e_CH02_SMSG_pdf.pdf"
## [267] "Grades.pdf"
## [268] "Graduate school info. packet.pdf"
## [269] "Graduate school sheet.doc"
## [270] "Grammarly.dmg"
## [271] "Handwritten_2021-12-10_144959.pdf"
## [272] "Handwritten_2022-02-01_123549.pdf"
## [273] "Handwritten-2021-09-14-213517.pdf"
## [274] "Hanks etal 2016.pdf"
## [275] "HelloWorld.class"
## [276] "HHMI Cell Cycle-4.docx"
## [277] "HIV-DNA-Integration-Student-fillable copy.pdf"
## [278] "HPV argument.docx"
## [279] "ideaIC-2022.2.3.dmg"
## [280] "IMG_0140.HEIC"
## [281] "IMG_0162.jpg"
## [282] "IMG_0394.HEIC"
## [283] "IMG_0395.HEIC"
## [284] "IMG_0562.HEIC"
## [285] "IMG_0991 2.HEIC"
## [286] "IMG_0991.HEIC"
## [287] "IMG_1034.heic"
## [288] "IMG_1172.HEIC"
## [289] "IMG_1252.HEIC"
## [290] "IMG_1311.HEIC"
## [291] "IMG_3143 2.heic"
## [292] "IMG_3143.heic"
## [293] "IMG_3144 2.jpg"
## [294] "IMG_3144.jpg"
## [295] "IMG_5297.heic"
## [296] "IMG_8141.JPG"
## [297] "IMG_8143 2.JPG"
## [298] "IMG_8143.JPG"
## [299] "IMG_8292.jpg"
## [300] "IMG_8292.jpg.pdf"
## [301] "IMG_8292.pdf"
## [302] "Informative peer review-1-1.docx"
## [303] "Install Respondus LockDown Browser (x64c) 346685215.pkg"
## [304] "InstallLDBPackage64c-2-0-8-00.zip"
## [305] "internship_agreement_fillable_fall_2020_-final.docx"
## [306] "Interview with Dr. Smyder.pdf"
## [307] "Java Overview.pdf"
## [308] "jdk-18_macos-aarch64_bin.dmg"
## [309] "Journal Prompt .pdf"
## [310] "JSocietyEssayPrompts2022Fall.docx"
## [311] "JSocietyLectOutline08_31_22.docx"
## [312] "JSocietyLectOutline09_07_22.docx"
## [313] "JSocietyLectOutline09_12_22.docx"
## [314] "JSocietyLectOutline09_14_22.docx"
## [315] "JSocietyLectOutline09_19_22.docx"
## [316] "JSocietyLectOutline09_21_22.docx"
## [317] "JSocietyLectOutline09_28_22.docx"
## [318] "JSocietyLectOutline10_10_22.docx"
## [319] "JSocietyLectOutline10_12_22.docx"
## [320] "JSocietyLectOutline10_17_22.docx"
## [321] "JSocietyLectOutline10_26_22.docx"
## [322] "JSocietyLectOutline11_07_22.docx"
## [323] "JSocietyLectOutline11_09_22.docx"
## [324] "JSocietyLectOutline11_14_22.docx"
## [325] "JSocietyLectOutline11_16_22.docx"
## [326] "Jung Dawoon_Plan_4:2022.pdf"
## [327] "KakaoTalk_Photo_2021-06-22-02-41-33.jpeg"
## [328] "Karyotyping Case Study Report.docx"
## [329] "katona_csete.pdf"
## [330] "Kinetics Cheat Sheet.pdf"
## [331] "lab report data.jpeg"
## [332] "Lab Safety Rules 2021.doc"
## [333] "Lab2.java"
## [334] "lab7.java"
## [335] "Laboratory Safety Rules-LAST PAGE TO SIGN.docx"
## [336] "Lecture 1 Outline (The Cell Cycle Chapter 12).docx"
## [337] "Lecture 10 Outline Regulation of Gene Expression (Chapter 18).pdf"
## [338] "Lecture 11 Outline DNA Tools and Biotechnology (Chapter 20).doc"
## [339] "Lecture 12 Outline Genomes and Their Evolution (Chapter 21).pdf"
## [340] "Lecture 13 Outline Descent with Modification (Chapter 22).docx"
## [341] "Lecture 14 Outline Evolution of Populations (Chapter 23).docx"
## [342] "Lecture 15 Outline Introduction to Ecology and the Biosphere (Chapter 52).docx"
## [343] "Lecture 16 Outline Animal Behavior (Chapter 51).docx"
## [344] "Lecture 17 Outline Population Ecology (Chapter 53).docx"
## [345] "Lecture 18 Outline Community Ecology (Chapter 54)(1).docx"
## [346] "Lecture 2 Outline (Meiosis & Life Cycles Chapter 13).docx"
## [347] "Lecture 3 Outline The Molecular Basis of Inheritance (Chapter 16).doc"
## [348] "Lecture 4 Outline Gene Expression From Gene to Protein (Chapter 17).doc"
## [349] "Lecture 6 Outline Chromosomal Basis of Inheritance (Chapter 15).pdf"
## [350] "Lecture 7 Outline Viruses (Chapter 19).doc"
## [351] "Lecture 7 Outline Viruses (Chapter 19).pdf"
## [352] "Lecture 8 Bacteria and Archaea (Chapter 27)-2.pptx"
## [353] "Lecture 8 Outline Bacteria and Archaea (Chapter 27).pdf"
## [354] "Lecture A Mendel Fall Fall 2022 pre.pdf"
## [355] "Lecture B Beyond Mendel Fall 2022 pre.pdf"
## [356] "Lecture Outline Phylogeny & the Tree of Life (Chapter 26).pdf"
## [357] "Lecture+1+Outline+%28The+Cell+Cycle+Chapter+12%29.docx.pdf"
## [358] "legalizing assisted suicide (1).docx"
## [359] "legalizing assisted suicide.docx"
## [360] "M&MWS2-2.docx"
## [361] "Map 1-1.pdf"
## [362] "Map 1.pdf"
## [363] "Meiosis_Lab2.docx"
## [364] "Meningitis 2022-2023.pdf"
## [365] "MIDTERM STUDY GUIDE_Bios0070 F'21.docx"
## [366] "Minds on Microscopy A Forensis Approach - student version.pdf"
## [367] "Module 1 Study Questions.docx"
## [368] "Module 4 Puzzle Key.docx"
## [369] "My ILP-2.pdf"
## [370] "My ILP.pdf"
## [371] "Order 2665087 Kimchi.docx"
## [372] "paint demaged..jpg"
## [373] "PCA-missing_data-KEY.Rmd"
## [374] "PCA-missing_data.Rmd"
## [375] "Peer Review #2.pdf"
## [376] "PITT_TSRPT Fall+Spring.pdf"
## [377] "PITT_TSRPT NEW.pdf"
## [378] "PITT_TSRPT.pdf"
## [379] "Plagiarism Assignment Sheet.pdf"
## [380] "Plagiarism Awarness Assignment.pdf"
## [381] "PlainLanguageDisclosure.pdf"
## [382] "PLR2.docx"
## [383] "PLR2.pdf"
## [384] "Post lab 4.pdf"
## [385] "Post lab 5.pdf"
## [386] "PostExamEvaluation_0110LectureF20Exam3.docx"
## [387] "PowerPoint Outline Anth1557 09.06.pdf"
## [388] "Practice final for PSC S2022.docx"
## [389] "Practice final for PSC S2022.pdf"
## [390] "Practice on Domain & Range.docx"
## [391] "Practice Problems on Uninhibited Exponential Growth and Decay S22.docx"
## [392] "Practice Quiz Module 2.docx"
## [393] "Practice Quiz on Laws of s & c.doc"
## [394] "Practice Quiz on Laws of s & c.pdf"
## [395] "Practice Quiz_Module 1 Exam.docx"
## [396] "Practice Test on Domain and Range, and Graphing Functions for PSC.docx"
## [397] "Practice Test on Logs and Functions S22.docx"
## [398] "Practice Test on Trig S22.docx"
## [399] "proposal.pdf"
## [400] "QFR \"tools for scientific Inquiry\" .pdf"
## [401] "QFR Pages 15-17.pdf"
## [402] "QFR Tools for Scientific Inquiry for Canvas.doc"
## [403] "Quiz 1 9-6.pptx"
## [404] "Quiz on Exponential Growth and Decay S22.docx"
## [405] "Quiz on Graphing Rational Functions S22.docx"
## [406] "R-3.0.2.pkg"
## [407] "R-4.2.1.pkg"
## [408] "R-4.2.2"
## [409] "R-4.2.2-arm64.pkg"
## [410] "R-4.2.2.tar"
## [411] "RandomNum-2.java"
## [412] "RandomNum-3.java"
## [413] "RandomNum.java"
## [414] "RandomNum.java.html"
## [415] "Receipt_2022-02-12_233656.pdf"
## [416] "Receipt_2022-02-23_011121.pdf"
## [417] "Receipt_2022-03-21_030146.pdf"
## [418] "Recitation 1 Fall 2022 PDF version.pdf"
## [419] "Reflection #2.docx"
## [420] "removing_fixed_alleles.Rmd"
## [421] "Research notes.docx"
## [422] "Resultsedited revised.docx"
## [423] "Resultsedited.docx"
## [424] "Review1_2022Fall.doc"
## [425] "Review2_2022Fall.docx"
## [426] "Rosalind Franklin and the double helix 1.1570771-2.pdf"
## [427] "Rosalind Franklin and the double helix 1.1570771.pdf"
## [428] "rough draft paper.pdf"
## [429] "rough draft.pdf"
## [430] "rsconnect"
## [431] "RStudio-2022.07.1-554"
## [432] "RStudio-2022.07.1-554.dmg"
## [433] "RStudio-2022.07.2-576.dmg"
## [434] "RStudio-2022.07.2-576.exe"
## [435] "rtools42-5355-5357.exe"
## [436] "rubric.pdf"
## [437] "Safety_Scavenger_Hunt-2.pages"
## [438] "Safety_Scavenger_Hunt-3.pages"
## [439] "Safety_Scavenger_Hunt-4.pages"
## [440] "Safety_Scavenger_Hunt.docx"
## [441] "Safety_Scavenger_Hunt.pages"
## [442] "SafetyAgreement_RemoteSignatures.pdf"
## [443] "SafetyAgreement_Rev8_21.pdf"
## [444] "Sample student Outline 1.docx"
## [445] "Scavenger Hunt F'17 Student copy.docx"
## [446] "Scavenger hunt.pdf"
## [447] "Schedule: 2231 BIOSC 0350 SEC1100 GENETICS.pdf"
## [448] "sciencesoption1.pdf"
## [449] "Scientific Articles & Data Analysis.pptx"
## [450] "Scott et al. 2022-2.pdf"
## [451] "Scott et al. 2022.pdf"
## [452] "Screen Shot 2022-09-18 at 2.10.37 PM.png"
## [453] "Screen Shot 2022-09-20 at 4.00.40 PM.png"
## [454] "Screen Shot 2022-09-20 at 4.51.21 PM.png"
## [455] "Screen Shot 2022-09-20 at 4.51.29 PM.png"
## [456] "Screen Shot 2022-10-12 at 1.21.49 AM.png"
## [457] "Screen Shot 2022-10-16 at 9.51.07 PM.png"
## [458] "Screen Shot 2022-10-25 at 5.16.19 PM.png"
## [459] "Solubility activity.pdf"
## [460] "Solubility Rules.pdf"
## [461] "Sources for Assisted Suicide.pdf"
## [462] "Spectrophotometry_Data_Results.xlsx"
## [463] "Spectrophotometry_Data_Results.xlsx - LA Docs.html"
## [464] "STAT course description.pdf"
## [465] "synthesis paragraph.pdf"
## [466] "synthesis paragraphs .pdf"
## [467] "Synthesis Paragraphs Sample Cont..docx"
## [468] "taylor and Barron-Ortiz 2021-2.pdf"
## [469] "taylor and Barron-Ortiz 2021.pdf"
## [470] "Test on Domain and Range, and Graphing Functions for PSC-1.docx"
## [471] "Test on Trig S22.docx"
## [472] "test.docx"
## [473] "Text Adventure Game Deliverable 2 CS 0007.pdf"
## [474] "The Bridges of Forbes- lease document.pdf"
## [475] "TokenFactoryIframe"
## [476] "TopicsDatesAssignmentsPoints F'21 .docx"
## [477] "transcript.txt"
## [478] "transpose_VCF_data.html"
## [479] "transpose_VCF_data.Rmd"
## [480] "Untitled document.pdf"
## [481] "Vaccine Case Study.docx"
## [482] "VariableLab.java"
## [483] "VariableLab.java-2.html"
## [484] "VariableLab.java-3.html"
## [485] "VariableLab.java.html"
## [486] "vcfR_test.vcf"
## [487] "vcfR_test.vcf.gz"
## [488] "vegan_PCA_amino_acids-STUDENT-2.Rmd"
## [489] "vegan_PCA_amino_acids-STUDENT.Rmd"
## [490] "vegan_pca_with_msleep-STUDENT.html"
## [491] "vegan_pca_with_msleep-STUDENT.Rmd"
## [492] "visit.ics"
## [493] "Vocab List.docx"
## [494] "walsh2017morphology-2.csv"
## [495] "walsh2017morphology.csv"
## [496] "WeeklySchedule_Fall2021v1.docx"
## [497] "Welcome Week Schedule of Events.pdf"
## [498] "Work Cited.pdf"
## [499] "working_directory_practice.Rmd"
## [500] "Zoom.pkg"
## [501] "선물 아이디어 final.jpg"
## [502] "선물 아이디어 final.pptx"
## [503] "카톡.jpeg"# call list.files(pattern = "vcf")
list.files(pattern = "vcf") #TODO## [1] "11.21443531-21683531.ALL.chr11_GRCh38.genotypes.20170504.vcf"
## [2] "all_loci-1.vcf"
## [3] "all_loci.vcf"
## [4] "ALL.chr11_GRCh38.genotypes.20170504.vcf"
## [5] "code_checkpoint_vcfR.html"
## [6] "code_checkpoint_vcfR.Rmd"
## [7] "vcfR_test.vcf"
## [8] "vcfR_test.vcf.gz"warning("Friendly remindeR: Make sure you have set your working directory")## Warning: Friendly remindeR: Make sure you have set your working directoryFourth, load the .vcf file.
# call vcfR::read.vcfR()
bird_snps_again <- vcfR::read.vcfR("all_loci.vcf",
convertNA = T) #TODO## Scanning file to determine attributes.
## File attributes:
## meta lines: 8
## header_line: 9
## variant count: 1929
## column count: 81
##
Meta line 8 read in.
## All meta lines processed.
## gt matrix initialized.
## Character matrix gt created.
## Character matrix gt rows: 1929
## Character matrix gt cols: 81
## skip: 0
## nrows: 1929
## row_num: 0
##
Processed variant 1000
Processed variant: 1929
## All variants processedwarning("RemindeR: If this didn't work, you may not have set your working directory to the location of the vcf file")## Warning: RemindeR: If this didn't work, you may not have set your working
## directory to the location of the vcf fileFifth, convert to numeric genotypic score (counts of the number of
the minor allele) with vcfR::extract.gt().
# call vcfR::extract.gt()
bird_snps_num_again <- vcfR::extract.gt(bird_snps_again, # TODO
element = "GT",
IDtoRowNames = F,
as.numeric = T,
convertNA = T)Sixth, transpose the data into the format that works with R with
t().
# call t() on bird_snps_num_again
bird_snps_num_t_again <- t(bird_snps_num_again) # TODOSeventh, remove the invariant columns with
invar_omit().
# call invar_omit() on bird_snps_num_t_again
bird_snps_no_invar <- invar_omit(bird_snps_num_t_again) #TODO## Dataframe of dim 72 1929 processed...
## 590 columns removedCompare the original and the new dfs
# call dim() on bird_snps_num_t_again
dim(bird_snps_num_t_again) #TODO## [1] 72 1929# call dim() on bird_snps_no_invar
dim(bird_snps_no_invar) #TODO## [1] 72 1339These data have a lot of NAs. If we just call na.omit()
on them, what happens? Call na.omit() on the
bird_snps_num_t_again object, then check the
dimensions.
# Call na.omit() on bird_snps_num_t
## and assign the output to no_NAs
no_NAs <- na.omit(bird_snps_num_t_again) # TODO
# what is the remaining size of the data?
# why
dim(no_NAs)## [1] 0 1929