Initial Load and Screen

Prior to loading in the data, I read through the surveys to make sure I understood the levels depicted in the data. I then load my libraries, set my directory, and load in the data.

#library(stargazer)
library(tidyverse)
library(shiny)

#set working directory and load in data
setwd("/Users/cassandrabayer/Downloads")
pre <- read.csv("Fall Pre_data.csv")
post <- read.csv("Spring Post_data.csv")

Merge

Join both datasets together using an inner join to get all rows of X (pre) that have a matching IDs in Y (post) but also will return all columnns from X and Y (pre and post, respectively).

I check to make sure the merge is successful by first adding the columns (I should have all 64 from pre and 40 from post, this should give me a dataframe of 103 columns, as the merge will share the ID column).

prepost_inner <- inner_join(pre,post, by="ID") 
#str(prepost_inner) #check variable types-- all integers or numeric, which is good for now
#browse the dataframe using View() and make sure there are not overwhelming numbers of NAs where they shouldn't be

#check for consistency (values should be 1 through 4 with the exception of a couple, like year or month)
#stargazer(prepost_inner, type="html", summary.logical=FALSE) 

Quick Clean

Most questions are on a ordinal scale from 1-4, with the exception of questions regarding grade or birth year/month. It’s easy to spot (using the summary function) where the odd ball numbers are (there are a couple). I replace random high values with NAs, especially for the Likert questions (1-4), rather than drop them from the set.

*Another alternative would be to get complete cases, but we’d lose too much data in this dataframe.

*In cases like this data set where there are many NAs, we could also impute using the mean, but the data set is so small that I’m not sure I trust the mean.

prepost_inner[ prepost_inner >=1999 ] <- NA #the highest intentional value that I saw
#stargazer(prepost_inner, type = "html") #do another check, looks good but I have some questions
#Some queries: why is NativeHawaiin.y logical? Are there really no respondents with Novemeber or December birthdays?

Quick Overview

There was a small subset of students who took both the pre and post surveys, and even fewer completed it in its entirety. We find that only 31 students took both pre and post surveys, 19 of which were exicted about science before starting (using the Science_Excite.x variable) and 18 were excited about it after the program (using Science_Excite.y); that’s 61% and 58% of the sample, repspectively.

#31 students took both pre and post survey
length(prepost_inner$ID)

## [1] 31

length(which(prepost_inner$Science_Excited.x >= 3)) 

## [1] 19

length(which(prepost_inner$Science_Excited.y >= 3)) 

## [1] 18

Sixth grade males

I made a separate subsetted dataframe just for ease of calculation. Here, we see that there are only 2 sixth grade males in the sample; only one of which thought was boring prior to the program (50% using a simple Boolean function). While we can be certain that one did not find science boring in the post period, the other respondent’s answer was missing. If you pull out that missing values (na.rm=TRUE), there was no sixth grade males who found science boring after the program.

sixth_male <- prepost_inner %>% filter(Grade.x==6 & Gender.x==2) 

sum(sixth_male$Science_Boring.x>=3) 

## [1] 1

sum(sixth_male$Science_Boring.y>=3) 

## [1] NA

sum(sixth_male$Science_Boring.y>=3, na.rm= TRUE) 

## [1] 0

Significance of Excitement on Hours Studying

To find if there was a signfificant bearing on how long students did science in the week and their level of agreement that science is something to be excited about,I performed a linear regression. I wanted to see what proportion, if any, of the number of hours devoted to science was explained by excitement for science. I found that the relationship was not signficant.

lm <- lm(HoursperWeek.y ~ Science_Excited.y, data=prepost_inner)
#Not statistically significant 

Double Checking

As a gut check, I broke the dataframe into a smaller subset with just the variables of interest (vars). I looked at the simple average difference between excitement and hours studied between pre and post (diffexcite and diffhours); the difference was not immense (about 1/3 point increase in excite post-program and a 20-minute increase in studying). I then used an ANOVA test to compare the mean hours studied between the four levels of excitement, and again found no statistically signficant relationship. As a final point of interest, I grouped the smaller science dataframe by each level of excitement’s respective mean hours of studying.

vars <- c("ID", "Gender.x", "Grade.x", "Science_Excited.x", "Science_Excited.y", "HoursperWeek.x", "HoursperWeek.y")
science <- prepost_inner[,vars]

science$diffexcite <- science$Science_Excited.y-science$Science_Excited.x
science$diffhours <- science$HoursperWeek.y-science$HoursperWeek.x
mean(science$diffexcite, na.rm=TRUE)

## [1] 0.3333333

mean(science$diffhours, na.rm=TRUE)

## [1] 0.3478261

#test for difference using ANOVA where H0 <- L1 = L2 = L3 = L4 (Ls represent each different level of excitement). The null hypothesis claims that these are all the same and that the excitement about science effectively has no bearing on the hours per week spent studying.

aov <- aov(science$HoursperWeek.y~science$Science_Excited.y)
summary(aov)

##                           Df Sum Sq Mean Sq F value Pr(>F)
## science$Science_Excited.y  1  0.225  0.2253   0.264  0.612
## Residuals                 23 19.615  0.8528               
## 6 observations deleted due to missingness

#again not a significant relationship. The F Stat is far too small and the P-value is huge, which tells us it's very likely we're seeing these results purely by chance. We fail to reject the null hypothesis-- as far as we know, the hours spent studying across kids with varying levels of excitment are all the same.

Triple Checking

Get mean for each level of excitement in the post period for a gut check. Oddly enough, the students most excited about science studied the least, which could either be showing that students with a proclivity for science study less or that the data is insufficient/noisy.

science_excite <- group_by(science, Science_Excited.y) %>%
 summarise(mean_hours_study=mean(HoursperWeek.y,na.rm=TRUE),count=n())
science_excite <- science_excite[-(5),] #drop the NA

head(science_excite)

## # A tibble: 4 × 3
##   Science_Excited.y mean_hours_study count
##               <int>            <dbl> <int>
## 1                 1         2.000000     4
## 2                 2         2.166667     8
## 3                 3         2.500000     8
## 4                 4         1.777778    10

Boys versus Girls

I selected a handful of variables to look at means in pre and post data between boys and girls; I use the mean given the small sample size/lack of outliers in the data. I chose a blend of variables that poignantly target liking or understanding science, and questions that are more cloaked; something similar to a proxy. I would have loved to use the Confidence metric, but I found it as an improper ground for comparison given it was not in the pre-survey. I found that greatest difference was in the level of exictement between boys and girls: There was a mean increase of 1 whole point for boths, but a decrease of 1/3 point for girls; there were similar results for boys’ and girls’ desire to visit museums.

However, there was a proportionately larger effect in girls for a desire to understand science: Boys’ interest remained stagnant, but girls’ desire to understand science raised by 1/3 a point on average. It seems that science piqued boys excitement (about the subject and about museums), but it also seemed to increase girls’ interest in learning more despite it not seeming to make them feel more successful or excited about it.

gender_diff <- group_by(prepost_inner, Gender.x) %>%
 summarise(diff_hours=mean(HoursperWeek.y,na.rm=TRUE)- #na.rms remove the missing values so they don't influence the mean
          mean(HoursperWeek.x,na.rm=TRUE),
          
          diff_excite=mean(Science_Excited.y,na.rm=TRUE)-
           mean(Science_Excited.x,na.rm=TRUE),
          
           diff_take_apart=mean(TakeApart_Learn.y,na.rm=TRUE)-
          mean(TakeApart_Learn.x,na.rm=TRUE),
            
          diff_boring=mean(Science_Boring.y,na.rm=TRUE)-
          mean(Science_Boring.x,na.rm=TRUE),
        
          diff_museum=mean(Enjoy_Museums.y,na.rm=TRUE)-
          mean(Enjoy_Museums.x,na.rm=TRUE),
          
         diff_understand=mean(Understand_Science.y,na.rm=TRUE)-
          mean(Understand_Science.x,na.rm=TRUE),
         
         diff_like_school= mean(Like_Scienceatschool.y,na.rm=TRUE)- 
         mean(Like_Scienceatschool.x,na.rm=TRUE), 
         
         diff_like_after= mean(Like_Scienceoutsideschool.y,na.rm=TRUE)-
        mean(Like_Scienceoutsideschool.x,na.rm=TRUE),
           
           count=n())
gender_diff <- gender_diff[-c(3),] 
#gender_diff
#drop NA category that comes from some not stating their gender either in the pre or post

Post-Program Questions

Out of curiousity, I wanted to see just the post survey questions between boys and girls. It seems that the program made science more fun and interesting for girls than for boys, but made boys feel like they were better students. I wonder if the improvement in boys’ attitudes regarding their performance is because boys’ disporoportionatey feel successful in science and the program allocated more time to a subject they feel comfortable with.

gender_1 <- prepost_inner %>% filter(Gender.x==1)
gender_1 <- gender_1[,c(89:103)]
#summary(gender_1)
gender_2 <- prepost_inner %>% filter(Gender.x==2)
gender_2 <- gender_2[,c(89:103)]
#summary(gender_2)

Questions for Future Inspection

*With unlimited time, I would have liked to explore the relationships within grades and/or race using a fixed-effect model or a two-way fixed effects model. I believe that there may be important systematic differences between these groups that may be lost when all ages and races are being blended together.

*I would also be interested to see how the program differentially affects groups that identify themselves as someone who enjoys or dislikes science. While the program may not affect students who already like science, it could be disparately affecting students who do not. I’d like to explore correlatory relationships between post-performance and existing involvement in after-school programs. It would fascinate me to know it that negatively or positively impacts their perspective on science.

*Lastly, I would like to look closely at students that are systematically not responding to questions in the survey by looking at both pre and post data. I’d like to investigate why they are not answering questions, whether there is a way to engage them more in the survey process, or if there is a common misunderstanding by all students of particular questions.

*The data as it stands could use bolstering, so I’d like to look closely at this sample to challenge whether the next data collection effort could be more holistic.

Visual

One interesting finding what how the program was received differently by boys and girls. On one hand, boys who were interested in science prior to the program had increased interest following the program. Yet, girls who had more interest prior to the program were less interested after the program.

prepost_inner <- prepost_inner %>% 
        mutate(gender_factor = as.factor(Gender.x))

scatter_plot<- prepost_inner %>% 
      ggplot(  aes(Curious_Science.x, Curious_Science.y, color=gender_factor))+ 
  
      geom_jitter(size = 2, aes(color= gender_factor), alpha = 1) +
      xlab("Pre-Interest in Science") + 
  ylab("Post-Interest in Science" ) + 
      ggtitle("Interest in Science Pre- Program Has Opposite Effects on Boys and Girls") +
    scale_color_manual(values=c("#FF0000", "#00A08A", "#F2AD00", "#5BBCD6"),
                         guide = guide_legend(title = 'Gender')) +
  scale_x_continuous(breaks= seq(0,180, 15)) +
        theme_minimal() + geom_smooth(se=FALSE) 

scatter_plot

###