Examing Image Saliency

The following csv file was created using MATLAB. Functions from the Saliency Toolbox and Image Processing Toolbox were used to generate the image salience means.

First, images were masked using BMP files created in photoshop. Then, saliency maps for each FAT and TAT ad was then calculated using the Saliency Toolbox. Finally, images were avgeraged using the mean2 function in matlab, yeilding a single mean salience value per ad. Diffmean variable corresponds to an image subtraction from matlab- but is not a true difference score because all values are forced positive. SimpleDiffmean is used in following calculations.

Let’s take a quick peek at the data:

library(ggplot2)
library(Hmisc)
imstats = read.csv('C:/Program Files/MATLAB/R2015a/work/SaliencyToolbox/img/CERT/OUTPUT/Stats/stats.csv')
head(imstats)
##      FATmean    TATmean MaskedFATmean MaskedTATmean MaskedDiffmean
## 1 0.01383585 0.01248438   0.001966085  0.0023241115  -0.0003580270
## 2 0.03879425 0.04233262   0.001328669  0.0112492953  -0.0099206265
## 3 0.03384343 0.03663632   0.005594294  0.0114992494  -0.0059049556
## 4 0.03826662 0.03786856   0.003083195  0.0020726333   0.0010105622
## 5 0.03705499 0.03605179   0.001044799  0.0000488511   0.0009959482
## 6 0.04232278 0.03830447   0.001439338  0.0001690104   0.0012703278
##   MaskedSimpleDiffmean         Scene        SceneCheck1        SceneCheck2
## 1        -2.050781e-04 s10Bitmap.bmp s10fatSaliency.png s10tatSaliency.png
## 2        -9.478516e-03 s11Bitmap.bmp s11fatSaliency.png s11tatSaliency.png
## 3        -5.833984e-03 s12Bitmap.bmp s12fatSaliency.png s12tatSaliency.png
## 4         1.134766e-03 s13Bitmap.bmp s13fatSaliency.png s13tatSaliency.png
## 5         5.078125e-05 s14Bitmap.bmp s14fatSaliency.png s14tatSaliency.png
## 6         1.158203e-03 s15Bitmap.bmp s15fatSaliency.png s15tatSaliency.png
describe(imstats)
## imstats 
## 
##  9  Variables      28  Observations
## ---------------------------------------------------------------------------
## FATmean 
##        n  missing distinct     Info     Mean      Gmd      .05      .10 
##       28        0       28        1  0.03334   0.0108  0.02049  0.02116 
##      .25      .50      .75      .90      .95 
##  0.02659  0.03404  0.03897  0.04508  0.04709 
## 
## lowest : 0.01383585 0.02027482 0.02088645 0.02128307 0.02317682
## highest: 0.04353359 0.04448992 0.04646392 0.04742719 0.05200902
## ---------------------------------------------------------------------------
## TATmean 
##        n  missing distinct     Info     Mean      Gmd      .05      .10 
##       28        0       28        1  0.03203 0.009873  0.02108  0.02208 
##      .25      .50      .75      .90      .95 
##  0.02615  0.03181  0.03694  0.04258  0.04513 
## 
## lowest : 0.01248438 0.02106310 0.02109986 0.02250679 0.02286701
## highest: 0.04163443 0.04233262 0.04315051 0.04620134 0.04985698
## ---------------------------------------------------------------------------
## MaskedFATmean 
##         n   missing  distinct      Info      Mean       Gmd       .05 
##        28         0        27         1  0.003678   0.00449 7.104e-06 
##       .10       .25       .50       .75       .90       .95 
## 3.533e-05 8.214e-04 1.939e-03 5.216e-03 9.492e-03 1.153e-02 
## 
## lowest : 0.000000e+00 2.029718e-05 4.177390e-05 9.453891e-05 1.249081e-04
## highest: 8.190755e-03 9.089813e-03 1.043203e-02 1.212851e-02 1.827299e-02
## ---------------------------------------------------------------------------
## MaskedTATmean 
##         n   missing  distinct      Info      Mean       Gmd       .05 
##        28         0        24     0.995  0.004758  0.006228 0.0000000 
##       .10       .25       .50       .75       .90       .95 
## 0.0000000 0.0000768 0.0021984 0.0096857 0.0135341 0.0152675 
## 
## lowest : 0.000000e+00 2.212776e-05 4.885110e-05 8.612132e-05 1.690104e-04
## highest: 1.174646e-02 1.317685e-02 1.436775e-02 1.575195e-02 1.767702e-02
## ---------------------------------------------------------------------------
## MaskedDiffmean 
##          n    missing   distinct       Info       Mean        Gmd 
##         28          0         27          1  -0.001079   0.004159 
##        .05        .10        .25        .50        .75        .90 
## -0.0082328 -0.0054660 -0.0030115 -0.0001790  0.0009996  0.0030579 
##        .95 
##  0.0044271 
## 
## lowest : -0.009920627 -0.009486267 -0.005904956 -0.005277941 -0.004793926
## highest:  0.002411780  0.002799188  0.003661474  0.004839292  0.005096147
## ---------------------------------------------------------------------------
## MaskedSimpleDiffmean 
##          n    missing   distinct       Info       Mean        Gmd 
##         28          0         23       0.99  -0.001226   0.004112 
##        .05        .10        .25        .50        .75        .90 
## -0.0082029 -0.0055154 -0.0030747  0.0000000  0.0006997  0.0027049 
##        .95 
##  0.0042216 
## 
## lowest : -0.010494141 -0.009478516 -0.005833984 -0.005378906 -0.004943359
## highest:  0.002097656  0.002410156  0.003392578  0.004667969  0.004841797
## ---------------------------------------------------------------------------
## Scene 
##        n  missing distinct 
##       28        0       28 
## 
## lowest : s10Bitmap.bmp s11Bitmap.bmp s12Bitmap.bmp s13Bitmap.bmp s14Bitmap.bmp
## highest: s5bitmap.bmp  s6Bitmap.bmp  s7Bitmap.bmp  s8bitmap.bmp  s9bitmap.bmp 
## ---------------------------------------------------------------------------
## SceneCheck1 
##        n  missing distinct 
##       28        0       28 
## 
## lowest : s10fatSaliency.png s11fatSaliency.png s12fatSaliency.png s13fatSaliency.png s14fatSaliency.png
## highest: s5fatSaliency.png  s6fatSaliency.png  s7fatSaliency.png  s8fatSaliency.png  s9fatSaliency.png 
## ---------------------------------------------------------------------------
## SceneCheck2 
##        n  missing distinct 
##       28        0       28 
## 
## lowest : s10tatSaliency.png s11tatSaliency.png s12tatSaliency.png s13tatSaliency.png s14tatSaliency.png
## highest: s5tatSaliency.png  s6tatSaliency.png  s7tatSaliency.png  s8tatSaliency.png  s9tatSaliency.png 
## ---------------------------------------------------------------------------

Testing differences in mean in full image

Is there a difference between FAT and TAT mean saliency scores for the entire scene? What about difference scores and 0?

t.test(imstats$FATmean, imstats$TATmean, paired= T)
## 
##  Paired t-test
## 
## data:  imstats$FATmean and imstats$TATmean
## t = 1.2282, df = 27, p-value = 0.23
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -0.000877162  0.003493468
## sample estimates:
## mean of the differences 
##             0.001308153
imstats$FATminusTAT <- (imstats$FATmean - imstats$TATmean)
t.test(imstats$FATminusTAT, mu=0)
## 
##  One Sample t-test
## 
## data:  imstats$FATminusTAT
## t = 1.2282, df = 27, p-value = 0.23
## alternative hypothesis: true mean is not equal to 0
## 95 percent confidence interval:
##  -0.000877162  0.003493468
## sample estimates:
##   mean of x 
## 0.001308153

Testing differences in mean in masked image

Ok, let’s take a look at the masked images only, to see if there are any significant effects within the ad region specifically.

t.test(imstats$MaskedFATmean, imstats$MaskedTATmean, paired= T)
## 
##  Paired t-test
## 
## data:  imstats$MaskedFATmean and imstats$MaskedTATmean
## t = -1.5367, df = 27, p-value = 0.136
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -0.0025204129  0.0003618274
## sample estimates:
## mean of the differences 
##            -0.001079293
imstats$MaskedDiffmean <- (imstats$MaskedFATmean- imstats$MaskedTATmean)

t.test(imstats$MaskedDiffmean, mu= 0)
## 
##  One Sample t-test
## 
## data:  imstats$MaskedDiffmean
## t = -1.5367, df = 27, p-value = 0.136
## alternative hypothesis: true mean is not equal to 0
## 95 percent confidence interval:
##  -0.0025204129  0.0003618274
## sample estimates:
##    mean of x 
## -0.001079293

No significant difference between FAT and TAT scores.

Now lets plot this difference in the form of a bar graph Before we plot lets create a few helper fuctions

## Gives count, mean, standard deviation, standard error of the mean, and confidence interval (default 95%).
##   data: a data frame.
##   measurevar: the name of a column that contains the variable to be summariezed
##   groupvars: a vector containing names of columns that contain grouping variables
##   na.rm: a boolean that indicates whether to ignore NA's
##   conf.interval: the percent range of the confidence interval (default is 95%)
summarySE <- function(data=NULL, measurevar, groupvars=NULL, na.rm=FALSE,
                      conf.interval=.95, .drop=TRUE) {
    library(plyr)

    # New version of length which can handle NA's: if na.rm==T, don't count them
    length2 <- function (x, na.rm=FALSE) {
        if (na.rm) sum(!is.na(x))
        else       length(x)
    }

    # This does the summary. For each group's data frame, return a vector with
    # N, mean, and sd
    datac <- ddply(data, groupvars, .drop=.drop,
      .fun = function(xx, col) {
        c(N    = length2(xx[[col]], na.rm=na.rm),
          mean = mean   (xx[[col]], na.rm=na.rm),
          sd   = sd     (xx[[col]], na.rm=na.rm)
        )
      },
      measurevar
    )

    # Rename the "mean" column    
    datac <- rename(datac, c("mean" = measurevar))

    datac$se <- datac$sd / sqrt(datac$N)  # Calculate standard error of the mean

    # Confidence interval multiplier for standard error
    # Calculate t-statistic for confidence interval: 
    # e.g., if conf.interval is .95, use .975 (above/below), and use df=N-1
    ciMult <- qt(conf.interval/2 + .5, datac$N-1)
    datac$ci <- datac$se * ciMult

    return(datac)
}

## Norms the data within specified groups in a data frame; it normalizes each
## subject (identified by idvar) so that they have the same mean, within each group
## specified by betweenvars.
##   data: a data frame.
##   idvar: the name of a column that identifies each subject (or matched subjects)
##   measurevar: the name of a column that contains the variable to be summariezed
##   betweenvars: a vector containing names of columns that are between-subjects variables
##   na.rm: a boolean that indicates whether to ignore NA's
normDataWithin <- function(data=NULL, idvar, measurevar, betweenvars=NULL,
                           na.rm=FALSE, .drop=TRUE) {
    library(plyr)

    # Measure var on left, idvar + between vars on right of formula.
    data.subjMean <- ddply(data, c(idvar, betweenvars), .drop=.drop,
     .fun = function(xx, col, na.rm) {
        c(subjMean = mean(xx[,col], na.rm=na.rm))
      },
      measurevar,
      na.rm
    )

    # Put the subject means with original data
    data <- merge(data, data.subjMean)

    # Get the normalized data in a new column
    measureNormedVar <- paste(measurevar, "_norm", sep="")
    data[,measureNormedVar] <- data[,measurevar] - data[,"subjMean"] +
                               mean(data[,measurevar], na.rm=na.rm)

    # Remove this subject mean column
    data$subjMean <- NULL

    return(data)
}

## Summarizes data, handling within-subjects variables by removing inter-subject variability.
## It will still work if there are no within-S variables.
## Gives count, un-normed mean, normed mean (with same between-group mean),
##   standard deviation, standard error of the mean, and confidence interval.
## If there are within-subject variables, calculate adjusted values using method from Morey (2008).
##   data: a data frame.
##   measurevar: the name of a column that contains the variable to be summariezed
##   betweenvars: a vector containing names of columns that are between-subjects variables
##   withinvars: a vector containing names of columns that are within-subjects variables
##   idvar: the name of a column that identifies each subject (or matched subjects)
##   na.rm: a boolean that indicates whether to ignore NA's
##   conf.interval: the percent range of the confidence interval (default is 95%)
summarySEwithin <- function(data=NULL, measurevar, betweenvars=NULL, withinvars=NULL,
                            idvar=NULL, na.rm=FALSE, conf.interval=.95, .drop=TRUE) {

  # Ensure that the betweenvars and withinvars are factors
  factorvars <- vapply(data[, c(betweenvars, withinvars), drop=FALSE],
    FUN=is.factor, FUN.VALUE=logical(1))

  if (!all(factorvars)) {
    nonfactorvars <- names(factorvars)[!factorvars]
    message("Automatically converting the following non-factors to factors: ",
            paste(nonfactorvars, collapse = ", "))
    data[nonfactorvars] <- lapply(data[nonfactorvars], factor)
  }

  # Get the means from the un-normed data
  datac <- summarySE(data, measurevar, groupvars=c(betweenvars, withinvars),
                     na.rm=na.rm, conf.interval=conf.interval, .drop=.drop)

  # Drop all the unused columns (these will be calculated with normed data)
  datac$sd <- NULL
  datac$se <- NULL
  datac$ci <- NULL

  # Norm each subject's data
  ndata <- normDataWithin(data, idvar, measurevar, betweenvars, na.rm, .drop=.drop)

  # This is the name of the new column
  measurevar_n <- paste(measurevar, "_norm", sep="")

  # Collapse the normed data - now we can treat between and within vars the same
  ndatac <- summarySE(ndata, measurevar_n, groupvars=c(betweenvars, withinvars),
                      na.rm=na.rm, conf.interval=conf.interval, .drop=.drop)

  # Apply correction from Morey (2008) to the standard error and confidence interval
  #  Get the product of the number of conditions of within-S variables
  nWithinGroups    <- prod(vapply(ndatac[,withinvars, drop=FALSE], FUN=nlevels,
                           FUN.VALUE=numeric(1)))
  correctionFactor <- sqrt( nWithinGroups / (nWithinGroups-1) )

  # Apply the correction factor
  ndatac$sd <- ndatac$sd * correctionFactor
  ndatac$se <- ndatac$se * correctionFactor
  ndatac$ci <- ndatac$ci * correctionFactor

  # Combine the un-normed means with the normed results
  merge(datac, ndatac)
}

Ok now we are ready

library(ggplot2)


#Load in the data in long format
statsLong = read.csv('C:/Users/londeree.4/Dropbox/Lab-Londeree/osu-certsproject/5-ExperimentEyetracking/Analysis/Low-Level ImStats/statsLong.csv')


statsBar <- summarySE(statsLong, measurevar="Maskedmean", groupvars=c("imageType"))
## Warning: package 'plyr' was built under R version 3.2.5
## 
## Attaching package: 'plyr'
## The following objects are masked from 'package:Hmisc':
## 
##     is.discrete, summarize
ggplot(statsBar, aes(x=imageType, y=Maskedmean, fill=imageType)) + 
    geom_bar(position=position_dodge(), stat="identity",
             colour="black", # Use black outlines,
             size=.3) +      # Thinner lines
    geom_errorbar(aes(ymin=Maskedmean-se, ymax=Maskedmean+se),
                  size=.3,    # Thinner lines
                  width=.2,
                  position=position_dodge(.9)) +
    xlab("Image Type") +
    ylab("Mean Salience") +
    ggtitle("Differences in Salience between Flavored and Unflavored Ads") +
    scale_y_continuous(breaks=0:20*4) +
    theme_bw()

statsWS <- summarySEwithin(statsLong, measurevar="Maskedmean", withinvars="imageType",
                        idvar="SceneNum", na.rm=FALSE, conf.interval=.95)

# Make the graph with the 95% confidence interval
ggplot(statsWS, aes(x=imageType, y=Maskedmean, group=1)) +
    geom_line() +
    geom_errorbar(width=.1, aes(ymin=Maskedmean-ci, ymax=Maskedmean+ci)) +
    geom_point(shape=21, size=3, fill="white") +
    ylim(0,.01)+theme_bw() +
    xlab("Image Type") +
    ylab("Mean Salience") +
    ggtitle("Differences in Salience between Flavored and Unflavored Ads") 

# Use a consistent y range
ymax <- max(statsLong$Maskedmean)
ymin <- min(statsLong$Maskedmean)

# Plot the individuals
ggplot(statsLong, aes(x=imageType, y=Maskedmean, colour=SceneNum, group=SceneNum)) +
    geom_line(size = 1.5) + geom_point(shape=21, fill="white") + 
    ylim(ymin,ymax)+ 
    scale_colour_gradientn(colours=rainbow(4))+
    theme_bw()+
    xlab("Image Type") +
    ylab("Mean Salience") +
    ggtitle("Differences in Salience between Flavored and Unflavored Ads Per Scene")+
    labs(colour = "Scene Pair")

Correlation between mean FAT and TAT salience scores in whole and Masked Images

Look at the relationship between FAT and TAT saliency scores.

## 
##  Pearson's product-moment correlation
## 
## data:  imstats$MaskedFATmean and imstats$MaskedTATmean
## t = 6.1388, df = 26, p-value = 1.723e-06
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  0.5556268 0.8875939
## sample estimates:
##       cor 
## 0.7692456

## 
##  Pearson's product-moment correlation
## 
## data:  imstats$FATmean and imstats$TATmean
## t = 6.9004, df = 26, p-value = 2.516e-07
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  0.6160011 0.9056027
## sample estimates:
##       cor 
## 0.8042489

Both full and masked images are tightly correlated. Although in masked images several TAT images are floating around 0, the mean of TAT images is still higher (note axis scaling).

Histogram of difference scores for Masked and Full images

hist(imstats$FATminusTAT, main = "Differences In Full Image Means (FAT-TAT)")

hist(imstats$MaskedDiffmean, main = "Masked Differences In Means (FAT-TAT)")

Most difference scores are centered around 0. Differences in magnitude are due to differences in number of 0 values being divided over (Masked means are still calculated by dividing over entire image).

Summary of Saliency Toolbox Findings

On this past, it is safe to say that there is no significant difference in salience between ad type.

Using the SHINE toolbox to look at low level visual features

We can also examine low level visual features using the SHINE toolbox. Image stats were generated for fat and tat ads and output into the following files:

  1. stats.histMat: luminance distributions for each image (one column per image)
  2. stats.meanVec: mean luminance for each image
  3. stats.stdVec: contrast (stdev) for each image
  4. stats.meanHist: the average histogram
  5. stats.meanLum: the average luminance across all images
  6. stats.meanStd: the average contrast (standard deviation) across all images

From this information we will test two main claims. That both mean luminace and contrast is the same between FAT and TAT scenes.

  • Luminance is a photometric measure of the luminous intensity per unit area of light travelling in a given direction. In our case, pixels.

  • Contrast is the difference in luminance that makes an object (or its representation in an image or display) distinguishable. In visual perception of the real world, contrast is determined by the difference in the color and brightness of the object and other objects within the same field of view.

Note that the following calculations are not from masked images, but the entire image set

Examining meanVec (Mean Luminance of Each Image)

fatlumstats = read.csv('C:/Program Files/MATLAB/R2015a/work/SHINEtoolbox/SHINE_OUTPUT/FAT/fatstats.meanVec.csv', header=F)
colnames(fatlumstats) <- "FATMeanLum"
tatlumstats = read.csv('C:/Program Files/MATLAB/R2015a/work/SHINEtoolbox/SHINE_OUTPUT/TAT/tatstats.meanVec.csv', header=F)
colnames(tatlumstats) <- "TATMeanLum"

describe(fatlumstats)
## fatlumstats 
## 
##  1  Variables      28  Observations
## ---------------------------------------------------------------------------
## FATMeanLum 
##        n  missing distinct     Info     Mean      Gmd      .05      .10 
##       28        0       28        1    111.9    22.49    75.08    86.50 
##      .25      .50      .75      .90      .95 
##    99.63   114.45   125.63   133.86   139.37 
## 
## lowest :  72.294  73.591  77.856  90.201  92.574
## highest: 129.360 132.840 136.240 141.050 147.470
## ---------------------------------------------------------------------------
describe(tatlumstats)
## tatlumstats 
## 
##  1  Variables      28  Observations
## ---------------------------------------------------------------------------
## TATMeanLum 
##        n  missing distinct     Info     Mean      Gmd      .05      .10 
##       28        0       28        1    112.2    22.58    75.80    87.58 
##      .25      .50      .75      .90      .95 
##    99.81   114.25   125.83   133.62   139.40 
## 
## lowest :  73.004  74.701  77.837  91.751  93.402
## highest: 130.170 132.490 136.240 141.100 151.400
## ---------------------------------------------------------------------------
t.test(fatlumstats$FATMeanLum, tatlumstats$TATMeanLum, paired= T)
## 
##  Paired t-test
## 
## data:  fatlumstats$FATMeanLum and tatlumstats$TATMeanLum
## t = -0.85904, df = 27, p-value = 0.3979
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -0.9029198  0.3699912
## sample estimates:
## mean of the differences 
##              -0.2664643

Let’s also examine a difference score as a sainity check

diffstats <- data.frame(matrix(ncol = 1,nrow= 28))
diffstats$meanVecDiff <- (fatlumstats$FATMeanLum-tatlumstats$TATMeanLum)
t.test(diffstats$meanVecDiff, mu= 0)
## 
##  One Sample t-test
## 
## data:  diffstats$meanVecDiff
## t = -0.85904, df = 27, p-value = 0.3979
## alternative hypothesis: true mean is not equal to 0
## 95 percent confidence interval:
##  -0.9029198  0.3699912
## sample estimates:
##  mean of x 
## -0.2664643

Are luminace values correlated in scene pairs?

cor.test(fatlumstats$FATMeanLum, tatlumstats$TATMeanLum)
## 
##  Pearson's product-moment correlation
## 
## data:  fatlumstats$FATMeanLum and tatlumstats$TATMeanLum
## t = 60.779, df = 26, p-value < 2.2e-16
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  0.9923487 0.9984001
## sample estimates:
##       cor 
## 0.9964993
r <- round(cor(fatlumstats$FATMeanLum, tatlumstats$TATMeanLum), 4)
r<- paste('r =',r)
p <- round(cor.test(fatlumstats$FATMeanLum, tatlumstats$TATMeanLum)$p.value, 4)
p <- paste('p =',p)
qplot(fatlumstats$FATMeanLum, tatlumstats$TATMeanLum, main= "Full scene Luminance Pairs") + geom_smooth(method = 'lm')+ annotate('text', 80,80,label= r)+annotate('text', 80,83,label= p)

Examining stdVec (contrast (stdev) for each image)

fatstdVecstats = read.csv('C:/Program Files/MATLAB/R2015a/work/SHINEtoolbox/SHINE_OUTPUT/FAT/fatstats.stdVec.csv', header=F)
tatstdVecstats = read.csv('C:/Program Files/MATLAB/R2015a/work/SHINEtoolbox/SHINE_OUTPUT/TAT/tatstats.stdVec.csv', header=F)

describe(fatstdVecstats)
## fatstdVecstats 
## 
##  1  Variables      28  Observations
## ---------------------------------------------------------------------------
## V1 
##        n  missing distinct     Info     Mean      Gmd      .05      .10 
##       28        0       27        1    63.34     11.5    48.54    50.70 
##      .25      .50      .75      .90      .95 
##    57.72    63.25    70.43    74.07    75.91 
## 
## lowest : 34.649 48.081 49.394 51.264 55.239, highest: 72.788 73.514 75.369 76.203 83.424
## ---------------------------------------------------------------------------
describe(tatstdVecstats)
## tatstdVecstats 
## 
##  1  Variables      28  Observations
## ---------------------------------------------------------------------------
## V1 
##        n  missing distinct     Info     Mean      Gmd      .05      .10 
##       28        0       28        1    63.76    11.34    48.56    51.37 
##      .25      .50      .75      .90      .95 
##    58.39    63.44    70.20    74.22    76.24 
## 
## lowest : 34.800 48.417 48.822 52.463 54.364, highest: 72.788 73.641 75.559 76.605 83.017
## ---------------------------------------------------------------------------
t.test(fatstdVecstats$V1, tatstdVecstats$V1, paired = T)
## 
##  Paired t-test
## 
## data:  fatstdVecstats$V1 and tatstdVecstats$V1
## t = -1.9884, df = 27, p-value = 0.05699
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -0.83974791  0.01317648
## sample estimates:
## mean of the differences 
##              -0.4132857

No significant difference between fat and tat contrast scores, but this is too close for comfort.

Let’s plot to examine if they are correlated

cor.test(fatstdVecstats$V1, tatstdVecstats$V1)
## 
##  Pearson's product-moment correlation
## 
## data:  fatstdVecstats$V1 and tatstdVecstats$V1
## t = 47.211, df = 26, p-value < 2.2e-16
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  0.9873796 0.9973558
## sample estimates:
##       cor 
## 0.9942179
r <- round(cor(fatstdVecstats$V1, tatstdVecstats$V1), 4)
r<- paste('r =',r)
p <- round(cor.test(fatstdVecstats$V1, tatstdVecstats$V1)$p.value, 4)
p <- paste('p =',p)
qplot(fatstdVecstats$V1, tatstdVecstats$V1, main= "Full scene Contrast Pairs") + geom_smooth(method = 'lm')+ annotate('text', 80,80,label= r)+annotate('text', 80,83,label= p)

Is contrast for each image directly related to the mean luminance of that image?

cor.test(fatstdVecstats$V1, fatlumstats$FATMeanLum)
## 
##  Pearson's product-moment correlation
## 
## data:  fatstdVecstats$V1 and fatlumstats$FATMeanLum
## t = 3.4354, df = 26, p-value = 0.001998
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  0.2345778 0.7710913
## sample estimates:
##       cor 
## 0.5587549
r <- round(cor(fatstdVecstats$V1, fatlumstats$FATMeanLum), 4)
r<- paste('r =',r)
p <- round(cor.test(fatstdVecstats$V1, fatlumstats$FATMeanLum)$p.value, 4)
p <- paste('p =',p)
qplot(fatstdVecstats$V1, fatlumstats$FATMeanLum, main= "FAT scene Contrast-LuminPairs") + geom_smooth(method = 'lm')+ annotate('text', 80,80,label= r)+annotate('text', 80,85,label= p)

cor.test(tatstdVecstats$V1, tatlumstats$TATMeanLum)
## 
##  Pearson's product-moment correlation
## 
## data:  tatstdVecstats$V1 and tatlumstats$TATMeanLum
## t = 3.2661, df = 26, p-value = 0.003057
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  0.2081829 0.7595949
## sample estimates:
##       cor 
## 0.5393681
r <- round(cor(tatstdVecstats$V1, tatlumstats$TATMeanLum), 4)
r<- paste('r =',r)
p <- round(cor.test(tatstdVecstats$V1, tatlumstats$TATMeanLum)$p.value, 4)
p <- paste('p =',p)
qplot(tatstdVecstats$V1,tatlumstats$TATMeanLum, main= "TAT scene Contrast-LuminPairs") + geom_smooth(method = 'lm')+ annotate('text', 80,80,label= r)+annotate('text', 80,85,label= p)

Not entirely sure what to make of this, FAT and TAT plots are pretty similar (which makes sense) but not completely sure why luminace and contrast is correlated.

Examining meanHist (The average luminance histogram)

fathiststats = read.csv('C:/Program Files/MATLAB/R2015a/work/SHINEtoolbox/SHINE_OUTPUT/FAT/fatstats.meanHist.csv', header=F)
colnames(fathiststats) <- "FATMeanHist"
tathiststats = read.csv('C:/Program Files/MATLAB/R2015a/work/SHINEtoolbox/SHINE_OUTPUT/TAT/tatstats.meanHist.csv', header=F)
colnames(tathiststats) <- "TATMeanHist"

describe(fathiststats)
## fathiststats 
## 
##  1  Variables      256  Observations
## ---------------------------------------------------------------------------
## FATMeanHist 
##        n  missing distinct     Info     Mean      Gmd      .05      .10 
##      256        0      255        1     2000    749.1    895.1   1069.7 
##      .25      .50      .75      .90      .95 
##   1606.1   1990.1   2378.2   3052.2   3082.8 
## 
## lowest :  167.43  277.93  436.00  493.18  496.46
## highest: 3101.70 3107.30 3113.80 3116.50 3126.20
## ---------------------------------------------------------------------------
describe(tathiststats)
## tathiststats 
## 
##  1  Variables      256  Observations
## ---------------------------------------------------------------------------
## TATMeanHist 
##        n  missing distinct     Info     Mean      Gmd      .05      .10 
##      256        0      251        1     2000    731.3    899.5   1096.3 
##      .25      .50      .75      .90      .95 
##   1612.1   2032.8   2338.0   3068.0   3098.4 
## 
## lowest :  276.39  322.14  557.07  573.82  606.89
## highest: 3132.50 3134.60 3137.90 3141.70 3147.10
## ---------------------------------------------------------------------------
t.test(fathiststats$FATMeanHist, tathiststats$TATMeanHist, paired= T)
## 
##  Paired t-test
## 
## data:  fathiststats$FATMeanHist and tathiststats$TATMeanHist
## t = -0.00036377, df = 255, p-value = 0.9997
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -7.191353  7.188697
## sample estimates:
## mean of the differences 
##            -0.001328125

Let’s also examine a difference score This gives a sense of the variation in luminace values across ad type

diffstats <- data.frame(matrix(ncol = 1,nrow= 256))
diffstats$meanHistDiff <- (fathiststats$FATMeanHist- tathiststats$TATMeanHist)
hist(diffstats$meanHistDiff)

t.test(diffstats$meanHistDiff, mu= 0)
## 
##  One Sample t-test
## 
## data:  diffstats$meanHistDiff
## t = -0.00036377, df = 255, p-value = 0.9997
## alternative hypothesis: true mean is not equal to 0
## 95 percent confidence interval:
##  -7.191353  7.188697
## sample estimates:
##    mean of x 
## -0.001328125

Examining histMat (luminance distributions for each image (one column per image))

These are in a per image basis, let’s first create a difference score per image to check for ad-related differences in luminance. Wait a minute, this doesn’t make sense because all histograms average to 2000, forcing the mean difference to be 0 , SEE OUTPUT BELOW

fathistMatstats = read.csv('C:/Program Files/MATLAB/R2015a/work/SHINEtoolbox/SHINE_OUTPUT/FAT/fatstats.histmat.csv', header=F)
tathistMatstats = read.csv('C:/Program Files/MATLAB/R2015a/work/SHINEtoolbox/SHINE_OUTPUT/TAT/tatstats.histmat.csv', header=F)

summary(fathistMatstats)
##        V1             V2             V3             V4      
##  Min.   :  85   Min.   : 378   Min.   :  97   Min.   : 558  
##  1st Qu.:1198   1st Qu.:1167   1st Qu.:1034   1st Qu.:1484  
##  Median :2124   Median :1758   Median :1818   Median :1724  
##  Mean   :2000   Mean   :2000   Mean   :2000   Mean   :2000  
##  3rd Qu.:2706   3rd Qu.:2457   3rd Qu.:2757   3rd Qu.:2118  
##  Max.   :3661   Max.   :4297   Max.   :4965   Max.   :5035  
##        V5             V6               V7               V8      
##  Min.   : 735   Min.   : 116.0   Min.   :  79.0   Min.   :   4  
##  1st Qu.:1660   1st Qu.: 849.2   1st Qu.: 918.2   1st Qu.: 706  
##  Median :1782   Median :1902.0   Median :1786.5   Median :1696  
##  Mean   :2000   Mean   :2000.0   Mean   :2000.0   Mean   :2000  
##  3rd Qu.:2191   3rd Qu.:2646.8   3rd Qu.:2491.8   3rd Qu.:2776  
##  Max.   :4376   Max.   :6399.0   Max.   :7516.0   Max.   :7880  
##        V9               V10            V11            V12      
##  Min.   :    0.0   Min.   :  30   Min.   : 280   Min.   :  42  
##  1st Qu.:  527.8   1st Qu.: 692   1st Qu.:1579   1st Qu.: 486  
##  Median :  746.5   Median :1352   Median :2070   Median :1987  
##  Mean   : 2000.0   Mean   :2000   Mean   :2000   Mean   :2000  
##  3rd Qu.: 1697.2   3rd Qu.:3876   3rd Qu.:2318   3rd Qu.:3166  
##  Max.   :11436.0   Max.   :4541   Max.   :3505   Max.   :5200  
##       V13            V14              V15            V16      
##  Min.   :   1   Min.   :  39.0   Min.   : 143   Min.   :   3  
##  1st Qu.: 902   1st Qu.: 936.2   1st Qu.:1436   1st Qu.:1205  
##  Median :1744   Median :1425.0   Median :2022   Median :1712  
##  Mean   :2000   Mean   :2000.0   Mean   :2000   Mean   :2000  
##  3rd Qu.:2758   3rd Qu.:2670.2   3rd Qu.:2559   3rd Qu.:2992  
##  Max.   :5559   Max.   :6001.0   Max.   :6083   Max.   :3676  
##       V17              V18              V19            V20      
##  Min.   :   1.0   Min.   :   0.0   Min.   :   0   Min.   :  27  
##  1st Qu.: 638.5   1st Qu.: 269.0   1st Qu.:1376   1st Qu.:1305  
##  Median :1049.0   Median : 859.5   Median :2006   Median :2030  
##  Mean   :2000.0   Mean   :2000.0   Mean   :2000   Mean   :2000  
##  3rd Qu.:3534.8   3rd Qu.:3361.5   3rd Qu.:2636   3rd Qu.:2504  
##  Max.   :7043.0   Max.   :8875.0   Max.   :3945   Max.   :5385  
##       V21            V22            V23            V24        
##  Min.   :  87   Min.   : 305   Min.   : 113   Min.   :  28.0  
##  1st Qu.:1193   1st Qu.:1116   1st Qu.:1566   1st Qu.: 681.5  
##  Median :1590   Median :1487   Median :1787   Median :1590.0  
##  Mean   :2000   Mean   :2000   Mean   :2000   Mean   :2000.0  
##  3rd Qu.:2116   3rd Qu.:2142   3rd Qu.:2436   3rd Qu.:2735.8  
##  Max.   :9015   Max.   :8056   Max.   :4591   Max.   :5406.0  
##       V25            V26              V27            V28         
##  Min.   : 442   Min.   : 107.0   Min.   :  50   Min.   :   0.00  
##  1st Qu.:1215   1st Qu.: 808.2   1st Qu.:1454   1st Qu.:  58.75  
##  Median :1608   Median :1618.5   Median :2150   Median : 650.00  
##  Mean   :2000   Mean   :2000.0   Mean   :2000   Mean   :2000.00  
##  3rd Qu.:2130   3rd Qu.:2386.5   3rd Qu.:2610   3rd Qu.:3905.75  
##  Max.   :6196   Max.   :6679.0   Max.   :3518   Max.   :6611.00
summary(tathistMatstats)
##        V1             V2             V3             V4      
##  Min.   :  61   Min.   : 392   Min.   :  76   Min.   : 547  
##  1st Qu.:1200   1st Qu.:1199   1st Qu.:1068   1st Qu.:1477  
##  Median :2094   Median :1597   Median :1774   Median :1742  
##  Mean   :2000   Mean   :2000   Mean   :2000   Mean   :2000  
##  3rd Qu.:2694   3rd Qu.:2531   3rd Qu.:2850   3rd Qu.:2099  
##  Max.   :3629   Max.   :4408   Max.   :4910   Max.   :5080  
##        V5             V6               V7               V8        
##  Min.   :1130   Min.   : 118.0   Min.   :  58.0   Min.   :   3.0  
##  1st Qu.:1568   1st Qu.: 864.2   1st Qu.: 932.2   1st Qu.: 759.2  
##  Median :1770   Median :1890.0   Median :1834.5   Median :1660.0  
##  Mean   :2000   Mean   :2000.0   Mean   :2000.0   Mean   :2000.0  
##  3rd Qu.:2114   3rd Qu.:2619.8   3rd Qu.:2486.0   3rd Qu.:2758.5  
##  Max.   :5918   Max.   :6305.0   Max.   :7519.0   Max.   :7787.0  
##        V9               V10              V11            V12        
##  Min.   :    2.0   Min.   :   0.0   Min.   : 250   Min.   :   2.0  
##  1st Qu.:  531.8   1st Qu.: 656.5   1st Qu.:1598   1st Qu.: 435.5  
##  Median :  751.5   Median :1713.5   Median :2046   Median :2268.0  
##  Mean   : 2000.0   Mean   :2000.0   Mean   :2000   Mean   :2000.0  
##  3rd Qu.: 1669.0   3rd Qu.:3339.5   3rd Qu.:2329   3rd Qu.:3206.8  
##  Max.   :11518.0   Max.   :4804.0   Max.   :3462   Max.   :4540.0  
##       V13              V14              V15            V16      
##  Min.   :  47.0   Min.   :  94.0   Min.   :  96   Min.   :   4  
##  1st Qu.: 983.2   1st Qu.: 879.5   1st Qu.:1426   1st Qu.:1254  
##  Median :1650.0   Median :1522.5   Median :2048   Median :1712  
##  Mean   :2000.0   Mean   :2000.0   Mean   :2000   Mean   :2000  
##  3rd Qu.:2652.0   3rd Qu.:2595.5   3rd Qu.:2491   3rd Qu.:2960  
##  Max.   :5539.0   Max.   :5943.0   Max.   :6122   Max.   :3723  
##       V17              V18              V19            V20      
##  Min.   :   0.0   Min.   :   0.0   Min.   :   3   Min.   :  27  
##  1st Qu.: 681.5   1st Qu.: 315.0   1st Qu.:1472   1st Qu.:1374  
##  Median :1089.5   Median : 943.5   Median :2052   Median :2022  
##  Mean   :2000.0   Mean   :2000.0   Mean   :2000   Mean   :2000  
##  3rd Qu.:3466.8   3rd Qu.:3279.0   3rd Qu.:2607   3rd Qu.:2508  
##  Max.   :7028.0   Max.   :8691.0   Max.   :3940   Max.   :5429  
##       V21            V22            V23            V24        
##  Min.   : 145   Min.   : 260   Min.   :  88   Min.   :  24.0  
##  1st Qu.:1272   1st Qu.:1134   1st Qu.:1559   1st Qu.: 721.5  
##  Median :1564   Median :1521   Median :1769   Median :1562.5  
##  Mean   :2000   Mean   :2000   Mean   :2000   Mean   :2000.0  
##  3rd Qu.:1964   3rd Qu.:2122   3rd Qu.:2486   3rd Qu.:2705.5  
##  Max.   :8885   Max.   :7985   Max.   :4675   Max.   :5299.0  
##       V25            V26            V27            V28        
##  Min.   : 438   Min.   : 103   Min.   :   7   Min.   :   0.0  
##  1st Qu.:1232   1st Qu.: 821   1st Qu.:1463   1st Qu.:  60.5  
##  Median :1606   Median :1654   Median :2142   Median : 624.5  
##  Mean   :2000   Mean   :2000   Mean   :2000   Mean   :2000.0  
##  3rd Qu.:2090   3rd Qu.:2386   3rd Qu.:2643   3rd Qu.:3907.5  
##  Max.   :6168   Max.   :6571   Max.   :3562   Max.   :6490.0
diffstats <- data.frame(matrix(ncol = 28,nrow= 256))
diffstats <- (fathistMatstats- tathistMatstats)
colMeans(diffstats)
##  V1  V2  V3  V4  V5  V6  V7  V8  V9 V10 V11 V12 V13 V14 V15 V16 V17 V18 
##   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 
## V19 V20 V21 V22 V23 V24 V25 V26 V27 V28 
##   0   0   0   0   0   0   0   0   0   0
t.test(fathiststats$FATMeanHist, tathiststats$TATMeanHist, paired= T)
## 
##  Paired t-test
## 
## data:  fathiststats$FATMeanHist and tathiststats$TATMeanHist
## t = -0.00036377, df = 255, p-value = 0.9997
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -7.191353  7.188697
## sample estimates:
## mean of the differences 
##            -0.001328125

Masked and Color separated images in the Shine toolbox.

One concern I have about the stats above is that they are “watered down” by using the values for the entire image. Additionally, there may be more to dig into for these histogram scores.

Summary of SHINE results.

Overall though, on this first pass I believe we can say that differences in luminace and contrast values are not significantly different from 0.