class: center, middle, inverse, title-slide .title[ # An Algorithm for Forensic Toolmark Comparisons ] .subtitle[ ## JSM 2022 ] .author[ ### Maria Cuellar (Penn), Heike Hofmann (Iowa State), Shane Jensen (Penn) ] .institute[ ### University of Pennsylvania ] .date[ ### 08/08/2022 ] --- class: left, left # Motivation - In the Genrich 1991 case, examiner John O’Neil testified saying he matched Genrich’s tools to four bombs “to the exclusion of any other tool” in the world. -- .center[] -- - From these images, it seems that although there is information on the marks, the examiner's "match" conclusion could have been incorrect. But how can we tell? -- - **Research questions**: Are toolmarks made by the same tool more similar to each other than marks made by different tools? How can we quantify this? Since toolmarks can be made at different settings, how does this affect comparisons? --- # Current toolmark comparisons are subjective - **Degrees of freedom**: In toolmarks, the perpetrator could have used the tool in many different ways. -- .center[] -- - **Test marks**: Examiners generate several test marks at different angles, direction, force, etc. to get a sense for the tool's characteristics. Then they select the "best" mark for the final comparison. -- - **Subjective comparison**: This comparison is usually done with a comparison light microscope, in which the examiner decides whether the marks are similar enough to be considered an identification. -- - **Algorithms**: We follow the calls to make comparisons more objective (see PCAST 2016, Kafadar, 2019). --- # Data generation ## First step: Collect materials .center[] -- - **Materials**: 60 consecutively manufactured flat-head screwdrivers, lead plates, mechanical rig to make controlled marks, handheld 3D scanner. - **Pre-processing marks**: using packages generated by Hofmann et al. for bullets. -- - **Timeline**: Collected ~600 3D toolmarks during the last two months, starting the modeling now. --- # Data generation ## Second step: Factorial design database .center[] --- # Are marks made by the same tool similar to each other? .center[] --- # Comparing sides A and B of same tool .center[] --- # Comparing directions .center[] --- # Comparing angles .center[] --- # Comparing sizes .center[] --- # Modeling Data (already aligned) is binned: .center[] --- # Modeling ### Variability - Marks as vectors: `\(y_{ijb}\)` = area under curve in bin `\(b\)` of mark `\(j\)` made with tool `\(i\)`. In Bin 1, we average over tools, `\(\underline{y}_{tj} = (y_{tj1}, \dots, y_{tj B})\)`. - Marks within tool: `\(\underline{y}_{tj} \sim N_B (\underline{\mu}_t, \Sigma_t)\)`, where `\(\Sigma_t\)` represents the variance of `\(\underline{y}_{tj}\)` across marks within a tool. It could be `\(\Sigma_t\)` or diagonal if the bins are uncorrelated. - Marks across tools: `\(\underline{\mu}_t \sim N_B (\underline{\mu}_0, \Omega)\)`, where `\(\Omega\)` represents the covariance of `\(\underline{\mu}_t\)` across tools. `\(\Omega\)` should ideally be diagonal, where `\(\omega_{jj}\)` = variance in `\(\underline{\mu}_t\)` across tools. - Incorporating angle, direction: Model `\(\underline{\mu}_t\)` as a function of angle and direction. `\(\underline{\mu}_t = f(\)`angle, direction, `\(\mu_t^o\)`), where `\(\mu_t^0\)` is a canonical mean defined at certain angle and direction. --- # Canonical mark for tool 1 .center[] --- # Canonical mark for tools 1-20 .center[] --- # Next steps 1. Decide on a common banded covariance structure for the `\(\Sigma_t\)` matrices and estimate those common correlations. 2. Evaluate an in-sample classification by calculating the probability density of each individual mark `\(y_{tj}\)` using all the different tool signatures ( `\(\mu_t\)`’s and `\(\Sigma_t’s\)`) and see if the correct tool tends to give the highest probability density (compared to the other tools) for each mark. Does a classification based on the highest probability density for each mark do better or worse than a simpler K-means clustering? 3. Model how `\(\mu_t\)` (and `\(\Sigma_t\)`) changes as a function of function of angle and direction. --- # Clustering .center[] --- class: center, center, inverse # Questions? --- # Correlation matrix This is the matrix for correlation across tools. Correlation drops off away from diagonal. This means contiguous bins are more similar to each other than distant bins. Can model this as a banded diagonal matrix for simplicity. .center[] --- # Area per bin, averaged over all tools Zero means that the variations above and below zero are random. .center[]