Activity 2.1 - dissimilarity measures

SUBMISSION INSTRUCTIONS:

Submit both:

Your source .qmd file;
A link to the published version of your rendered html.

Question 1

A)

Consider the data set below:

Person	Age (years)	Income (k$)	Gender	Education
P1	25	35	Female	Bachelor
P2	40	50	Male	Bachelor
P3	30	65	Female	Master

Which distance measure would be best for measuring the dissimilarity between these individuals? Find the 3 pairwise distances using your selected metric “from scratch”. Which pair is most different? Most similar?

Best Metric: Gower Metric

Variable ranges: $\text{Range}(\text{Age}) = 40 - 25 = 15$; $\text{Range}(\text{Income}) = 65 - 35 = 30$. $P_1$ vs $P_2$: $\frac{1}{4}\left(\frac{\vert{}25-40\vert{}}{15} + \frac{\vert{}35-50\vert{}}{30} + 1 + 0\right) = \frac{1 + 0.50 + 1 + 0}{4} = \mathbf{0.625}$ $P_1$ vs $P_3$: $\frac{1}{4}\left(\frac{\vert{}25-30\vert{}}{15} + \frac{\vert{}35-65\vert{}}{30} + 0 + 1\right) = \frac{0.333 + 1.00 + 0 + 1}{4} \approx \mathbf{0.583}$ $P_2$ vs $P_3$: $\frac{1}{4}\left(\frac{\vert{}40-30\vert{}}{15} + \frac{\vert{}50-65\vert{}}{30} + 1 + 1\right) = \frac{0.667 + 0.50 + 1 + 1}{4} \approx \mathbf{0.792}$

Most different: $P_2$ and $P_3$ ($0.792$). Most similar: $P_1$ and $P_3$ ($0.583$)

B)

Consider the data set below:

Person	Preferred Cuisine	Hobby	Device Used Most
P1	Italian	Reading	Laptop
P2	Mexican	Reading	Phone
P3	Italian	Hiking	Laptop

Best Metric: Jaccard Distance

\[d_J = \frac{b + c}{a + b + c} = 1 - \frac{\text{matches}}{6 - \text{matches}}\] $P_1$ vs $P_2$: 1 match (Reading) $\rightarrow d_J = \frac{4}{1 + 4} = \mathbf{0.80}$ $P_1$ vs $P_3$: 2 matches (Italian, Laptop) $\rightarrow d_J = \frac{2}{2 + 2} = \mathbf{0.50}$ $P_2$ vs $P_3$: 0 matches $\rightarrow d_J = \frac{6}{0 + 6} = \mathbf{1.00}$

Most different: $P_2$ and $P_3$ ($1.00$). Most similar: $P_1$ and $P_3$ ($0.50$)

C)

Consider the data set below representing counts of three different species at three locations:

Site	Species A	Species B	Species C
Site 1	4	2	0
Site 2	1	3	1
Site 3	0	5	2

Best Metric: Bray-Curtis Dissimilarity

\[BC = \frac{\sum \vert{}a_k - b_k\vert{}}{\sum (a_k + b_k)}\] Site 1 vs Site 2: $\frac{\vert{}4-1\vert{} + \vert{}2-3\vert{} + \vert{}0-1\vert{}}{6 + 5} = \frac{3 + 1 + 1}{11} = \frac{5}{11} \approx \mathbf{0.455}$ Site 1 vs Site 3: $\frac{\vert{}4-0\vert{} + \vert{}2-5\vert{} + \vert{}0-2\vert{}}{6 + 7} = \frac{4 + 3 + 2}{13} = \frac{9}{13} \approx \mathbf{0.692}$ Site 2 vs Site 3: $\frac{\vert{}1-0\vert{} + \vert{}3-5\vert{} + \vert{}1-2\vert{}}{5 + 7} = \frac{1 + 2 + 1}{12} = \frac{4}{12} \approx \mathbf{0.333}$

Most different: Site 1 and Site 3 ($0.692$). Most similar: Site 2 and Site 3 ($0.333$)

Question 2

You’re a city manager in charge of building a new fire station. The fire station serves a city of only 4 households. There are 3 locations in the city suitable for the fire station.

The grid below shows the locations of the houses (grey squares) and suitable fire station locations (red circles):

A)

Which distance metric would be most appropriate for determining the optimal place to build the fire station? Explain your reasoning.

L1. Emergency fire vehicles travel along perpendicular roads and city blocks rather than flying directly over obstacles ($L_2$).

B)

Which is the optimal place of the available locations to build the fire station that minimizes the cumulative distance between the station and the houses?

Houses are at $(1,1), (2,4), (4,2), (5,5)$. Candidate stations: $S_1(2,2)$, $S_2(3,3)$, and $S_3(4,3)$. Cumulative $L_1$ distance for all three candidates:$S_1(2,2) = 2 + 2 + 2 + 6 = \[S_2(3,3) = 4 + 2 + 2 + 4 = \mathbf{12} \]S_3(4,3) = 5 + 3 + 1 + 3 = $

C)

Now consider a much larger city:

library(tidyverse)

large_city <- read.csv('Data/large_city.csv')


ggplot(large_city, aes(x = X, y = Y)) +
  geom_point(pch = 15) +           
  theme_bw(base_size = 14) +
  theme(panel.grid.minor = element_blank(),
        panel.grid.major = element_line(color='lightgrey'),
        axis.text = element_blank(),
        axis.ticks = element_blank()) +
  scale_x_continuous(expand = c(0.01,0.01), limits = c(0,100), breaks = 0:100) +
  scale_y_continuous(expand = c(0.01,0.01), limits = c(0,100), breaks = 0:100)

The code below produces the coordinates of all locations in the city:

(possibilities <- expand.grid(X_FS = 0:100, Y_FS = 0:100) 
      %>% mutate(FireStationID = 1:n())
) %>% head

  X_FS Y_FS FireStationID
1    0    0             1
2    1    0             2
3    2    0             3
4    3    0             4
5    4    0             5
6    5    0             6

The next chunk of code crosses each house coordinate with all city coordinates:

(distances <- large_city
              %>% crossing(possibilities)
            # %>% your code here!
) %>% head

# A tibble: 6 × 6
  HouseID     X     Y  X_FS  Y_FS FireStationID
    <int> <int> <int> <int> <int>         <int>
1       1    12    23     0     0             1
2       1    12    23     0     1           102
3       1    12    23     0     2           203
4       1    12    23     0     3           304
5       1    12    23     0     4           405
6       1    12    23     0     5           506

opt_L1 <- large_city %>%
  crossing(possibilities) %>%
  anti_join(large_city, by = c("X_FS" = "X", "Y_FS" = "Y")) %>%
  mutate(dist_L1 = abs(X - X_FS) + abs(Y - Y_FS)) %>%
  group_by(FireStationID, X_FS, Y_FS) %>%
  summarize(total_dist = sum(dist_L1), .groups = "drop") %>%
  slice_min(total_dist, with_ties = TRUE)

ggplot(large_city, aes(x = X, y = Y)) +
  geom_point(pch = 15) +
  geom_point(data = opt_L1, aes(x = X_FS, y = Y_FS), color = "red", size = 2) +
  theme_bw(base_size = 14) +
  scale_x_continuous(expand = c(0.01,0.01), limits = c(0,100), breaks = seq(0,100,20)) +
  scale_y_continuous(expand = c(0.01,0.01), limits = c(0,100), breaks = seq(0,100,20)) +
  labs(title = "Optimal Fire Station Locations (L1)")

Suppose any location where a house is not already built is a candidate for a fire station location. Complete the dplyr chain above to find the fire station location that minimizes the L1 distance from all houses (there may be multiple optimal locations!). Add the location(s) of the fire station to the graph of the city using red dot(s). You should of course make sure that you don’t build a fire station on top of an existing house!

D)

Modify your code to find the optimal location of the station if L2 distance is used instead. Add the new location to the grid.

opt_L2 <- large_city %>%
  crossing(possibilities) %>%
  anti_join(large_city, by = c("X_FS" = "X", "Y_FS" = "Y")) %>%
  mutate(dist_L2 = sqrt((X - X_FS)^2 + (Y - Y_FS)^2)) %>%
  group_by(FireStationID, X_FS, Y_FS) %>%
  summarize(total_dist = sum(dist_L2), .groups = "drop") %>%
  slice_min(total_dist, with_ties = TRUE)

ggplot(large_city, aes(x = X, y = Y)) +
  geom_point(pch = 15) +
  geom_point(data = opt_L2, aes(x = X_FS, y = Y_FS), color = "red", size = 3) +
  theme_bw(base_size = 14) +
  labs(title = "Optimal Fire Station Location (L2)")

Question 3

Consider the R code below which produced one of the plots from the lectures slides:

library(plotly)

# Read and wrangle
city77 <- (read.csv("Data/City77.csv")
        %>% mutate(City = gsub('\\.',' ',City))
        %>% mutate(highlight = ifelse(City %in% c("Minneapolis MN", "Kansas City MO", "Milwaukee WI"), 'yes','no'))
) 

#Plot
plot_ly(city77,
        x = ~popdens,
        y = ~k12enr,
        z = ~medinc,
        type = "scatter3d",
        mode = "markers+text",
        text = ~ifelse(highlight == "yes", City, ""),
        textposition = "top center",
        color = ~highlight,
        colors = c("gray", "red"),
        marker = list(size = 5)) %>%
  layout(title = "3D Plot: Population Density vs K-12 vs Median Income",
         scene = list(
           xaxis = list(title = "Population Density"),
           yaxis = list(title = "K-12 enrollment %"),
           zaxis = list(title = "Median Income")
         ))

Use this code as a template to analyze bankruptcy data (source: Johnson and Wichern 6e page 657). This data set contains four metrics on both bankrupt and financially sound firms. The metrics were recorded 2 years prior to bankruptcy for the bankrupt firms, and about the same time for the financially sound firms.

Variables:

bankrupt = 1 for yes, 0 for financially sound
x1 = cash flow / total debt
x2 = net income/total assets
x3 = current assets/current liabilities
x4 = current assets/net sales

bankruptcy <- read.csv('Data/bankruptcy_data.csv')

A)

Create an interactive 3D scatterplot of this data using x1, x2, and x3. Color-code the points by bankruptcy status. Imagine your boss is a bank manager. Use your plot to explain what are some warning signs to look out for in terms of the characteristics of bankrupt banks.

plot_ly(bankruptcy,
        x = ~x1,
        y = ~x2,
        z = ~x3,
        color = ~as.factor(bankrupt),
        colors = c("firebrick", "navy"),
        type = "scatter3d",
        mode = "markers",
        marker = list(size = 5)) %>%
  layout(title = "Bankruptcy 3D Scatter Plot",
         scene = list(
           xaxis = list(title = "x1 (Cash Flow / Debt)"),
           yaxis = list(title = "x2 (Net Income / Assets)"),
           zaxis = list(title = "x3 (Current Assets / Liab)")
         ))

Failing firms cluster near negative or near-zero x1 (cash flow cannot service total debt) and negative x2 (negative returns on assets). Sound banks maintain positive operational cash flow and profit margins.

B)

Find the mean vectors of x1 through x4 for bankrupt and sound banks (so 2 mean vectors). Then consider a new bank with the following metrics: x1 = 0.31, x2 = 0.04, x3 = 4.47, x4 = 0.30

Group 0: $\mu_0 = (-0.0690, -0.0814, 1.3667, 0.4376)$Group 1: $\mu_1 = (0.2352, 0.0556, 2.5936, 0.4268)$

New Bank: $v = (0.31, 0.04, 4.47, 0.30)$

Distance to Group 0:

\[\sqrt{(0.31 - (-0.069))^2 + (0.04 - (-0.081))^2 + (4.47 - 1.367)^2 + (0.30 - 0.438)^2} = \mathbf{3.132}\] Distance to Group 1:

\[\sqrt{(0.31 - 0.235)^2 + (0.04 - 0.056)^2 + (4.47 - 2.594)^2 + (0.30 - 0.427)^2} = \mathbf{1.882}\]

Is this bank doing well or are they in trouble? Justify your answer using distance between this bank’s metrics and the mean vector for each bank type.

The bank is doing well. Its distance to the solvent cluster (1.882) is substantially smaller than to the distressed cluster (3.132).

Question 4

These data come from a collaboration with WSU biology faculty and students and published in Water. The Whitewater River has three forks: north, middle, and south.

Image source: Water

Electrofishing at over 60 locations on the three forks was used to sample fish populations. The data are below.

whitewater <- read.csv('Data/Whitewater.csv')

A)

Explain why Bray-Curtis is appropriate for measuring similarity/dissimilarity in populations across sites. Use Bray-Curtis and data management techniques to find the two most similar, and the two most dissimilar sites.

Fish survey counts vary heavily by sampling effort, and species absent at both sites (joint absences) do not represent ecological similarity.

Most Similar Sites: MF5 (Crow Spring d/s Hwy 9) and MF4 (Crow Spring confluence) with a minimum Bray-Curtis dissimilarity of 0.0804.Most Dissimilar Sites: Multiple pairs have completely disjoint species profiles ($BC = \mathbf{1.00}$), including NF32 (Pries source 1) vs NF33 (Pries source 2).R

B)

The code below transforms the data set to measure only presence/absence of each species:

present <- \(species) ifelse(species>0, 1, 0)

whitewater_present <- (whitewater
                       %>% mutate(across(.cols=American.brook.lamprey:slimy.sculpin, 
                          .fns=present))
                      )

Consider these two sites:

whitewater_present %>% 
  slice(2, 4)

  fork            site siteID      lat      long American.brook.lamprey
1   NF  Pries source 2   NF33 44.04202 -92.26845                      0
2   NF         Pries 2   NF31 44.04505 -92.26420                      0
  Rainbow.trout Brown.trout Brook.trout Central.stoneroller Common.shiner
1             0           0           0                   0             0
2             0           0           0                   0             0
  Sand.shiner Southern.redbelly.dace Bluntnose.minnow Fathead.minnow
1           0                      0                0              0
2           0                      1                0              0
  Blacknose.dace Longnose.dace Creek.chub White.sucker Brook.stickleback
1              1             0          1            0                 0
2              1             1          1            0                 1
  Green.sunfish Bluegill Fantail.darter Johnny.darter mottled.sculpin
1             0        0              1             1               0
2             0        0              1             1               0
  slimy.sculpin
1             0
2             0

Compute the Jaccard dissimilarity metric between these two sites from scratch.

$a$ (both present): $4$ species (Blacknose dace, Creek chub, Fantail darter, Johnny darter)$b$ (Site 2 only): $0$ species$c$ (Site 4 only): $3$ species (Southern redbelly dace, Longnose dace, Brook stickleback)$d$ (both absent): $14$ species

\[d_J = \frac{b + c}{a + b + c} = \frac{0 + 3}{4 + 0 + 3} = \frac{3}{7} \approx \mathbf{0.429}\]

C)

Use the Gower measure on the count data, and also find the Jaccard dissimilarities for all sites using the present/absent data. Turn each into a data frame and join them with the Bray-Curtis measures (so you have 3 measures for each unique pair of site comparisons). Then create pairwise scatterplots of the 3 dissimilarity measures versus each other. How do they compare?

Comparison: Bray-Curtis and Jaccard correlate positively, but Bray-Curtis spreads out more widely across $[0, 1]$ due to count scaling. Gower produces smaller absolute values because ranges in species abundance compress the individual variable fractions.

D)

Find the 2 pairs of sites with the largest discrepancies between the Bray-Curtis and Jaccard measures. Can you identify why they are in disagreement?

NF21 (Borgen) vs NF20 (BReiter): $\vert{}\text{BC} - \text{Jaccard}\vert{} = \vert{}0.801 - 0.000\vert{} = \mathbf{0.801}$NF36 (Shea Crossing) vs SF52 (Irke Source): $\vert{}\text{BC} - \text{Jaccard}\vert{} = \vert{}0.723 - 0.000\vert{} = \mathbf{0.723}$

Why They Disagree: Both pairs share 100% identical species presence ($a > 0, b = 0, c = 0$), yielding a Jaccard distance of 0.00. However, their raw abundances differ drastically (e.g., at Borgen vs BReiter, slimy sculpin count is $1$ vs $125$, and Longnose dace is $10$ vs $150$), which drives the Bray-Curtis dissimilarity above $0.70$–$0.80$.