R Notebook

This is an R Markdown Notebook. When you execute code within the notebook, the results appear beneath the code.

Try executing this chunk by clicking the Run button within the chunk or by placing your cursor inside it and pressing Cmd+Shift+Enter.

---

1 a) Hyperplane: 1 + 3X1 - X2 = 0

This equation represents a straight line in a two-dimensional plane. It can be rewritten as:

X2 = 3X1 + 1

This line has a slope of 3 and a y-intercept of 1. It divides the plane into three parts:

On the line: The equation is exactly 1 + 3X1 - X2 = 0.

Above the line: Points where X2 is less than 3X1 + 1. In this region, the expression 1 + 3X1 - X2 is greater than 0.

Below the line: Points where X2 is greater than 3X1 + 1. In this region, the expression 1 + 3X1 - X2 is less than 0.

So, the hyperplane separates the space into two regions: one where the expression is positive and one where it is negative.

(b) Hyperplane: -2 + X1 + 2X2 = 0

This is also a straight line in the two-dimensional space. It can be rewritten as:

X2 = (-X1 + 2) / 2

This line has a slope of -0.5 and a y-intercept of 1. It divides the plane into three parts:

On the line: The equation is exactly -2 + X1 + 2X2 = 0.

Above the line: Points where X2 is greater than (-X1 + 2) / 2. In this region, the expression -2 + X1 + 2X2 is greater than 0.

Below the line: Points where X2 is less than (-X1 + 2) / 2. In this region, the expression -2 + X1 + 2X2 is less than 0.

Again, the hyperplane separates the space into two regions based on whether the expression is positive or negative.

library(ggplot2)

# Create grid of values
x1_vals <- seq(-10, 10, length.out = 400)
x2_vals <- seq(-10, 10, length.out = 400)
grid <- expand.grid(X1 = x1_vals, X2 = x2_vals)

# Calculate the hyperplane values
grid$z1 <- 1 + 3 * grid$X1 - grid$X2
grid$z2 <- -2 + grid$X1 + 2 * grid$X2

# Plot
ggplot(grid, aes(x = X1, y = X2)) +
  # Region for hyperplane 1
  geom_raster(aes(fill = factor(sign(z1))), alpha = 0.3) +
  scale_fill_manual(values = c("-1" = "#add8e6", "1" = "#0000ff"), name = "1 + 3X1 - X2") +
  
  # Region for hyperplane 2 (fixed)
  geom_raster(data = grid, aes(x = X1, y = X2, alpha = factor(sign(z2))), fill = "green") +
  scale_alpha_manual(values = c("1" = 0.2, "-1" = 0.5), guide = FALSE) +
  
  # Add contour lines for both hyperplanes
  geom_contour(aes(z = z1), breaks = 0, color = "blue", size = 1) +
  geom_contour(aes(z = z2), breaks = 0, color = "darkgreen", size = 1) +
  
  labs(title = "Hyperplanes and Regions",
       x = "X1", y = "X2") +
  theme_minimal() +
  coord_fixed()

Warning: The `guide` argument in `scale_*()` cannot be `FALSE`. This was deprecated in ggplot2 3.3.4.
ℹ Please use "none" instead.
This warning is displayed once every 8 hours.
Call `lifecycle::last_lifecycle_warnings()` to see where this warning was generated.

2) Exploring a Non-Linear Decision Boundary in Two Dimensions

library(ggplot2)

# Create a grid of values
x1_vals <- seq(-5, 5, length.out = 400)
x2_vals <- seq(-5, 10, length.out = 400)
grid <- expand.grid(X1 = x1_vals, X2 = x2_vals)

# Compute the decision value
grid$Z <- (1 + grid$X1)^2 + (2 - grid$X2)^2

# Define points to classify
points <- data.frame(
  X1 = c(0, -1, 2, 3),
  X2 = c(0, 1, 2, 8),
  label = c("(0,0)", "(-1,1)", "(2,2)", "(3,8)")
)
points$class <- ifelse((1 + points$X1)^2 + (2 - points$X2)^2 > 4, "Blue", "Red")

# Plot
ggplot(grid, aes(x = X1, y = X2)) +
  # Fill regions
  geom_raster(aes(fill = Z <= 4), alpha = 0.4) +
  scale_fill_manual(values = c("TRUE" = "#add8e6", "FALSE" = "#ff9999"),
                    labels = c("<= 4", "> 4"),
                    name = "Decision Value") +
  
  # Contour line where Z = 4
  geom_contour(aes(z = Z), breaks = 4, color = "black", size = 1) +
  
  # Points and labels
  geom_point(data = points, aes(x = X1, y = X2, color = class), size = 3) +
  geom_text(data = points, aes(label = label), hjust = -0.3, vjust = -0.3, size = 3.5) +
  scale_color_manual(values = c("Red" = "red", "Blue" = "blue")) +
  
  # Labels and theme
  labs(title = "Non-linear Decision Boundary",
       subtitle = "(1 + X1)^2 + (2 - X2)^2 = 4",
       x = "X1", y = "X2") +
  coord_fixed() +
  theme_minimal()

(a) Curve: (1 + X1)^2 + (2 - X2)^2 = 4

This equation represents a circle in two-dimensional space. The center of the circle is at the point (-1, 2), and the radius is 2 units. The curve itself is the set of all points that lie exactly 2 units away from the center.

(b) Region interpretation

The circle divides the plane into two regions: Inside the circle: This includes all points where (1 + X1)^2 + (2 - X2)^2 is less than or equal to 4. Outside the circle: This includes all points where (1 + X1)^2 + (2 - X2)^2 is greater than 4.

(c) Classification of points

Suppose we classify a point as blue if (1 + X1)^2 + (2 - X2)^2 is greater than 4, and red otherwise. Then:

The point (0, 0) is classified as blue, because the value is 5. The point (-1, 1) is classified as red, because the value is 1. The point (2, 2) is classified as blue, because the value is 9. The point (3, 8) is classified as blue, because the value is 52.

(d) Linearity in transformed variables

Although the decision boundary (the circle) is not a straight line when viewed in terms of the original variables X1 and X2, it becomes a linear boundary when we use transformed variables:

1. X1
2. X1 squared
3. X2
4. X2 squared

if we include both the original variables and their squares as new features, then the equation becomes a linear combination of those features. So, the boundary is non-linear in the original space but linear in the transformed feature space.