Q1.C Answer

import numpy as np

XTX = np.array([[15, 27, 46, 66],
                [27, 51, 82, 118],
                [46, 82, 174, 240],
                [66, 118, 240, 334]])

eigenvalues, eigenvectors = np.linalg.eig(XTX)
max_eigenvalue = max(eigenvalues)

L2_norm = np.sqrt(max_eigenvalue).round(2)
print("L2 norm of X:", L2_norm)
L2 norm of X: 23.7

Q2.B Answer

# Define the Householder matrix U
U = np.array([[0.853, -0.197, -0.295, -0.295, -0.226, -0.098],
              [-0.197, 0.738, -0.393, -0.393, -0.301, -0.131],
              [-0.295, -0.393, 0.410, -0.590, -0.452, -0.197],
              [-0.295, -0.393, -0.590, 0.410, -0.452, -0.197],
              [-0.226, -0.301, -0.452, -0.452, 0.654, -0.151],
              [-0.098, -0.131, -0.197, -0.197, -0.151, 0.935]])

# Calculate C = UU
C = np.dot(U, U)

print("Matrix C =\n", C)
Matrix C =
 [[ 1.001148e+00 -6.930000e-04  3.440000e-04  3.440000e-04  1.930000e-04
   9.390000e-04]
 [-6.930000e-04  1.000113e+00  6.800000e-04  6.800000e-04  5.830000e-04
   4.360000e-04]
 [ 3.440000e-04  6.800000e-04  1.000787e+00  7.870000e-04  4.620000e-04
  -9.000000e-05]
 [ 3.440000e-04  6.800000e-04  7.870000e-04  1.000787e+00  4.620000e-04
  -9.000000e-05]
 [ 1.930000e-04  5.830000e-04  4.620000e-04  4.620000e-04  1.000802e+00
  -2.720000e-04]
 [ 9.390000e-04  4.360000e-04 -9.000000e-05 -9.000000e-05 -2.720000e-04
   1.001409e+00]]

(c) Find Ux. What’s its relation with x?

# Vector x
x = np.array([1.5, 2, 3, 3, 2.3, 1])

# Matrix-vector multiplication: U * x
result = np.dot(U, x)

print(result)
[-1.5023 -2.0008 -3.0051 -3.0051 -2.2998 -1.0033]

What’s its relation with x?

When a Householder transformation is applied to a vector, it performs a reflection about a hyperplane determined by the Householder matrix, resulting in an orthogonal transformation of the vector.

(d) Find ∥U ∥∞ :

Note: To find the infinity norm of the given Householder matrix, find the maximum absolute row sum.

infinity_norm = np.max(np.sum(np.abs(U), axis=1))

print("Infinity norm of U:", infinity_norm, "represents the maximum absolute row sum of the matrix")
Infinity norm of U: 2.337 represents the maximum absolute row sum of the matrix

(e) Print the elements of the third column below the diagonal in U:

U_slice = U[3:, 2]

print("Elements of the third column below the diagonal in U:", U_slice)
Elements of the third column below the diagonal in U: [-0.59  -0.452 -0.197]

(f) Find P = AAT , where A is the first three columns of U:

A = U[:, :3]

print("Matrix A:")
print(A)
print()

# P = A * A^T
P = np.dot(A, A.T)

print("Matrix P =\n", P)
Matrix A:
[[ 0.853 -0.197 -0.295]
 [-0.197  0.738 -0.393]
 [-0.295 -0.393  0.41 ]
 [-0.295 -0.393 -0.59 ]
 [-0.226 -0.301 -0.452]
 [-0.098 -0.131 -0.197]]

Matrix P =
 [[ 8.53443e-01 -1.97492e-01 -2.95164e-01 -1.64000e-04 -1.41000e-04
   3.28000e-04]
 [-1.97492e-01  7.37902e-01 -3.93049e-01 -4.90000e-05  2.00000e-05
   4.90000e-05]
 [-2.95164e-01 -3.93049e-01  4.09574e-01 -4.26000e-04 -3.57000e-04
  -3.77000e-04]
 [-1.64000e-04 -4.90000e-05 -4.26000e-04  5.89574e-01  4.51643e-01
   1.96623e-01]
 [-1.41000e-04  2.00000e-05 -3.57000e-04  4.51643e-01  3.45981e-01
   1.50623e-01]
 [ 3.28000e-04  4.90000e-05 -3.77000e-04  1.96623e-01  1.50623e-01
   6.55740e-02]]

(g) Find ∥P − P P ∥1:

Note: P is idempotent, that is P = PP

# P * P = P^2
P_squared = np.dot(P, P)

# Absolute difference between P and P^2
P_diff = np.abs(P - P_squared)

# L1-norm of the absolute difference
norm_1 = np.sum(P_diff).round(3)

print("||P - P^2||_1 =", norm_1)
||P - P^2||_1 = 0.008

h) Find Q = BBT , where B is the last three columns of U:

B = U[:, -3:]

print("Matrix B:")
print(B)
print()

# Q = B * B^T
Q = np.dot(B, B.T)

print("Matrix Q:")
print(Q)
Matrix B:
[[-0.295 -0.226 -0.098]
 [-0.393 -0.301 -0.131]
 [-0.59  -0.452 -0.197]
 [ 0.41  -0.452 -0.197]
 [-0.452  0.654 -0.151]
 [-0.197 -0.151  0.935]]

Matrix Q:
[[ 1.47705e-01  1.96799e-01  2.95508e-01  5.08000e-04  3.34000e-04
   6.11000e-04]
 [ 1.96799e-01  2.62211e-01  3.93729e-01  7.29000e-04  5.63000e-04
   3.87000e-04]
 [ 2.95508e-01  3.93729e-01  5.91213e-01  1.21300e-03  8.19000e-04
   2.87000e-04]
 [ 5.08000e-04  7.29000e-04  1.21300e-03  4.11213e-01 -4.51181e-01
  -1.96713e-01]
 [ 3.34000e-04  5.63000e-04  8.19000e-04 -4.51181e-01  6.54821e-01
  -1.50895e-01]
 [ 6.11000e-04  3.87000e-04  2.87000e-04 -1.96713e-01 -1.50895e-01
   9.35835e-01]]

(i) Find ∥Q − QQ∥∞:

Note: Q is idempotent, that is Q = QQ.

# Q * Q = Q^2
Q_squared = np.dot(Q, Q)

# Absolute difference between Q and Q^2
diff = np.abs(Q - Q_squared)

# Infinity norm of the absolute difference
norm_infinity = np.max(np.sum(diff, axis=1)).round(3)

print("||Q - Q^2||∞ =", norm_infinity)
||Q - Q^2||∞ = 0.002

(j) Find P + Q. What is the resulting matrix?

P_plus_Q = P + Q

print("Matrix (P + Q):")
print(P_plus_Q)
Matrix (P + Q):
[[ 1.001148e+00 -6.930000e-04  3.440000e-04  3.440000e-04  1.930000e-04
   9.390000e-04]
 [-6.930000e-04  1.000113e+00  6.800000e-04  6.800000e-04  5.830000e-04
   4.360000e-04]
 [ 3.440000e-04  6.800000e-04  1.000787e+00  7.870000e-04  4.620000e-04
  -9.000000e-05]
 [ 3.440000e-04  6.800000e-04  7.870000e-04  1.000787e+00  4.620000e-04
  -9.000000e-05]
 [ 1.930000e-04  5.830000e-04  4.620000e-04  4.620000e-04  1.000802e+00
  -2.720000e-04]
 [ 9.390000e-04  4.360000e-04 -9.000000e-05 -9.000000e-05 -2.720000e-04
   1.001409e+00]]

Q3.B: Using R and your formula from (a) to write a function named mygradient(x) whose output should be the gradient vector.

# i) Python Answer:

def mygradient(x):
    """
    Goal: To compute the gradient vector of the function f(x) = 1 / (x[0] * x[1]) at point x.

    Argument:
    x (ndarray): The point at which to evaluate the gradient vector.

    Return:
    The gradient vector of f at the point x (ndarray).
    """
    # Gradient components per partial derivates from answer a)
    df_dx1 = -1/ ((x[0]**2) * x[1])
    df_dx2 = -1/ (x[0] * (x[1]**2))

    # Gradient vector
    gradient_v = np.array([df_dx1, df_dx2])

    return gradient_v

Q3.C: Apply x = (3, 2)T to the function you defined. Print out your function outputs.

# ii) Python Answer:

# Define the point x
x = np.array([3, 2])

# Gradient vector at point x
result = mygradient(x).round(3)

# Print the gradient vector
print("Gradient vector at x = (3, 2)^T:")
print(result)
Gradient vector at x = (3, 2)^T:
[-0.056 -0.083]