CS 6316 Machine Learning
Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: daixieit
Homework 02: The Bias-variance Tradeoff, SVMs and Kernel Methods
CS 6316 Machine Learning
Questions
1. Bias-Variance Tradeoff (11 points)
Please refer to the attached iPython notebook file
2. KKT Conditions (4 points) The Lagrangian form of SVMs with slack variable s is formulated as
L(w, b, s , a, β) =
lwl2(2) + C ξi
i=1
m
- αi (yi (wT zi + b) - 1 + ξi ) i=1
m
- βi ξi i=1
Similar to SVMs in separable cases (lecture 04, page 21), to find the KKT conditions (as in page 28), we need to compute the derivative with respect to all parameters {w, b, s , a, β}. Overall, please show that the KKT conditions can be represented to the following equations
w
m αi yi
i=1
αi + βi αi = 0 βi = 0
m
= αi yi zi
i=1
= 0
= C
or yi (wT zi + b) = 1 - ξi
or ξi = 0
3. Kernel Methods (3 points) In our lecture on kernel methods (page 39), we show that a special case of the polynomial kernels
K(z, z干 ) = (〈z, z干）+ c)d
with d = 2 and z, z干 e /2 . On our lecture slides, we show how this special case can be decomposed as a dot product with a nonlinear mapping Φ(.)
K(z, z干 ) =〈Φ(z), Φ(z干 )）.
In this problem, consider d = 3 with z, z干 e /2 and show how the Φ(z) is defined in this case.
2022-03-27