[toc]
常見的損失函數(shù)
y_i表示實際值，f_i表示預測值
0-1損失函數(shù)
L(y_i, f_i) = \left\{\begin{matrix} 1, ~y_i = f_i\\
0, ~y_i \neq f_i\end{matrix}\right.
等價形式：
L(y_i, f_i) = \frac{1}{2}(1 - sign(y_i\cdot f_i)), ~y_i\in\{\pm1\}
Perceptron感知損失函數(shù)(感知機)
L(y_i, f_i) = \left\{\begin{matrix} 1, ~|y_i - f_i| > t\\
0, ~|y_i - f_i| \leq t\end{matrix}\right.
等價形式：
L(y_i, f_i) = max\{0,~-(y_i\cdot f_i)\}, ~y_i\in\{\pm1\}
證明
因為當y_i = {-1, +1}時，|y_i - f_i| = {0, +2}，第一個式子等價于
L(y_i, f_i) = \left\{\begin{matrix} 1, ~|y_i - f_i| = 2~/~y_i\cdot f_i = -1\\
0, ~|y_i - f_i| = 0~/~y_i\cdot f_i = 1\end{matrix}\right.
又等價于
L(y_i, f_i) = max\{0,~-(y_i\cdot f_i)\}, ~y_i\in\{\pm1\}
Hinge損失函數(shù)(SVM)
L(y_i, f_i) = max\{0,~1 - y_i\cdot f_i\},~ y \in \{\pm1\}
Loss損失函數(shù)(Logistic回歸)
L(y_i, f_i) = -\left(y_i\log f_i + (1-y_i)\log{(1-f_i)}\right),~y_i\in \{0,1\}
其中
f(x) = 1/\exp(-w^T\cdot x)
等價于
L(y_i, f_i) = log(1 + \exp(y_i\cdot f_i)),~ y_i \in \{\pm1\}
證明
因為當y_i = {0, +1}時，第一個式子等價于
L(y_i,f_i) = \left\{\begin{matrix} log(1+\exp(-w^T\cdot x),~y_i = 1\\
log(1+\exp(w^T\cdot x),~y_i=0\end{matrix}\right.
等價于，當y_i = {-1, +1}時
L(y_i,f_i) = \left\{\begin{matrix} log(1+\exp(-w^T\cdot x),~y_i = 1\\
log(1+\exp(w^T\cdot x),~y_i=-1\end{matrix}\right.
等價于
L(y_i, f_i) = log(1 + \exp(y_i\cdot f_i)),~ y_i \in \{\pm1\}
指數(shù)損失函數(shù)(Adaboost)
L(y_i,f_i)=\exp(-y_i\cdot f_i), y_i\in \{\pm1\}
幾個損失函數(shù)的圖像
image
回歸損失函數(shù)
Square損失函數(shù)
L(y_i,f_i)=(y_i - f_i)^2
Absolute損失函數(shù)
L(y_i,f_i)=|y_i-f_i|
