Kernel Support Vector Machine(Kernel SVM)

What is Kernel SVM?

It is a concept that was developed to compensate for the disadvantage that SVM cannot handle non-linear datasets like image below.

So, how does Kernel SVM classify non-linear datasets? The answer is by mapping them to a higher dimension like image below.

By mapping to a higher dimension, we can obtain a linearly separable dataset divided by a hyperplane in that space. After this process, by projecting back to the lower dimension, you can get a result with a round hyperplane, as shown in the image below.

Can Kernel SVM then classify a more complex dataset, as shown in the image below?”

The answer is YES. By adding more than two kernel formulas such as the Gaussian RBF kernel, it can classify the dataset.

Here is a simple Gaussian RBF kernel formula:

$K($ $\overrightarrow{x}$ $, \overrightarrow{l}^{1} ) = $ $e^{-\vert \vec{x} - \vec{l} \vert^{2}\over 2\sigma^{2} }$

So, the result of the kernel SVM will appear like this:

• Green when:
  $K($ $\overrightarrow{x}$ $, \overrightarrow{l}^{1} )$ $+ K($ $\overrightarrow{x}$ $, \overrightarrow{l}^{2} ) > 0$
• Red when:
  $K($ $\overrightarrow{x}$ $, \overrightarrow{l}^{1} )$ $+ K($ $\overrightarrow{x}$ $, \overrightarrow{l}^{2} ) = 0$

Types of Kernel Functions

• Gaussian RBF Kernel
  

$K($ $\overrightarrow{x}$ $, \overrightarrow{l}^{1} ) = $ $e^{-\vert \vec{x} - \vec{l} \vert^{2}\over 2\sigma^{2} }$

• Sigmoid Kernel
  

$K(X, Y) = $ $\tanh(\gamma \times X^{T}Y + r)$

• Polynomial Kernel
  

$K(X,Y) = $ $(\gamma \times X^{T}Y + r)^{d},r > 0$

Example

Code

---

from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform(X_test)


classifier = SVC(kernel='rbf', random_state=0)
classifier.fit(X_train, y_train)

y_pred = classifier.predict(X_test)

Result

---

Implementation

• Kernel SVM