Support Vector Machine algorithm


Support vector machine algorithm is a popular classification algorithm. SVM is also used for regression problem but widely used in classification. In SVM algorithm, we plot each data item as a point in n-dimensional space (where n is a number of features you have) with the value of each feature being the value of a particular coordinate. Then, we perform classification by finding the hyperplane that differentiates the two classes very well.






                        Fig :  the basic concept of SVM
To separate two class data there are many ways to select the hyperplane. Here the black line is the hyper plan which classifies the data point in two class as shown in fig. the green data point is known as support vector(support data point). Our objective to find max margin hyper plan.

What is Support Vector Machine?

The objective of the support vector machine algorithm is to find a hyperplane in N-dimensional space(N — the number of features) that distinctly classifies the data points.


Type of  Support Vector Machine :
1.    Linear SVM classifier
2.    Non-Linear SVM classifier

1.    Linear SVM classifier:  In linear SVM  its  predicts a straight hyperplane dividing 2 classes. The primary focus while drawing the hyperplane is on maximizing the distance from hyperplane to the nearest data point of either class. The drawn hyperplane called a maximum-margin hyperplane.


  


2.    Non-linear SVM classifier: Suppose a situation where we cannot plot a straight hyperplane to classify the data set. Then we apply the kernel trick to maximum-margin hyperplanes. In Non-Linear SVM Classification, data points plotted in a higher dimensional space.






Fig: Non-linear problem of SVM classification
As shown in fig. we cannot able to draw hyper plan but SVM the algorithm can easily classify the data point using kernel tricks. Here, we will add a new feature z=x^2+y^2. Now, let’s plot the data points on axis x and z.

Kernel trick is functions which take low dimensional input space and transform it to a higher dimensional space i.e. it converts not separable problem to separable problem, these functions are called kernels. It is most useful in non-linear separation problem. Simply put, it does some extremely complex data transformations, then find out the process to separate the data based on the labels or outputs you’ve defined.

Pros and Cons associated with SVM
  • Pros:
    • It works really well with a clear margin of separation
    • It is effective in cases where the number of dimensions is greater than the number of samples.
    • It uses a a subset of training points in the decision function (called support vectors), so it is also memory efficient.
    • It is effective in high dimensional spaces. 
  • Cons:
    • It doesn’t perform well, when we have a large data set because of the required training time is higher.
    •  The biggest limitation of Support Vector Machine is the choice of the kernel
    • It also doesn’t perform very well, when the data set has more noise i.e. target classes are overlapping
    • SVM doesn’t directly provide probability estimates, these are calculated using an expensive five-fold cross-validation. It is related SVC method of Python scikit-learn library.