In the previous blogs we learned about classification, its types, and the algorithms related to those types, so in this blog post we will discuss one query which was pending in the classification blog
How can we use binary classification algorithms for doing multi-class classification?
Here I am assuming that you are aware of the difference between binary classification and multi-class classification so we will directly jump on to the ways to use some of binary classification algorithms such as ( Logistic Regression and Support Vector Machine ) for doing the multi-class classification.
In short we can not directly use these algorithms for doing the multi-class classification but with 2 approaches that we will discuss in this blog post you can get your job done . So the 2 approaches using which we can use binary classificaton algorithms for multi-class classification are :
- One vs Rest Approach
- One vs One Approach
One Vs Rest Approach
The one vs rest approach is the most common approach that you will see in the machine learning models or in the machine learning algorithms provided by the sci-kit learn library in python.
Now as the name suggests we will basically separate one class from all other remaining class, will make a binary classification model for that one class which we selected for our consideration, and this same process will be repeated for the other classes as well. To better understand what you just read take a look at the diagram just below
Let us assume that we have to make a machine learning model that will predict the name of the fruit based on the image provided or fed to the machine learning model as input, but here there are more than 2 classes and we want to use let say logistic regression algorithm for doing the multi-class classification, so we will use one vs rest approach in which we will make 3 binary classification model for each class.
Now when we will make 3 binary classification model , one for each then it is obvious that for every binary classification mode we will get some sort prediction with the probability value
After we have got the probability value from each one of the binary classification models we will evaluate these values and will find the greatest probability value, and will eliminate the smaller values.
So after the analysis since we got the highest value from 1st binary classification model, thus we can say that for the given image the fruit is apple, and that's it you learned the theory for how we can use the binary classification algorithm for doing the multi-class classification using the one vs rest approach. Now let's move on to the next approach using which we can use the binary classification model for doing the multi-class classification and that approach is one vs one , but wait a minute !!
If we already have one vs rest approach then why we are still going for the one vs one approach ?
The answer to this query is that some times using this approach we get stuck in the situation where we face imbalance in number of sample in some particular class , for example if we want to make multi-class classification model that will predict that whether a particular question is hard , easy or not and the data which we are having is containing 100 easy , 100 medium and 100 hard questions , then if we will use the one vs rest approach then in order to train our model we will only be having 100 ( easy questions ) : 200 ( medium + hard questions )
Thus in order to dodge such type of situation we use one vs one approach
One vs One Approach
In case of one vs one approach we again make binary classification models but in this case the total number of binary classification models that we will make can be determined by the formula given below
As you have read above that in one vs one approach we make n ( n -1 ) /2 binary classification model , but this is not the only difference , in addition to it , in case of one vs one approach we do pairing of class in a single binary classification model so if we have 4 classes it means we will make 6 binary classification models and in each model there will be a pair of any 2 classes out of 4 classes , to better understand take a look at the illustration below
After making 6 different binary classification models , every classification model will make predictions for given sample input , and then total number of votes will be calculated from 6 different models , for example : For a given input out 6 models 4 models are saying the given input data belongs to banana class , so overall our machine learning model will give us an output that for the given input image contains banana