support vector machines in matlab

by

SVMs were originally designed for binary classification. They have then been extended to handle multi-class problems. The idea is to decompose the problem into many binary-class problems and then combine them to obtain the prediction.

One approach called one-against-all, builds as many binary classifiers as there are classes, each trained to separate one class from the rest. To predict a new instance, we choose the classifier with the largest decision function value.

Another approach called one-against-one (which I believe is used in LibSVM), builds k(k-1)/2 binary classifiers, trained to separate each pair of classes against each other, and uses a majority voting scheme (max-win strategy) to determine the output prediction.

There are also other approaches such as using Error Correcting Output Code (ECOC) to build many somewhat-redundant binary-classifiers, and use this redundancy to obtain more robust classifications (uses the same idea as Hamming codes).

Example (one-against-one):

%# load dataset
load fisheriris
[g gn] = grp2idx(species);                      %# nominal class to numeric

%# split training/testing sets
[trainIdx testIdx] = crossvalind('HoldOut', species, 1/3);

pairwise = nchoosek(1:length(gn),2);            %# 1-vs-1 pairwise models
svmModel = cell(size(pairwise,1),1);            %# store binary-classifers
predTest = zeros(sum(testIdx),numel(svmModel)); %# store binary predictions

%# classify using one-against-one approach, SVM with 3rd degree poly kernel
for k=1:numel(svmModel)
    %# get only training instances belonging to this pair
    idx = trainIdx & any( bsxfun(@eq, g, pairwise(k,:)) , 2 );

    %# train
    svmModel{k} = svmtrain(meas(idx,:), g(idx), ...
        'BoxConstraint',2e-1, 'Kernel_Function','polynomial', 'Polyorder',3);

    %# test
    predTest(:,k) = svmclassify(svmModel{k}, meas(testIdx,:));
end
pred = mode(predTest,2);   %# voting: clasify as the class receiving most votes

%# performance
cmat = confusionmat(g(testIdx),pred);
acc = 100*sum(diag(cmat))./sum(cmat(:));
fprintf('SVM (1-against-1):\naccuracy = %.2f%%\n', acc);
fprintf('Confusion Matrix:\n'), disp(cmat)

Here is a sample output:

SVM (1-against-1):
accuracy = 93.75%
Confusion Matrix:
    16     0     0
     0    14     2
     0     1    15

More Related Contents:

  1. Multi-class classification in libsvm [closed]
  2. Retraining after Cross Validation with libsvm
  3. 10 fold cross-validation in one-against-all SVM (using LibSVM)
  4. Finding K-nearest neighbors and its implementation
  5. Example of 10-fold SVM classification in MATLAB
  6. Cost function in logistic regression gives NaN as a result
  7. Understanding concept of Gaussian Mixture Models
  8. using precomputed kernels with libsvm
  9. how to get a single feature vector from SIFT descriptor [closed]
  10. Speed-efficient classification in Matlab
  11. Neural network in MATLAB
  12. How does this code for standardizing data work?
  13. PCA in matlab selecting top n components
  14. Machine learning – Linear regression using batch gradient descent
  15. Error in Confusion Matrix : the data and reference factors must have the same number of levels
  16. Plotting 4 curves in a single plot, with 3 y-axes
  17. Construct adjacency matrix in MATLAB
  18. Replace specific columns in a matrix with a constant column vector
  19. draw ellipse and ellipsoid in MATLAB
  20. How do I compare all elements of two arrays?
  21. Consider preallocating for speed [duplicate]
  22. How to set the plot in matlab to a specific size?
  23. Loading multiple images in MATLAB
  24. Neural network XOR gate not learning
  25. How to abort a running program in MATLAB?
  26. Scikit learn – fit_transform on the test set
  27. MATLAB debugging: smarter way to stop the code with an specific condition?
  28. Generating all combinations with repetition using MATLAB
  29. Rounding to n significant digits
  30. How do I double the size of a vector in MATLAB with interpolation?

Leave a Comment