A Parametric Optimization Method for Machine Learning

The classification problem of constructing a plane to separate the members of two sets can be formulated as a parametric bilinear program. This approach was originally created to minimize the number of points misclassified. However, a novel interpretation of the algorithm is that the subproblems represent alternative error functions of the misclassified points. Each subproblem identifies a specified number of outliers and minimizes the magnitude of the errors on the remaining points. A tuning set is used to select the best result among the subproblems. A parametric Frank-Wolfe method was used to solve the bilinear subproblems. Computational results on a number of datasets indicate that the results compare very favorably with linear programming and heuristic search approaches. The algorithm can be used as part of a decision tree algorithm to create nonlinear classifiers.