A Derivative-Free Optimization Method for Solving Classification Problem

Abstract

Problem statement: The aim of data classification is to establish rules for the classification of some observations assuming that we have a database, which includes of at least two classes. There is a training set for each class. Those problems occur in a wide range of human activity. One of the most promising ways to data classification is based on methods of mathematical optimization. Approach: The problem of data classification was studied as a problem of global, nonsmooth and nonconvex optimization; this approach consists of describing clusters for the given training sets. The data vectors are assigned to the closest cluster and correspondingly to the set, which contains this cluster and an algorithm based on a derivative-free method is applied to the solution of this problem. Results: Proposed method had been tested on real-world datasets. Results of numerical experiments had been presented which demonstrate the effectiveness of the proposed algorithm. Conclusion: In this study we had studied a derivative-free optimization approach to the classification. For optimization generalized pattern search method has been applied. The results of numerical experiments allowed us to say the proposed algorithms are effective for solving classification problems at least for databases considered in this study.