Max-margin stacking with group sparse regularization for classifier combination

Official URL: http://192.168.1.20/record=b1379245 (Table of Contents)

Multiple classifier systems are shown to be effective in terms of accuracy for multiclass classification problems with the expense of increased complexity. Classifier combination studies deal with the methods of combining the outputs of base classifiers of an ensemble. Stacked generalization, or stacking, is shown to be a strong combination scheme among combination algorithms; and in this thesis, we improve stacking's performance further in terms of both accuracy and complexity. We investigate four main issues for this purpose. First, we show that margin maximizing combiners outperform the conventional least-squares estimation of the weights. Second we incorporate the idea of group sparsity into regularization to facilitate classifier selection. Third, we develop non-linear versions of class-conscious linear combination types by transforming datasets into binary classification datasets; then applying the kernel trick. And finally, we derive a new optimization algorithm based on the majorization-minimization framework for a particular linear combination type, which we show is the most preferable one.