title   
  

Linear classifier combination and selection using group sparse regularization and hinge loss

Şen, Mehmet Umut and Erdoğan, Hakan (2013) Linear classifier combination and selection using group sparse regularization and hinge loss. Pattern Recognition Letters, 34 (3). pp. 265-274. ISSN 0167-8655

This is the latest version of this item.

Full text not available from this repository.

Official URL: http://dx.doi.org/10.1016/j.patrec.2012.10.008

Abstract

The main principle of stacked generalization is using a second-level generalizer to combine the outputs of base classifiers in an ensemble. In this paper, after presenting a short survey of the literature on stacked generalization, we propose to use regularized empirical risk minimization (RERM) as a framework for learning the weights of the combiner which generalizes earlier proposals and enables improved learning methods. Our main contribution is using group sparsity for regularization to facilitate classifier selection. In addition, we propose and analyze using the hinge loss instead of the conventional least squares loss. We performed experiments on three different ensemble setups with differing diversities on 13 real-world datasets of various applications. Results show the power of group sparse regularization over the conventional norm regularization. We are able to reduce the number of selected classifiers of the diverse ensemble without sacrificing accuracy. With the non-diverse ensembles, we even gain accuracy on average by using group sparse regularization. In addition, we show that the hinge loss outperforms the least squares loss which was used in previous studies of stacked generalization.

Item Type:Article
Uncontrolled Keywords:Classifier combination; Group sparsity; Classifier selection; Regularized empirical risk minimization; Hinge loss
Subjects:UNSPECIFIED
ID Code:22983
Deposited By:Hakan Erdoğan
Deposited On:21 Jan 2014 12:13
Last Modified:17 Oct 2014 15:12

Available Versions of this Item

Repository Staff Only: item control page