Identifying visual attributes for object recognition from text and taxonomy

Tırkaz, Çağlar and Eisenstein, Jacob and Sezgin, Metin T. and Yanıkoğlu, Berrin (2015) Identifying visual attributes for object recognition from text and taxonomy. Computer Vision and Image Understanding, 137 . pp. 12-23. ISSN 1077-3142 (Print) 1090-235X (Online)

This is the latest version of this item.

[img]PDF - Registered users only - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader

Official URL: http://dx.doi.org/10.1016/j.cviu.2015.02.006


Attributes of objects such as "square", "metallic", and "red" allow a way for humans to explain or discriminate object categories. These attributes also provide a useful intermediate representation for object recognition, including support for zero-shot learning from textual descriptions of object appearance. However, manual selection of relevant attributes among thousands of potential candidates is labor intensive. Hence, there is increasing interest in mining attributes for object recognition. In this paper, we introduce two novel techniques for nominating attributes and a method for assessing the suitability of candidate attributes for object recognition. The first technique for attribute nomination estimates attribute qualities based on their ability to discriminate objects at multiple levels of the taxonomy. The second technique leverages the linguistic concept of distributional similarity to further refine the estimated qualities. Attribute nomination is followed by our attribute assessment procedure, which assesses the quality of the candidate attributes based on their performance in object recognition. Our evaluations demonstrate that both taxonomy and distributional similarity serve as useful sources of information for attribute nomination, and our methods can effectively exploit them. We use the mined attributes in supervised and zero-shot learning settings to show the utility of the selected attributes in object recognition. Our experimental results show that in the supervised case we can improve on a state of the art classifier while in the zero-shot scenario we make accurate predictions outperforming previous automated techniques.

Item Type:Article
Uncontrolled Keywords:Object recognition; Zero-shot learning; Attribute mining; Attribute-based classification
ID Code:27612
Deposited By:Berrin Yanıkoğlu
Deposited On:08 Dec 2015 15:56
Last Modified:08 Dec 2015 15:56

Available Versions of this Item

Repository Staff Only: item control page