Identifying visual attributes for object recognition from text and taxonomy

Tırkaz, Çağlar and Eisenstein, Jacob and Sezgin, Metin T. and Yanıkoğlu, Berrin (2015) Identifying visual attributes for object recognition from text and taxonomy. Computer Vision and Image Understanding, 137 . pp. 12-23. ISSN 1077-3142 (Print) 1090-235X (Online)

This is the latest version of this item.

[thumbnail of caglar-cviu-2015.pdf] PDF
caglar-cviu-2015.pdf
Restricted to Registered users only

Download (1MB) | Request a copy

Abstract

Attributes of objects such as "square", "metallic", and "red" allow a way for humans to explain or discriminate object categories. These attributes also provide a useful intermediate representation for object recognition, including support for zero-shot learning from textual descriptions of object appearance. However, manual selection of relevant attributes among thousands of potential candidates is labor intensive. Hence, there is increasing interest in mining attributes for object recognition. In this paper, we introduce two novel techniques for nominating attributes and a method for assessing the suitability of candidate attributes for object recognition. The first technique for attribute nomination estimates attribute qualities based on their ability to discriminate objects at multiple levels of the taxonomy. The second technique leverages the linguistic concept of distributional similarity to further refine the estimated qualities. Attribute nomination is followed by our attribute assessment procedure, which assesses the quality of the candidate attributes based on their performance in object recognition. Our evaluations demonstrate that both taxonomy and distributional similarity serve as useful sources of information for attribute nomination, and our methods can effectively exploit them. We use the mined attributes in supervised and zero-shot learning settings to show the utility of the selected attributes in object recognition. Our experimental results show that in the supervised case we can improve on a state of the art classifier while in the zero-shot scenario we make accurate predictions outperforming previous automated techniques.
Item Type: Article
Uncontrolled Keywords: Object recognition; Zero-shot learning; Attribute mining; Attribute-based classification
Divisions: Faculty of Engineering and Natural Sciences > Academic programs > Computer Science & Eng.
Faculty of Engineering and Natural Sciences
Depositing User: Berrin Yanıkoğlu
Date Deposited: 08 Dec 2015 15:56
Last Modified: 23 Aug 2019 11:48
URI: https://research.sabanciuniv.edu/id/eprint/27612

Available Versions of this Item

Actions (login required)

View Item
View Item