Context-aware confidence sets for fine-grained product recognition

Baz, İpek and Yörük, Erdem and Çetin, Müjdat (2019) Context-aware confidence sets for fine-grained product recognition. IEEE Access, 7 . pp. 76376-76393. ISSN 2169-3536

PDF (Listed in DOAJ as an open access journal) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader

Official URL: http://dx.doi.org/10.1109/ACCESS.2019.2921994


We present a new approach for fine-grained classification of retail products, which learns and exploits statistical context information about likely product arrangements on shelves, incorporates visual hierarchies across brands, and returns recognition results as “confidence sets” that are guaranteed to contain the true class at a given confidence level. Our system consists of three important components: 1) a nested hierarchy of product classes are automatically constructed based on visual similarities, 2) a confidence set predictor is trained based on class posteriors by using coarse-to-fine binary classifiers to discriminate each nested cluster of the hierarchy from the remainder of classes and a Bayesian network (BN) model that encodes the joint distribution of classifier scores with the fine-level class variable, and 3) n hidden Markov model (HMM) is trained with nested hidden states from the class hierarchy to model spatial transition across the nodes of product class hierarchy and resolve errors in the context-free confidence set results. Novel aspects of the proposed method include 1) combining confidence sets and context information via a HMM, 2) applying this concept to fine grained recognition of products arranged in retail shelves, and 3) presenting experimental results on four large datasets, collected from actual retail stores. We compare our approach with existing confidence set approaches and state-of-the-art convolutional neural networks classifiers including SENet-154, DenseNet-161, B-CNN, and Inception-Resnet-v2. Our approach performs comparably or better than state-of-the-art deep classifiers and exhibits high accuracy for relatively small confidence set sizes.

Item Type:Article
Uncontrolled Keywords:Confidence sets; context-aware classification; hidden Markov model; fine-grained classification; hierarchical classification
Subjects:T Technology > TK Electrical engineering. Electronics Nuclear engineering
ID Code:38728
Deposited By:Müjdat Çetin
Deposited On:24 Aug 2019 23:42
Last Modified:24 Aug 2019 23:42

Repository Staff Only: item control page