2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

Visit the conference page.

Abstract

In this paper, we introduce a novel approach for semantic description of object features based on the prototypicality effects of the Prototype Theory. Our prototype-based description model encodes and stores the semantic meaning of an object, while describing its features using the semantic prototype computed by CNN-classifications models. Our method uses semantic prototypes to create discriminative descriptor signatures that describe an object highlighting its most distinctive features within the category. Our experiments show that: i) our descriptor preserves the semantic information used by the CNN-models in classification tasks; ii) our distance metric can be used as the object’s typicality score; iii) our descriptor signatures are semantically interpretable and enables the simulation of the prototypical organization of objects within a category.

Motivation and Concepts

Schematic of our prototype-based description model

Schematic of our prototype-based description model. a) features extraction; b) object features recognition; c) categorization; d) object features; e) central semantic meaning of a category. The human visual system is able to observe an object and to build a global semantic description highlighting the object features that make it distinctive within the category. We propose how to simulate this behavior through the processing flow from a) to e).

Prototype-based description model

Overview of our prototype-based description model. Set of steps to transform the visual information received as input into a Global Semantic Descriptor signature. a) input image; b) extracted features using a CNN-classification model; c) classification and category prototype selection; d) Global semantic description of object using the category prototype; e) graphic representation of the Global Semantic Descriptor signature resulting from the dimensionality reduction function (f(x)); and f) Global Semantic Descriptor signature.


Source code (NEW!)

Coming Soon.

Citation

@InProceedings{vidal2019wacv,
title = {Prototypicality effects in global semantic description of objects},
booktitle = {2019 IEEE Winter Conference on Applications of Computer Vision (WACV)},
author = {Omar Vidal Pino and Erickson R. Nascimento and Mario F. M. Campos},
Year = {2019},
Address = {Hawaii, USA},
month = {January},
intype = {to appear in},
pages = {},
volume = {},
number = {},
doi = {},
ISBN = {}
}

Authors


Back to the project page.