Towards Context-aware Interaction Recognition
Bohan Zhuang
and
Lingqiao Liu
and
Chunhua Shen
and
Ian Reid
arXiv e-Print archive - 2017 via Local arXiv
Keywords:
cs.CV
First published: 2017/03/18 (7 years ago) Abstract: Recognizing how objects interact with each other is a crucial task in visual
recognition. If we define the context of the interaction to be the objects
involved, then most current methods can be categorized as either: (i) training
a single classifier on the combination of the interaction and its context; or
(ii) aiming to recognize the interaction independently of its explicit context.
Both methods suffer limitations: the former scales poorly with the number of
combinations and fails to generalize to unseen combinations, while the latter
often leads to poor interaction recognition performance due to the difficulty
of designing a context-independent interaction classifier. To mitigate those
drawbacks, this paper proposes an alternative, context-aware interaction
recognition framework. The key to our method is to explicitly construct an
interaction classifier which combines the context, and the interaction. The
context is encoded via word2vec into a semantic space, and is used to derive a
classification result for the interaction.
The proposed method still builds one classifier for one interaction (as per
type (ii) above), but the classifier built is adaptive to context via weights
which are context dependent. The benefit of using the semantic space is that it
naturally leads to zero-shot generalizations in which semantically similar
contexts (subjectobject pairs) can be recognized as suitable contexts for an
interaction, even if they were not observed in the training set.