Video-based emotion recognition in the wild using deep transfer learning and score fusion
Yükleniyor...
Dosyalar
Tarih
2017
Dergi Başlığı
Dergi ISSN
Cilt Başlığı
Yayıncı
Elsevier
Erişim Hakkı
info:eu-repo/semantics/closedAccess
Özet
Multimodal recognition of affective states is a difficult problem, unless the recording conditions are carefully controlled. For recognition in the wild, large variances in face pose and illumination, cluttered backgrounds, occlusions, audio and video noise, as well as issues with subtle cues of expression are some of the issues to target. In this paper, we describe a multimodal approach for video-based emotion recognition in the wild. We propose using summarizing functionals of complementary visual descriptors for video modeling. These features include deep convolutional neural network (CNN) based features obtained via transfer learning, for which we illustrate the importance of flexible registration and fine-tuning. Our approach combines audio and visual features with least squares regression based classifiers and weighted score level fusion. We report state-of-the-art results on the EmotiW Challenge for in the wild facial expression recognition. Our approach scales to other problems, and ranked top in the ChaLearn-LAP First Impressions Challenge 2016 from video clips collected in the wild. (C) 2017 Elsevier B.V. All rights reserved.
Açıklama
Anahtar Kelimeler
EmotiW, Emotion recognition in the wild, Multimodal fusion, Convolutional neural networks, Kernel extreme learning machine, Partial least squares, Machine
Kaynak
Image and Vision Computing
WoS Q Değeri
Q1
Scopus Q Değeri
Q2
Cilt
65
