Deep learning for facial action unit detection under large head poses

Zoltán Tősér, László A. Jeni, A. Lőrincz, Jeffrey F. Cohn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

8 Citations (Scopus)

Abstract

Facial expression communicates emotion, intention, and physical state, and regulates interpersonal behavior. Automated face analysis (AFA) for the detection, synthesis, and understanding of facial expression is a vital focus of basic research with applications in behavioral science, mental and physical health and treatment, marketing, and human-robot interaction among other domains. In previous work, facial action unit (AU) detection becomes seriously degraded when head orientation exceeds 15 ° to 20 °. To achieve reliable AU detection over a wider range of head pose, we used 3D information to augment video data and a deep learning approach to feature selection and AU detection. Source video were from the BP4D database (n=41) and the FERA test set of BP4D-extended (n=20). Both consist of naturally occurring facial expression in response to a variety of emotion inductions. In augmented video, pose ranged between −18 ° and 90 ° for yaw and between −54 ° and 54 ° for pitch angles. Obtained results for action unit detection exceeded state-of-the-art, with as much as a 10% increase in F1 measures.

Original languageEnglish
Title of host publicationComputer Vision – ECCV 2016 Workshops, Proceedings
PublisherSpringer Verlag
Pages359-371
Number of pages13
Volume9915 LNCS
ISBN (Print)9783319494081
DOIs
Publication statusPublished - 2016
Event14th European Conference on Computer Vision, ECCV 2016 - Amsterdam, Netherlands
Duration: Oct 8 2016Oct 16 2016

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9915 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other14th European Conference on Computer Vision, ECCV 2016
CountryNetherlands
CityAmsterdam
Period10/8/1610/16/16

Fingerprint

Human robot interaction
Feature extraction
Marketing
Health
Facial Expression
Unit
Human-robot Interaction
Test Set
Feature Selection
Proof by induction
Exceed
Deep learning
Learning
Face
Synthesis
Angle
Range of data

Keywords

  • Deep learning
  • Facial action unit detection
  • Pose dependence

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Tősér, Z., Jeni, L. A., Lőrincz, A., & Cohn, J. F. (2016). Deep learning for facial action unit detection under large head poses. In Computer Vision – ECCV 2016 Workshops, Proceedings (Vol. 9915 LNCS, pp. 359-371). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9915 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-49409-8_29

Deep learning for facial action unit detection under large head poses. / Tősér, Zoltán; Jeni, László A.; Lőrincz, A.; Cohn, Jeffrey F.

Computer Vision – ECCV 2016 Workshops, Proceedings. Vol. 9915 LNCS Springer Verlag, 2016. p. 359-371 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9915 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Tősér, Z, Jeni, LA, Lőrincz, A & Cohn, JF 2016, Deep learning for facial action unit detection under large head poses. in Computer Vision – ECCV 2016 Workshops, Proceedings. vol. 9915 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9915 LNCS, Springer Verlag, pp. 359-371, 14th European Conference on Computer Vision, ECCV 2016, Amsterdam, Netherlands, 10/8/16. https://doi.org/10.1007/978-3-319-49409-8_29
Tősér Z, Jeni LA, Lőrincz A, Cohn JF. Deep learning for facial action unit detection under large head poses. In Computer Vision – ECCV 2016 Workshops, Proceedings. Vol. 9915 LNCS. Springer Verlag. 2016. p. 359-371. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-49409-8_29
Tősér, Zoltán ; Jeni, László A. ; Lőrincz, A. ; Cohn, Jeffrey F. / Deep learning for facial action unit detection under large head poses. Computer Vision – ECCV 2016 Workshops, Proceedings. Vol. 9915 LNCS Springer Verlag, 2016. pp. 359-371 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{e4f990b01edb4e67aa01e6e6ecf07296,
title = "Deep learning for facial action unit detection under large head poses",
abstract = "Facial expression communicates emotion, intention, and physical state, and regulates interpersonal behavior. Automated face analysis (AFA) for the detection, synthesis, and understanding of facial expression is a vital focus of basic research with applications in behavioral science, mental and physical health and treatment, marketing, and human-robot interaction among other domains. In previous work, facial action unit (AU) detection becomes seriously degraded when head orientation exceeds 15 ° to 20 °. To achieve reliable AU detection over a wider range of head pose, we used 3D information to augment video data and a deep learning approach to feature selection and AU detection. Source video were from the BP4D database (n=41) and the FERA test set of BP4D-extended (n=20). Both consist of naturally occurring facial expression in response to a variety of emotion inductions. In augmented video, pose ranged between −18 ° and 90 ° for yaw and between −54 ° and 54 ° for pitch angles. Obtained results for action unit detection exceeded state-of-the-art, with as much as a 10{\%} increase in F1 measures.",
keywords = "Deep learning, Facial action unit detection, Pose dependence",
author = "Zolt{\'a}n Tős{\'e}r and Jeni, {L{\'a}szl{\'o} A.} and A. Lőrincz and Cohn, {Jeffrey F.}",
year = "2016",
doi = "10.1007/978-3-319-49409-8_29",
language = "English",
isbn = "9783319494081",
volume = "9915 LNCS",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "359--371",
booktitle = "Computer Vision – ECCV 2016 Workshops, Proceedings",

}

TY - GEN

T1 - Deep learning for facial action unit detection under large head poses

AU - Tősér, Zoltán

AU - Jeni, László A.

AU - Lőrincz, A.

AU - Cohn, Jeffrey F.

PY - 2016

Y1 - 2016

N2 - Facial expression communicates emotion, intention, and physical state, and regulates interpersonal behavior. Automated face analysis (AFA) for the detection, synthesis, and understanding of facial expression is a vital focus of basic research with applications in behavioral science, mental and physical health and treatment, marketing, and human-robot interaction among other domains. In previous work, facial action unit (AU) detection becomes seriously degraded when head orientation exceeds 15 ° to 20 °. To achieve reliable AU detection over a wider range of head pose, we used 3D information to augment video data and a deep learning approach to feature selection and AU detection. Source video were from the BP4D database (n=41) and the FERA test set of BP4D-extended (n=20). Both consist of naturally occurring facial expression in response to a variety of emotion inductions. In augmented video, pose ranged between −18 ° and 90 ° for yaw and between −54 ° and 54 ° for pitch angles. Obtained results for action unit detection exceeded state-of-the-art, with as much as a 10% increase in F1 measures.

AB - Facial expression communicates emotion, intention, and physical state, and regulates interpersonal behavior. Automated face analysis (AFA) for the detection, synthesis, and understanding of facial expression is a vital focus of basic research with applications in behavioral science, mental and physical health and treatment, marketing, and human-robot interaction among other domains. In previous work, facial action unit (AU) detection becomes seriously degraded when head orientation exceeds 15 ° to 20 °. To achieve reliable AU detection over a wider range of head pose, we used 3D information to augment video data and a deep learning approach to feature selection and AU detection. Source video were from the BP4D database (n=41) and the FERA test set of BP4D-extended (n=20). Both consist of naturally occurring facial expression in response to a variety of emotion inductions. In augmented video, pose ranged between −18 ° and 90 ° for yaw and between −54 ° and 54 ° for pitch angles. Obtained results for action unit detection exceeded state-of-the-art, with as much as a 10% increase in F1 measures.

KW - Deep learning

KW - Facial action unit detection

KW - Pose dependence

UR - http://www.scopus.com/inward/record.url?scp=85006061955&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85006061955&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-49409-8_29

DO - 10.1007/978-3-319-49409-8_29

M3 - Conference contribution

SN - 9783319494081

VL - 9915 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 359

EP - 371

BT - Computer Vision – ECCV 2016 Workshops, Proceedings

PB - Springer Verlag

ER -