Spatio-temporal event classification using time-series kernel based structured sparsity

László A. Jeni, A. Lőrincz, Zoltán Szabó, Jeffrey F. Cohn, Takeo Kanade

Research output: Chapter in Book/Report/Conference proceedingConference contribution

15 Citations (Scopus)

Abstract

In many behavioral domains, such as facial expression and gesture, sparse structure is prevalent. This sparsity would be well suited for event detection but for one problem. Features typically are confounded by alignment error in space and time. As a consequence, high-dimensional representations such as SIFT and Gabor features have been favored despite their much greater computational cost and potential loss of information. We propose a Kernel Structured Sparsity (KSS) method that can handle both the temporal alignment problem and the structured sparse reconstruction within a common framework, and it can rely on simple features. We characterize spatio-temporal events as time-series of motion patterns and by utilizing time-series kernels we apply standard structured-sparse coding techniques to tackle this important problem. We evaluated the KSS method using both gesture and facial expression datasets that include spontaneous behavior and differ in degree of difficulty and type of ground truth coding. KSS outperformed both sparse and non-sparse methods that utilize complex image features and their temporal extensions. In the case of early facial event classification KSS had 10% higher accuracy as measured by F 1 score over kernel SVM methods.

Original languageEnglish
Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
PublisherSpringer Verlag
Pages135-150
Number of pages16
Volume8692 LNCS
EditionPART 4
ISBN (Print)9783319105925
DOIs
Publication statusPublished - 2014
Event13th European Conference on Computer Vision, ECCV 2014 - Zurich, Switzerland
Duration: Sep 6 2014Sep 12 2014

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
NumberPART 4
Volume8692 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other13th European Conference on Computer Vision, ECCV 2014
CountrySwitzerland
CityZurich
Period9/6/149/12/14

Fingerprint

Sparsity
Time series
kernel
Facial Expression
Gesture
Alignment
Sparse Coding
Costs
Event Detection
Scale Invariant Feature Transform
Computational Cost
High Accuracy
High-dimensional
Coding
Motion

Keywords

  • facial expression classification
  • gesture recognition
  • structured sparsity
  • time-series kernels

ASJC Scopus subject areas

  • Computer Science(all)
  • Theoretical Computer Science

Cite this

Jeni, L. A., Lőrincz, A., Szabó, Z., Cohn, J. F., & Kanade, T. (2014). Spatio-temporal event classification using time-series kernel based structured sparsity. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (PART 4 ed., Vol. 8692 LNCS, pp. 135-150). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8692 LNCS, No. PART 4). Springer Verlag. https://doi.org/10.1007/978-3-319-10593-2_10

Spatio-temporal event classification using time-series kernel based structured sparsity. / Jeni, László A.; Lőrincz, A.; Szabó, Zoltán; Cohn, Jeffrey F.; Kanade, Takeo.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 8692 LNCS PART 4. ed. Springer Verlag, 2014. p. 135-150 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8692 LNCS, No. PART 4).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Jeni, LA, Lőrincz, A, Szabó, Z, Cohn, JF & Kanade, T 2014, Spatio-temporal event classification using time-series kernel based structured sparsity. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). PART 4 edn, vol. 8692 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), no. PART 4, vol. 8692 LNCS, Springer Verlag, pp. 135-150, 13th European Conference on Computer Vision, ECCV 2014, Zurich, Switzerland, 9/6/14. https://doi.org/10.1007/978-3-319-10593-2_10
Jeni LA, Lőrincz A, Szabó Z, Cohn JF, Kanade T. Spatio-temporal event classification using time-series kernel based structured sparsity. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). PART 4 ed. Vol. 8692 LNCS. Springer Verlag. 2014. p. 135-150. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 4). https://doi.org/10.1007/978-3-319-10593-2_10
Jeni, László A. ; Lőrincz, A. ; Szabó, Zoltán ; Cohn, Jeffrey F. ; Kanade, Takeo. / Spatio-temporal event classification using time-series kernel based structured sparsity. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 8692 LNCS PART 4. ed. Springer Verlag, 2014. pp. 135-150 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 4).
@inproceedings{72a7107eade54fa0bdf2163d8a89d4d6,
title = "Spatio-temporal event classification using time-series kernel based structured sparsity",
abstract = "In many behavioral domains, such as facial expression and gesture, sparse structure is prevalent. This sparsity would be well suited for event detection but for one problem. Features typically are confounded by alignment error in space and time. As a consequence, high-dimensional representations such as SIFT and Gabor features have been favored despite their much greater computational cost and potential loss of information. We propose a Kernel Structured Sparsity (KSS) method that can handle both the temporal alignment problem and the structured sparse reconstruction within a common framework, and it can rely on simple features. We characterize spatio-temporal events as time-series of motion patterns and by utilizing time-series kernels we apply standard structured-sparse coding techniques to tackle this important problem. We evaluated the KSS method using both gesture and facial expression datasets that include spontaneous behavior and differ in degree of difficulty and type of ground truth coding. KSS outperformed both sparse and non-sparse methods that utilize complex image features and their temporal extensions. In the case of early facial event classification KSS had 10{\%} higher accuracy as measured by F 1 score over kernel SVM methods.",
keywords = "facial expression classification, gesture recognition, structured sparsity, time-series kernels",
author = "Jeni, {L{\'a}szl{\'o} A.} and A. Lőrincz and Zolt{\'a}n Szab{\'o} and Cohn, {Jeffrey F.} and Takeo Kanade",
year = "2014",
doi = "10.1007/978-3-319-10593-2_10",
language = "English",
isbn = "9783319105925",
volume = "8692 LNCS",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
number = "PART 4",
pages = "135--150",
booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
edition = "PART 4",

}

TY - GEN

T1 - Spatio-temporal event classification using time-series kernel based structured sparsity

AU - Jeni, László A.

AU - Lőrincz, A.

AU - Szabó, Zoltán

AU - Cohn, Jeffrey F.

AU - Kanade, Takeo

PY - 2014

Y1 - 2014

N2 - In many behavioral domains, such as facial expression and gesture, sparse structure is prevalent. This sparsity would be well suited for event detection but for one problem. Features typically are confounded by alignment error in space and time. As a consequence, high-dimensional representations such as SIFT and Gabor features have been favored despite their much greater computational cost and potential loss of information. We propose a Kernel Structured Sparsity (KSS) method that can handle both the temporal alignment problem and the structured sparse reconstruction within a common framework, and it can rely on simple features. We characterize spatio-temporal events as time-series of motion patterns and by utilizing time-series kernels we apply standard structured-sparse coding techniques to tackle this important problem. We evaluated the KSS method using both gesture and facial expression datasets that include spontaneous behavior and differ in degree of difficulty and type of ground truth coding. KSS outperformed both sparse and non-sparse methods that utilize complex image features and their temporal extensions. In the case of early facial event classification KSS had 10% higher accuracy as measured by F 1 score over kernel SVM methods.

AB - In many behavioral domains, such as facial expression and gesture, sparse structure is prevalent. This sparsity would be well suited for event detection but for one problem. Features typically are confounded by alignment error in space and time. As a consequence, high-dimensional representations such as SIFT and Gabor features have been favored despite their much greater computational cost and potential loss of information. We propose a Kernel Structured Sparsity (KSS) method that can handle both the temporal alignment problem and the structured sparse reconstruction within a common framework, and it can rely on simple features. We characterize spatio-temporal events as time-series of motion patterns and by utilizing time-series kernels we apply standard structured-sparse coding techniques to tackle this important problem. We evaluated the KSS method using both gesture and facial expression datasets that include spontaneous behavior and differ in degree of difficulty and type of ground truth coding. KSS outperformed both sparse and non-sparse methods that utilize complex image features and their temporal extensions. In the case of early facial event classification KSS had 10% higher accuracy as measured by F 1 score over kernel SVM methods.

KW - facial expression classification

KW - gesture recognition

KW - structured sparsity

KW - time-series kernels

UR - http://www.scopus.com/inward/record.url?scp=84906491165&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84906491165&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-10593-2_10

DO - 10.1007/978-3-319-10593-2_10

M3 - Conference contribution

SN - 9783319105925

VL - 8692 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 135

EP - 150

BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

PB - Springer Verlag

ER -