Tracking the saliency features in images based on human observation statistics

Szilard Szalai, T. Szirányi, Z. Vidnyánszky

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

We address the statistical inference of saliency features in the images based on human eye-tracking measurements. Training videos were recorded by a head-mounted wearable eye-tracker device, where the position of the eye fixation relative to the recorded image was annotated. From the same video records, artificial saliency points (SIFT) were measured by computer vision algorithms which were clustered to describe the images with a manageable amount of descriptors. The measured human eye-tracking (fixation pattern) and the estimated saliency points are fused in a statistical model, where the eye-tracking supports us with transition probabilities among the possible image feature points. This HVS-based statistical model results in the estimation of possible tracking paths and region of interest areas of the human vision. The proposed method may help in image saliency analysis, better compression of region of interest areas and in the development of more efficient human-computer-interaction devices.

Original languageEnglish
Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Pages219-233
Number of pages15
Volume7252 LNCS
DOIs
Publication statusPublished - 2012
EventInternational Workshop on Multimedia Understanding Through Semantics, Computation, and Learning, MUSCLE 2011 - Pisa, Italy
Duration: Dec 13 2011Dec 15 2011

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume7252 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

OtherInternational Workshop on Multimedia Understanding Through Semantics, Computation, and Learning, MUSCLE 2011
CountryItaly
CityPisa
Period12/13/1112/15/11

Fingerprint

Saliency
Eye Tracking
Statistics
Human computer interaction
Image analysis
Computer vision
Fixation
Region of Interest
Statistical Model
Path Tracking
Human Vision
Scale Invariant Feature Transform
Feature Point
Statistical Inference
Transition Probability
Computer Vision
Descriptors
Compression
Observation
Human

Keywords

  • descriptors
  • eye-tracking
  • fixation pattern
  • human observation
  • saliency

ASJC Scopus subject areas

  • Computer Science(all)
  • Theoretical Computer Science

Cite this

Szalai, S., Szirányi, T., & Vidnyánszky, Z. (2012). Tracking the saliency features in images based on human observation statistics. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 7252 LNCS, pp. 219-233). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7252 LNCS). https://doi.org/10.1007/978-3-642-32436-9_19

Tracking the saliency features in images based on human observation statistics. / Szalai, Szilard; Szirányi, T.; Vidnyánszky, Z.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 7252 LNCS 2012. p. 219-233 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7252 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Szalai, S, Szirányi, T & Vidnyánszky, Z 2012, Tracking the saliency features in images based on human observation statistics. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). vol. 7252 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 7252 LNCS, pp. 219-233, International Workshop on Multimedia Understanding Through Semantics, Computation, and Learning, MUSCLE 2011, Pisa, Italy, 12/13/11. https://doi.org/10.1007/978-3-642-32436-9_19
Szalai S, Szirányi T, Vidnyánszky Z. Tracking the saliency features in images based on human observation statistics. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 7252 LNCS. 2012. p. 219-233. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-642-32436-9_19
Szalai, Szilard ; Szirányi, T. ; Vidnyánszky, Z. / Tracking the saliency features in images based on human observation statistics. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 7252 LNCS 2012. pp. 219-233 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{8f38a861c14b4b25baad22aad43dcfd7,
title = "Tracking the saliency features in images based on human observation statistics",
abstract = "We address the statistical inference of saliency features in the images based on human eye-tracking measurements. Training videos were recorded by a head-mounted wearable eye-tracker device, where the position of the eye fixation relative to the recorded image was annotated. From the same video records, artificial saliency points (SIFT) were measured by computer vision algorithms which were clustered to describe the images with a manageable amount of descriptors. The measured human eye-tracking (fixation pattern) and the estimated saliency points are fused in a statistical model, where the eye-tracking supports us with transition probabilities among the possible image feature points. This HVS-based statistical model results in the estimation of possible tracking paths and region of interest areas of the human vision. The proposed method may help in image saliency analysis, better compression of region of interest areas and in the development of more efficient human-computer-interaction devices.",
keywords = "descriptors, eye-tracking, fixation pattern, human observation, saliency",
author = "Szilard Szalai and T. Szir{\'a}nyi and Z. Vidny{\'a}nszky",
year = "2012",
doi = "10.1007/978-3-642-32436-9_19",
language = "English",
isbn = "9783642324352",
volume = "7252 LNCS",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
pages = "219--233",
booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

}

TY - GEN

T1 - Tracking the saliency features in images based on human observation statistics

AU - Szalai, Szilard

AU - Szirányi, T.

AU - Vidnyánszky, Z.

PY - 2012

Y1 - 2012

N2 - We address the statistical inference of saliency features in the images based on human eye-tracking measurements. Training videos were recorded by a head-mounted wearable eye-tracker device, where the position of the eye fixation relative to the recorded image was annotated. From the same video records, artificial saliency points (SIFT) were measured by computer vision algorithms which were clustered to describe the images with a manageable amount of descriptors. The measured human eye-tracking (fixation pattern) and the estimated saliency points are fused in a statistical model, where the eye-tracking supports us with transition probabilities among the possible image feature points. This HVS-based statistical model results in the estimation of possible tracking paths and region of interest areas of the human vision. The proposed method may help in image saliency analysis, better compression of region of interest areas and in the development of more efficient human-computer-interaction devices.

AB - We address the statistical inference of saliency features in the images based on human eye-tracking measurements. Training videos were recorded by a head-mounted wearable eye-tracker device, where the position of the eye fixation relative to the recorded image was annotated. From the same video records, artificial saliency points (SIFT) were measured by computer vision algorithms which were clustered to describe the images with a manageable amount of descriptors. The measured human eye-tracking (fixation pattern) and the estimated saliency points are fused in a statistical model, where the eye-tracking supports us with transition probabilities among the possible image feature points. This HVS-based statistical model results in the estimation of possible tracking paths and region of interest areas of the human vision. The proposed method may help in image saliency analysis, better compression of region of interest areas and in the development of more efficient human-computer-interaction devices.

KW - descriptors

KW - eye-tracking

KW - fixation pattern

KW - human observation

KW - saliency

UR - http://www.scopus.com/inward/record.url?scp=84867134047&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84867134047&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-32436-9_19

DO - 10.1007/978-3-642-32436-9_19

M3 - Conference contribution

AN - SCOPUS:84867134047

SN - 9783642324352

VL - 7252 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 219

EP - 233

BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

ER -