Maintain and improve mental health by smart virtual reality serious games

András Sárkány, Zoltán Tősér, Anita L. Verő, A. Lőrincz, Takumi Toyama, Ehsan N. Toosi, Daniel Sonntag

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

Serious games for mental health is seen as the groundwork for assistive technology to maintain and improve mental health. We present a technical system layout we partly implemented for demonstration purposes and highlight vision-based perception and manipulation capabilities. These include physical interactions employing artificial general intelligence in virtual reality applications. We employ hand gesture tracking, as well as an Oculus Rift integrated gaze and eye tracking system. The resulting serious games should eventually cover daily life activities, which we additionally monitor. The dynamic and contextual modelling of obstacles are central issues, and capabilities required for serious games include knowledge about the 3D world. Such knowledge include gaze and hand sensors interpretations for multimedia information extraction in causal relationships. Towards this goal, we envision to make use of virtual reality with a physics engine (rigid and soft body dynamics including collision detection) for the observed objects. We also exploit semantic networks to enable the machine to filter information and infer ongoing complex events including hidden BDI (beliefs, desires, intentions) variables. We see this combination of employed technology as the relevant groundwork for reaching human-level general intelligence and to enable real-world applications. Future applications and user groups we target on include dementia patients.

Original languageEnglish
Title of host publicationCommunications in Computer and Information Science
PublisherSpringer Verlag
Pages220-229
Number of pages10
Volume604
ISBN (Print)9783319322698
DOIs
Publication statusPublished - 2016
Event5th International Conference on Pervasive Computing Paradigms for Mental Health, MindCare 2015 - Milan, Italy
Duration: Sep 24 2015Sep 25 2015

Publication series

NameCommunications in Computer and Information Science
Volume604
ISSN (Print)18650929

Other

Other5th International Conference on Pervasive Computing Paradigms for Mental Health, MindCare 2015
CountryItaly
CityMilan
Period9/24/159/25/15

Fingerprint

Virtual reality
Health
Demonstrations
Physics
Semantics
Engines
Sensors
Serious games

ASJC Scopus subject areas

  • Computer Science(all)

Cite this

Sárkány, A., Tősér, Z., Verő, A. L., Lőrincz, A., Toyama, T., Toosi, E. N., & Sonntag, D. (2016). Maintain and improve mental health by smart virtual reality serious games. In Communications in Computer and Information Science (Vol. 604, pp. 220-229). (Communications in Computer and Information Science; Vol. 604). Springer Verlag. https://doi.org/10.1007/978-3-319-32270-4_22

Maintain and improve mental health by smart virtual reality serious games. / Sárkány, András; Tősér, Zoltán; Verő, Anita L.; Lőrincz, A.; Toyama, Takumi; Toosi, Ehsan N.; Sonntag, Daniel.

Communications in Computer and Information Science. Vol. 604 Springer Verlag, 2016. p. 220-229 (Communications in Computer and Information Science; Vol. 604).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Sárkány, A, Tősér, Z, Verő, AL, Lőrincz, A, Toyama, T, Toosi, EN & Sonntag, D 2016, Maintain and improve mental health by smart virtual reality serious games. in Communications in Computer and Information Science. vol. 604, Communications in Computer and Information Science, vol. 604, Springer Verlag, pp. 220-229, 5th International Conference on Pervasive Computing Paradigms for Mental Health, MindCare 2015, Milan, Italy, 9/24/15. https://doi.org/10.1007/978-3-319-32270-4_22
Sárkány A, Tősér Z, Verő AL, Lőrincz A, Toyama T, Toosi EN et al. Maintain and improve mental health by smart virtual reality serious games. In Communications in Computer and Information Science. Vol. 604. Springer Verlag. 2016. p. 220-229. (Communications in Computer and Information Science). https://doi.org/10.1007/978-3-319-32270-4_22
Sárkány, András ; Tősér, Zoltán ; Verő, Anita L. ; Lőrincz, A. ; Toyama, Takumi ; Toosi, Ehsan N. ; Sonntag, Daniel. / Maintain and improve mental health by smart virtual reality serious games. Communications in Computer and Information Science. Vol. 604 Springer Verlag, 2016. pp. 220-229 (Communications in Computer and Information Science).
@inproceedings{c9843012e7cb4c3bb29a95a94d3c6503,
title = "Maintain and improve mental health by smart virtual reality serious games",
abstract = "Serious games for mental health is seen as the groundwork for assistive technology to maintain and improve mental health. We present a technical system layout we partly implemented for demonstration purposes and highlight vision-based perception and manipulation capabilities. These include physical interactions employing artificial general intelligence in virtual reality applications. We employ hand gesture tracking, as well as an Oculus Rift integrated gaze and eye tracking system. The resulting serious games should eventually cover daily life activities, which we additionally monitor. The dynamic and contextual modelling of obstacles are central issues, and capabilities required for serious games include knowledge about the 3D world. Such knowledge include gaze and hand sensors interpretations for multimedia information extraction in causal relationships. Towards this goal, we envision to make use of virtual reality with a physics engine (rigid and soft body dynamics including collision detection) for the observed objects. We also exploit semantic networks to enable the machine to filter information and infer ongoing complex events including hidden BDI (beliefs, desires, intentions) variables. We see this combination of employed technology as the relevant groundwork for reaching human-level general intelligence and to enable real-world applications. Future applications and user groups we target on include dementia patients.",
author = "Andr{\'a}s S{\'a}rk{\'a}ny and Zolt{\'a}n Tős{\'e}r and Verő, {Anita L.} and A. Lőrincz and Takumi Toyama and Toosi, {Ehsan N.} and Daniel Sonntag",
year = "2016",
doi = "10.1007/978-3-319-32270-4_22",
language = "English",
isbn = "9783319322698",
volume = "604",
series = "Communications in Computer and Information Science",
publisher = "Springer Verlag",
pages = "220--229",
booktitle = "Communications in Computer and Information Science",

}

TY - GEN

T1 - Maintain and improve mental health by smart virtual reality serious games

AU - Sárkány, András

AU - Tősér, Zoltán

AU - Verő, Anita L.

AU - Lőrincz, A.

AU - Toyama, Takumi

AU - Toosi, Ehsan N.

AU - Sonntag, Daniel

PY - 2016

Y1 - 2016

N2 - Serious games for mental health is seen as the groundwork for assistive technology to maintain and improve mental health. We present a technical system layout we partly implemented for demonstration purposes and highlight vision-based perception and manipulation capabilities. These include physical interactions employing artificial general intelligence in virtual reality applications. We employ hand gesture tracking, as well as an Oculus Rift integrated gaze and eye tracking system. The resulting serious games should eventually cover daily life activities, which we additionally monitor. The dynamic and contextual modelling of obstacles are central issues, and capabilities required for serious games include knowledge about the 3D world. Such knowledge include gaze and hand sensors interpretations for multimedia information extraction in causal relationships. Towards this goal, we envision to make use of virtual reality with a physics engine (rigid and soft body dynamics including collision detection) for the observed objects. We also exploit semantic networks to enable the machine to filter information and infer ongoing complex events including hidden BDI (beliefs, desires, intentions) variables. We see this combination of employed technology as the relevant groundwork for reaching human-level general intelligence and to enable real-world applications. Future applications and user groups we target on include dementia patients.

AB - Serious games for mental health is seen as the groundwork for assistive technology to maintain and improve mental health. We present a technical system layout we partly implemented for demonstration purposes and highlight vision-based perception and manipulation capabilities. These include physical interactions employing artificial general intelligence in virtual reality applications. We employ hand gesture tracking, as well as an Oculus Rift integrated gaze and eye tracking system. The resulting serious games should eventually cover daily life activities, which we additionally monitor. The dynamic and contextual modelling of obstacles are central issues, and capabilities required for serious games include knowledge about the 3D world. Such knowledge include gaze and hand sensors interpretations for multimedia information extraction in causal relationships. Towards this goal, we envision to make use of virtual reality with a physics engine (rigid and soft body dynamics including collision detection) for the observed objects. We also exploit semantic networks to enable the machine to filter information and infer ongoing complex events including hidden BDI (beliefs, desires, intentions) variables. We see this combination of employed technology as the relevant groundwork for reaching human-level general intelligence and to enable real-world applications. Future applications and user groups we target on include dementia patients.

UR - http://www.scopus.com/inward/record.url?scp=84964068346&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84964068346&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-32270-4_22

DO - 10.1007/978-3-319-32270-4_22

M3 - Conference contribution

AN - SCOPUS:84964068346

SN - 9783319322698

VL - 604

T3 - Communications in Computer and Information Science

SP - 220

EP - 229

BT - Communications in Computer and Information Science

PB - Springer Verlag

ER -