Validity of reliability: Comparison of interrater reliabilities of psychopathological symptoms

Bertalan Pethõ, Gábor Tusnády, A. Vargha, Judit Tolna, Márta Farkas, Györgyi Vizkeleti, Agoston Tóth, András Szilágyi, I. Bitter, András Kelemen, P. Czobor

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

We have tested the stability of interrater reliability of psychiatric symptoms over a quarter of century using 2 rating scales. Interrater reliabilities of items of 2 psychiatric rating scales employed by 2 consecutive follow-ups were compared. Interrater reliabilites proved to be by and large stable. Interrater reliability depends on the standard deviation of the items scores. In addition to the traditional approach, a new statistical method for unifying the assessments from multiple raters is also presented. Using this method, we demonstrated that probabilities of correct ratings are higher in the absence of manifest symptoms, or in the presence of symptoms, as compared with cases characterized by middle scores. To interpret the relationships revealed in the setting of the experiment, we introduce for its theoretical designation the term "validity of reliability." It is recommended for evaluation of results of rating scales in the context of psychiatric nosology.

Original languageEnglish
Pages (from-to)606-613
Number of pages8
JournalJournal of Nervous and Mental Disease
Volume195
Issue number7
DOIs
Publication statusPublished - Jul 2007

Fingerprint

Reproducibility of Results
Psychiatry

Keywords

  • Interrater reliability
  • Long-term follow-up
  • Psychopathological symptoms
  • Validity

ASJC Scopus subject areas

  • Psychiatry and Mental health

Cite this

Validity of reliability : Comparison of interrater reliabilities of psychopathological symptoms. / Pethõ, Bertalan; Tusnády, Gábor; Vargha, A.; Tolna, Judit; Farkas, Márta; Vizkeleti, Györgyi; Tóth, Agoston; Szilágyi, András; Bitter, I.; Kelemen, András; Czobor, P.

In: Journal of Nervous and Mental Disease, Vol. 195, No. 7, 07.2007, p. 606-613.

Research output: Contribution to journalArticle

Pethõ, Bertalan ; Tusnády, Gábor ; Vargha, A. ; Tolna, Judit ; Farkas, Márta ; Vizkeleti, Györgyi ; Tóth, Agoston ; Szilágyi, András ; Bitter, I. ; Kelemen, András ; Czobor, P. / Validity of reliability : Comparison of interrater reliabilities of psychopathological symptoms. In: Journal of Nervous and Mental Disease. 2007 ; Vol. 195, No. 7. pp. 606-613.
@article{95435d3b9df74ac799cd850ba7a2fa54,
title = "Validity of reliability: Comparison of interrater reliabilities of psychopathological symptoms",
abstract = "We have tested the stability of interrater reliability of psychiatric symptoms over a quarter of century using 2 rating scales. Interrater reliabilities of items of 2 psychiatric rating scales employed by 2 consecutive follow-ups were compared. Interrater reliabilites proved to be by and large stable. Interrater reliability depends on the standard deviation of the items scores. In addition to the traditional approach, a new statistical method for unifying the assessments from multiple raters is also presented. Using this method, we demonstrated that probabilities of correct ratings are higher in the absence of manifest symptoms, or in the presence of symptoms, as compared with cases characterized by middle scores. To interpret the relationships revealed in the setting of the experiment, we introduce for its theoretical designation the term {"}validity of reliability.{"} It is recommended for evaluation of results of rating scales in the context of psychiatric nosology.",
keywords = "Interrater reliability, Long-term follow-up, Psychopathological symptoms, Validity",
author = "Bertalan Peth{\~o} and G{\'a}bor Tusn{\'a}dy and A. Vargha and Judit Tolna and M{\'a}rta Farkas and Gy{\"o}rgyi Vizkeleti and Agoston T{\'o}th and Andr{\'a}s Szil{\'a}gyi and I. Bitter and Andr{\'a}s Kelemen and P. Czobor",
year = "2007",
month = "7",
doi = "10.1097/NMD.0b013e318093f45d",
language = "English",
volume = "195",
pages = "606--613",
journal = "Journal of Nervous and Mental Disease",
issn = "0022-3018",
publisher = "Lippincott Williams and Wilkins",
number = "7",

}

TY - JOUR

T1 - Validity of reliability

T2 - Comparison of interrater reliabilities of psychopathological symptoms

AU - Pethõ, Bertalan

AU - Tusnády, Gábor

AU - Vargha, A.

AU - Tolna, Judit

AU - Farkas, Márta

AU - Vizkeleti, Györgyi

AU - Tóth, Agoston

AU - Szilágyi, András

AU - Bitter, I.

AU - Kelemen, András

AU - Czobor, P.

PY - 2007/7

Y1 - 2007/7

N2 - We have tested the stability of interrater reliability of psychiatric symptoms over a quarter of century using 2 rating scales. Interrater reliabilities of items of 2 psychiatric rating scales employed by 2 consecutive follow-ups were compared. Interrater reliabilites proved to be by and large stable. Interrater reliability depends on the standard deviation of the items scores. In addition to the traditional approach, a new statistical method for unifying the assessments from multiple raters is also presented. Using this method, we demonstrated that probabilities of correct ratings are higher in the absence of manifest symptoms, or in the presence of symptoms, as compared with cases characterized by middle scores. To interpret the relationships revealed in the setting of the experiment, we introduce for its theoretical designation the term "validity of reliability." It is recommended for evaluation of results of rating scales in the context of psychiatric nosology.

AB - We have tested the stability of interrater reliability of psychiatric symptoms over a quarter of century using 2 rating scales. Interrater reliabilities of items of 2 psychiatric rating scales employed by 2 consecutive follow-ups were compared. Interrater reliabilites proved to be by and large stable. Interrater reliability depends on the standard deviation of the items scores. In addition to the traditional approach, a new statistical method for unifying the assessments from multiple raters is also presented. Using this method, we demonstrated that probabilities of correct ratings are higher in the absence of manifest symptoms, or in the presence of symptoms, as compared with cases characterized by middle scores. To interpret the relationships revealed in the setting of the experiment, we introduce for its theoretical designation the term "validity of reliability." It is recommended for evaluation of results of rating scales in the context of psychiatric nosology.

KW - Interrater reliability

KW - Long-term follow-up

KW - Psychopathological symptoms

KW - Validity

UR - http://www.scopus.com/inward/record.url?scp=34447516656&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34447516656&partnerID=8YFLogxK

U2 - 10.1097/NMD.0b013e318093f45d

DO - 10.1097/NMD.0b013e318093f45d

M3 - Article

C2 - 17632252

AN - SCOPUS:34447516656

VL - 195

SP - 606

EP - 613

JO - Journal of Nervous and Mental Disease

JF - Journal of Nervous and Mental Disease

SN - 0022-3018

IS - 7

ER -