Sum of ranking differences compares methods or models fairly

Research output: Contribution to journalArticle

145 Citations (Scopus)

Abstract

This review covers a novel approach to comparing methods, based on the sum of ranking differences (SRD). Many method-comparison studies suffer from ambiguity or from comparisons not being quite fair. This problem can be avoided if there are differences between ideal and actual rankings. The absolute values of differences for the ideal and actual ranking are summed up and the procedure is repeated for each (actual) method. The SRD values obtained such a way order the methods simply. If the ideal ranking is not known, it can be replaced by the average (maximum or minimum of all methods or by a known sequence). SRD corresponds to the principle of parsimony and provides an easy tool to evaluate the methods: the smaller the sum the better the method. Models and other items can be similarly ranked. Validation can be carried out using simulated random numbers for comparison: an empirical histogram (bootstrap-like) shows whether the SRD values are far from random. Two case studies (clustering of HPLC columns and prediction of retention data) illustrate and validate the applicability of this novel approach to comparing methods. The technique is entirely general; it can be used in different fields (e.g., for stationary-phase (column) selection in chromatography, model and descriptor selection, comparing analytical and chemometric techniques, determination of panel consistency, etc.). The only prerequisite is that the data can be arranged in matrix form without empty cells.

Original languageEnglish
Pages (from-to)101-109
Number of pages9
JournalTrAC - Trends in Analytical Chemistry
Volume29
Issue number1
DOIs
Publication statusPublished - Jan 2010

Fingerprint

ranking
Chromatography
method
histogram
chromatography
matrix
prediction
comparison

Keywords

  • HPLC-column selection
  • Method comparison
  • Modeling
  • Principal component analysis (PCA)
  • Quantitative structure-activity relationship (QSAR)
  • Quantitative Structure-retention relationships (QSRR)
  • Ranking
  • Sum of ranking differences (SRD)
  • Toxicity prediction
  • Variable selection

ASJC Scopus subject areas

  • Analytical Chemistry
  • Spectroscopy
  • Environmental Chemistry

Cite this

Sum of ranking differences compares methods or models fairly. / Heberger, K.

In: TrAC - Trends in Analytical Chemistry, Vol. 29, No. 1, 01.2010, p. 101-109.

Research output: Contribution to journalArticle

@article{ddb22e692c304268a86d612e25fa32a9,
title = "Sum of ranking differences compares methods or models fairly",
abstract = "This review covers a novel approach to comparing methods, based on the sum of ranking differences (SRD). Many method-comparison studies suffer from ambiguity or from comparisons not being quite fair. This problem can be avoided if there are differences between ideal and actual rankings. The absolute values of differences for the ideal and actual ranking are summed up and the procedure is repeated for each (actual) method. The SRD values obtained such a way order the methods simply. If the ideal ranking is not known, it can be replaced by the average (maximum or minimum of all methods or by a known sequence). SRD corresponds to the principle of parsimony and provides an easy tool to evaluate the methods: the smaller the sum the better the method. Models and other items can be similarly ranked. Validation can be carried out using simulated random numbers for comparison: an empirical histogram (bootstrap-like) shows whether the SRD values are far from random. Two case studies (clustering of HPLC columns and prediction of retention data) illustrate and validate the applicability of this novel approach to comparing methods. The technique is entirely general; it can be used in different fields (e.g., for stationary-phase (column) selection in chromatography, model and descriptor selection, comparing analytical and chemometric techniques, determination of panel consistency, etc.). The only prerequisite is that the data can be arranged in matrix form without empty cells.",
keywords = "HPLC-column selection, Method comparison, Modeling, Principal component analysis (PCA), Quantitative structure-activity relationship (QSAR), Quantitative Structure-retention relationships (QSRR), Ranking, Sum of ranking differences (SRD), Toxicity prediction, Variable selection",
author = "K. Heberger",
year = "2010",
month = "1",
doi = "10.1016/j.trac.2009.09.009",
language = "English",
volume = "29",
pages = "101--109",
journal = "TrAC - Trends in Analytical Chemistry",
issn = "0165-9936",
publisher = "Elsevier",
number = "1",

}

TY - JOUR

T1 - Sum of ranking differences compares methods or models fairly

AU - Heberger, K.

PY - 2010/1

Y1 - 2010/1

N2 - This review covers a novel approach to comparing methods, based on the sum of ranking differences (SRD). Many method-comparison studies suffer from ambiguity or from comparisons not being quite fair. This problem can be avoided if there are differences between ideal and actual rankings. The absolute values of differences for the ideal and actual ranking are summed up and the procedure is repeated for each (actual) method. The SRD values obtained such a way order the methods simply. If the ideal ranking is not known, it can be replaced by the average (maximum or minimum of all methods or by a known sequence). SRD corresponds to the principle of parsimony and provides an easy tool to evaluate the methods: the smaller the sum the better the method. Models and other items can be similarly ranked. Validation can be carried out using simulated random numbers for comparison: an empirical histogram (bootstrap-like) shows whether the SRD values are far from random. Two case studies (clustering of HPLC columns and prediction of retention data) illustrate and validate the applicability of this novel approach to comparing methods. The technique is entirely general; it can be used in different fields (e.g., for stationary-phase (column) selection in chromatography, model and descriptor selection, comparing analytical and chemometric techniques, determination of panel consistency, etc.). The only prerequisite is that the data can be arranged in matrix form without empty cells.

AB - This review covers a novel approach to comparing methods, based on the sum of ranking differences (SRD). Many method-comparison studies suffer from ambiguity or from comparisons not being quite fair. This problem can be avoided if there are differences between ideal and actual rankings. The absolute values of differences for the ideal and actual ranking are summed up and the procedure is repeated for each (actual) method. The SRD values obtained such a way order the methods simply. If the ideal ranking is not known, it can be replaced by the average (maximum or minimum of all methods or by a known sequence). SRD corresponds to the principle of parsimony and provides an easy tool to evaluate the methods: the smaller the sum the better the method. Models and other items can be similarly ranked. Validation can be carried out using simulated random numbers for comparison: an empirical histogram (bootstrap-like) shows whether the SRD values are far from random. Two case studies (clustering of HPLC columns and prediction of retention data) illustrate and validate the applicability of this novel approach to comparing methods. The technique is entirely general; it can be used in different fields (e.g., for stationary-phase (column) selection in chromatography, model and descriptor selection, comparing analytical and chemometric techniques, determination of panel consistency, etc.). The only prerequisite is that the data can be arranged in matrix form without empty cells.

KW - HPLC-column selection

KW - Method comparison

KW - Modeling

KW - Principal component analysis (PCA)

KW - Quantitative structure-activity relationship (QSAR)

KW - Quantitative Structure-retention relationships (QSRR)

KW - Ranking

KW - Sum of ranking differences (SRD)

KW - Toxicity prediction

KW - Variable selection

UR - http://www.scopus.com/inward/record.url?scp=73549117440&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=73549117440&partnerID=8YFLogxK

U2 - 10.1016/j.trac.2009.09.009

DO - 10.1016/j.trac.2009.09.009

M3 - Article

AN - SCOPUS:73549117440

VL - 29

SP - 101

EP - 109

JO - TrAC - Trends in Analytical Chemistry

JF - TrAC - Trends in Analytical Chemistry

SN - 0165-9936

IS - 1

ER -