Modelling methods and cross-validation variants in QSAR: a multi-level analysis$

A. Rácz, D. Bajusz, K. Héberger

Research output: Contribution to journalArticle

3 Citations (Scopus)


Prediction performance often depends on the cross- and test validation protocols applied. Several combinations of different cross-validation variants and model-building techniques were used to reveal their complexity. Two case studies (acute toxicity data) were examined, applying five-fold cross-validation (with random, contiguous and Venetian blind forms) and leave-one-out cross-validation (CV). External test sets showed the effects and differences between the validation protocols. The models were generated with multiple linear regression (MLR), principal component regression (PCR), partial least squares (PLS) regression, artificial neural networks (ANN) and support vector machines (SVM). The comparisons were made by the sum of ranking differences (SRD) and factorial analysis of variance (ANOVA). The largest bias and variance could be assigned to the MLR method and contiguous block cross-validation. SRD can provide a unique and unambiguous ranking of methods and CV variants. Venetian blind cross-validation is a promising tool. The generated models were also compared based on their basic performance parameters (r2 and Q2). MLR produced the largest gap, while PCR gave the smallest. Although PCR is the best validated and balanced technique, SVM always outperformed the other methods, when experimental values were the benchmark. Variable selection was advantageous, and the modelling had a larger influence than CV variants.

Original languageEnglish
Pages (from-to)661-674
Number of pages14
JournalSAR and QSAR in environmental research
Issue number9
Publication statusPublished - Sep 2 2018


  • ANN
  • MLR
  • PCR
  • PLS
  • QSAR
  • SRD
  • SVM
  • cross-validation
  • toxicity
  • validation

ASJC Scopus subject areas

  • Bioengineering
  • Molecular Medicine
  • Drug Discovery

Fingerprint Dive into the research topics of 'Modelling methods and cross-validation variants in QSAR: a multi-level analysis<sup>$</sup>'. Together they form a unique fingerprint.

  • Cite this