Multi-Level Comparison of Machine Learning Classifiers and Their Performance Metrics

Anita Rácz, Dávid Bajusz, Károly Héberger

Research output: Contribution to journalArticle

1 Citation (Scopus)


Machine learning classification algorithms are widely used for the prediction and classification of the different properties of molecules such as toxicity or biological activity. the prediction of toxic vs. non-toxic molecules is important due to testing on living animals, which has ethical and cost drawbacks as well. The quality of classification models can be determined with several performance parameters. which often give conflicting results. In this study, we performed a multi-level comparison with the use of different performance metrics and machine learning classification methods. Well-established and standardized protocols for the machine learning tasks were used in each case. The comparison was applied to three datasets (acute and aquatic toxicities) and the robust, yet sensitive, sum of ranking differences (SRD) and analysis of variance (ANOVA) were applied for evaluation. The effect of dataset composition (balanced vs. imbalanced) and 2-class vs. multiclass classification scenarios was also studied. Most of the performance metrics are sensitive to dataset composition, especially in 2-class classification problems. The optimal machine learning algorithm also depends significantly on the composition of the dataset.

Original languageEnglish
JournalMolecules (Basel, Switzerland)
Issue number15
Publication statusPublished - Aug 1 2019


  • ROC
  • classifiers
  • machine learning
  • performance metrics
  • ranking
  • toxicity prediction

ASJC Scopus subject areas

  • Analytical Chemistry
  • Chemistry (miscellaneous)
  • Molecular Medicine
  • Pharmaceutical Science
  • Drug Discovery
  • Physical and Theoretical Chemistry
  • Organic Chemistry

Fingerprint Dive into the research topics of 'Multi-Level Comparison of Machine Learning Classifiers and Their Performance Metrics'. Together they form a unique fingerprint.

  • Cite this