A combination-of-tools method for learning interpretable fuzzy rule-based classifiers from support vector machines

Tamas Kenesei, J. Roubos, J. Abonyi

Research output: Conference contribution

Abstract

A new approach is proposed for the data-based identificar tion of transparent fuzzy rule-based classifiers. It is observed that fuzzy rule-based classifiers work in a similar manner as kernel function-based support vector machines (SVMs) since both model the input space by nonlinearly maps into a feature space where the decision can be easily made. Accordingly, trained SVM can be used for the construction of fuzzy rule-based classifiers. However, the transformed SVM does not automatically result in an interpretable fuzzy model because the SVM results in a complex rule-base, where the number of rules is approximately 40-60% of the number of the training data. Hence, reduction of the SVM-initialized classifier is an essential task. For this purpose, a three-step reduction algorithm is developed based on the combination of previously published model reduction techniques. In the first step, the identification of the SVM is followed by the application of the Reduced Set method to decrease the number of kernel functions. The reduced SVM is then transformed into a fuzzy rule-based classifier. The interpretability of a fuzzy model highly depends on the distribution of the membership functions. Hence, the second reduction step is achieved by merging similar fuzzy sets based on a similarity measure. Finally, in the third step, an orthogonal least-squares method is used to reduce the number of rules and re-estimate the consequent parameters of the fuzzy rule-based classifier. The proposed approach is applied for the Wisconsin Breast Cancer, Iris and Wine classification problems to compare its performance to other methods.

Original languageEnglish
Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Pages477-486
Number of pages10
Volume4881 LNCS
Publication statusPublished - 2007
Event8th International Conference on Intelligent Data Engineering and Automated Learning, IDEAL 2007 - Birmingham, United Kingdom
Duration: dec. 16 2007dec. 19 2007

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume4881 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other8th International Conference on Intelligent Data Engineering and Automated Learning, IDEAL 2007
CountryUnited Kingdom
CityBirmingham
Period12/16/0712/19/07

Fingerprint

Fuzzy rules
Fuzzy Rules
Support vector machines
Support Vector Machine
Classifiers
Classifier
Learning
Kernel Function
Fuzzy Model
Orthogonal Least Squares
Space Simulation
Wine
Rule Base
Iris
Interpretability
Model Reduction
Membership functions
Fuzzy sets
Feature Space
Least-Squares Analysis

ASJC Scopus subject areas

  • Computer Science(all)
  • Biochemistry, Genetics and Molecular Biology(all)
  • Theoretical Computer Science

Cite this

Kenesei, T., Roubos, J., & Abonyi, J. (2007). A combination-of-tools method for learning interpretable fuzzy rule-based classifiers from support vector machines. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 4881 LNCS, pp. 477-486). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4881 LNCS).

A combination-of-tools method for learning interpretable fuzzy rule-based classifiers from support vector machines. / Kenesei, Tamas; Roubos, J.; Abonyi, J.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 4881 LNCS 2007. p. 477-486 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4881 LNCS).

Research output: Conference contribution

Kenesei, T, Roubos, J & Abonyi, J 2007, A combination-of-tools method for learning interpretable fuzzy rule-based classifiers from support vector machines. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). vol. 4881 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 4881 LNCS, pp. 477-486, 8th International Conference on Intelligent Data Engineering and Automated Learning, IDEAL 2007, Birmingham, United Kingdom, 12/16/07.
Kenesei T, Roubos J, Abonyi J. A combination-of-tools method for learning interpretable fuzzy rule-based classifiers from support vector machines. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 4881 LNCS. 2007. p. 477-486. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
Kenesei, Tamas ; Roubos, J. ; Abonyi, J. / A combination-of-tools method for learning interpretable fuzzy rule-based classifiers from support vector machines. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 4881 LNCS 2007. pp. 477-486 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{e051ebbbac8042f48e68022ca4263b17,
title = "A combination-of-tools method for learning interpretable fuzzy rule-based classifiers from support vector machines",
abstract = "A new approach is proposed for the data-based identificar tion of transparent fuzzy rule-based classifiers. It is observed that fuzzy rule-based classifiers work in a similar manner as kernel function-based support vector machines (SVMs) since both model the input space by nonlinearly maps into a feature space where the decision can be easily made. Accordingly, trained SVM can be used for the construction of fuzzy rule-based classifiers. However, the transformed SVM does not automatically result in an interpretable fuzzy model because the SVM results in a complex rule-base, where the number of rules is approximately 40-60{\%} of the number of the training data. Hence, reduction of the SVM-initialized classifier is an essential task. For this purpose, a three-step reduction algorithm is developed based on the combination of previously published model reduction techniques. In the first step, the identification of the SVM is followed by the application of the Reduced Set method to decrease the number of kernel functions. The reduced SVM is then transformed into a fuzzy rule-based classifier. The interpretability of a fuzzy model highly depends on the distribution of the membership functions. Hence, the second reduction step is achieved by merging similar fuzzy sets based on a similarity measure. Finally, in the third step, an orthogonal least-squares method is used to reduce the number of rules and re-estimate the consequent parameters of the fuzzy rule-based classifier. The proposed approach is applied for the Wisconsin Breast Cancer, Iris and Wine classification problems to compare its performance to other methods.",
keywords = "Classification, Fuzzy classifier, Model reduction, Support vector machine",
author = "Tamas Kenesei and J. Roubos and J. Abonyi",
year = "2007",
language = "English",
isbn = "9783540772255",
volume = "4881 LNCS",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
pages = "477--486",
booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

}

TY - GEN

T1 - A combination-of-tools method for learning interpretable fuzzy rule-based classifiers from support vector machines

AU - Kenesei, Tamas

AU - Roubos, J.

AU - Abonyi, J.

PY - 2007

Y1 - 2007

N2 - A new approach is proposed for the data-based identificar tion of transparent fuzzy rule-based classifiers. It is observed that fuzzy rule-based classifiers work in a similar manner as kernel function-based support vector machines (SVMs) since both model the input space by nonlinearly maps into a feature space where the decision can be easily made. Accordingly, trained SVM can be used for the construction of fuzzy rule-based classifiers. However, the transformed SVM does not automatically result in an interpretable fuzzy model because the SVM results in a complex rule-base, where the number of rules is approximately 40-60% of the number of the training data. Hence, reduction of the SVM-initialized classifier is an essential task. For this purpose, a three-step reduction algorithm is developed based on the combination of previously published model reduction techniques. In the first step, the identification of the SVM is followed by the application of the Reduced Set method to decrease the number of kernel functions. The reduced SVM is then transformed into a fuzzy rule-based classifier. The interpretability of a fuzzy model highly depends on the distribution of the membership functions. Hence, the second reduction step is achieved by merging similar fuzzy sets based on a similarity measure. Finally, in the third step, an orthogonal least-squares method is used to reduce the number of rules and re-estimate the consequent parameters of the fuzzy rule-based classifier. The proposed approach is applied for the Wisconsin Breast Cancer, Iris and Wine classification problems to compare its performance to other methods.

AB - A new approach is proposed for the data-based identificar tion of transparent fuzzy rule-based classifiers. It is observed that fuzzy rule-based classifiers work in a similar manner as kernel function-based support vector machines (SVMs) since both model the input space by nonlinearly maps into a feature space where the decision can be easily made. Accordingly, trained SVM can be used for the construction of fuzzy rule-based classifiers. However, the transformed SVM does not automatically result in an interpretable fuzzy model because the SVM results in a complex rule-base, where the number of rules is approximately 40-60% of the number of the training data. Hence, reduction of the SVM-initialized classifier is an essential task. For this purpose, a three-step reduction algorithm is developed based on the combination of previously published model reduction techniques. In the first step, the identification of the SVM is followed by the application of the Reduced Set method to decrease the number of kernel functions. The reduced SVM is then transformed into a fuzzy rule-based classifier. The interpretability of a fuzzy model highly depends on the distribution of the membership functions. Hence, the second reduction step is achieved by merging similar fuzzy sets based on a similarity measure. Finally, in the third step, an orthogonal least-squares method is used to reduce the number of rules and re-estimate the consequent parameters of the fuzzy rule-based classifier. The proposed approach is applied for the Wisconsin Breast Cancer, Iris and Wine classification problems to compare its performance to other methods.

KW - Classification

KW - Fuzzy classifier

KW - Model reduction

KW - Support vector machine

UR - http://www.scopus.com/inward/record.url?scp=38449094329&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=38449094329&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:38449094329

SN - 9783540772255

VL - 4881 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 477

EP - 486

BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

ER -