Document Type

Conference Paper

Rights

Available under a Creative Commons Attribution Non-Commercial Share Alike 4.0 International Licence

Disciplines

1.2 COMPUTER AND INFORMATION SCIENCE

Publication Details

Proceedings of 21st ACM Symposium on Applied Computing. pp582-587, ACM, New York. 2006.

Abstract

This paper presents False Positive-Critical Classifiers Comparison a new technique for pairwise comparison of classi- fiers that allow the control of bias. An evaluation of Naıve Bayes, k-Nearest Neighbour and Support Vector Machine classifiers has been carried out on five datasets containing unsolicited and legitimate e-mail messages to confirm the advantage of the technique over Receiver Operating Charac- teristic curves. The evaluation results suggest that the technique may be useful for choosing the better classifier when the ROC curves do not show comprehensive differences, as well as to prove that the difference between two classifiers is not significant, when ROC suggests that it might be. Spam filtering is a typical application for such a comparison tool, as it requires a classifier to be biased toward negative prediction and to have some upper limit on the rate of false positives. Finally the particular evaluation summary is pre-sented, which confirms that Support Vector Machines outperform other methods in most cases, while the Naıve Bayes classifier works well in a narrow, but relevant range of false positive rate.

Download


Share

COinS