Statistical Comparisons of Classifiers over Multiple Data Sets on ShortScience.org

dl.acm.org
sci-hub
scholar.google.com

Statistical Comparisons of Classifiers over Multiple Data Sets
Dem\v{s}ar, Janez
JMLR.org J. Mach. Learn. Res. - 2006 via Local Bibsonomy
Keywords: significance, testing, prediction, classification

Summaries/Notes 1

[link] Summary by Martin Thoma 7 years ago

Describes how to compare classifiers when they were evaluated on multiple datasets (e.g. CIFAR 10, MNIST and SVHN). Recommends Wilcoxon signed ranks test and Friedman test with the corresponding post-hoc tests. Introduce CD (critical difference) diagrams.

* McNemar test and 5x2cv are good when comparing two classifiers on one dataset
* Describes the Wilcoxon Signed-Ranks Test in section 3.1.3 in detail

Your comment:

Write your summary here (You can use $\LaTeX$ and markdown syntax):

Anon Private