C4.5: Programs for Machine Learning (書評)

https://sci2s.ugr.es/sites/default/files/ficherosPublicaciones/1371_2011-pr-galar-ovo-ova.pdf

An overview of ensemble methods for binary classifiers in multi-class problems: Experimental study on one-vs-one and one-vs-all schemes

The notion of uncertainty is of major importance in machine learning and constitutes a key element of machine learning methodology. In line with the statistical tradition, uncertainty has long been perceived as almost synonymous with standard probability and probabilistic predictions. Yet, due to the steadily increasing relevance of machine learning for practical applications and related issues such as safety requirements, new problems and challenges have recently been identified by machine learning scholars, and these problems may call for new methodological developments. In particular, this includes the importance of distinguishing between (at least) two different types of uncertainty, often referred to as aleatoric and epistemic. In this paper, we provide an introduction to the topic of uncertainty in machine learning as well as an overview of attempts so far at handling uncertainty in general and formalizing this distinction in particular.

/pdf/aleatoric-and-epistemic-uncertainty-in-machine-learning-an-5c7lignu40.pdf

Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods

Meta-learning, or learning to learn, is the science of systematically observing how different machine learning approaches perform on a wide range of learning tasks, and then learning from this experience, or meta-data, to learn new tasks much faster than otherwise possible. Not only does this dramatically speed up and improve the design of machine learning pipelines or neural architectures, it also allows us to replace hand-engineered algorithms with novel approaches learned in a data-driven way. In this chapter, we provide an overview of the state of the art in this fascinating and continuously evolving field.

Meta-Learning: A Survey

This paper introduces a novel fuzzy rule-based classification method called FURIA, which is short for Fuzzy Unordered Rule Induction Algorithm. FURIA extends the well-known RIPPER algorithm, a state-of-the-art rule learner, while preserving its advantages, such as simple and comprehensible rule sets. In addition, it includes a number of modifications and extensions. In particular, FURIA learns fuzzy rules instead of conventional rules and unordered rule sets instead of rule lists. Moreover, to deal with uncovered examples, it makes use of an efficient rule stretching method. Experimental results show that FURIA significantly outperforms the original RIPPER, as well as other classifiers such as C4.5, in terms of classification accuracy.

/pdf/furia-an-algorithm-for-unordered-fuzzy-rule-induction-5eba52xjsk.pdf

FURIA: an algorithm for unordered fuzzy rule induction

The label ranking problem consists of learning a model that maps instances to total orders over a finite set of predefined labels. This paper introduces new methods for label ranking that complement and improve upon existing approaches. More specifically, we propose extensions of two methods that have been used extensively for classification and regression so far, namely instance-based learning and decision tree induction. The unifying element of the two methods is a procedure for locally estimating predictive probability models for label rankings.

/pdf/decision-tree-and-instance-based-learning-for-label-ranking-520kb0t8sa.pdf

Decision tree and instance-based learning for label ranking

This paper introduces a fuzzy-rule-based classification method called fuzzy round robin [repeated incremental pruning to produce error reduction (RIPPER)] (FR3). As the name suggests, FR3 builds upon the RIPPER algorithm, a state-of-the-art rule learner. More specifically, in the context of polychotomous classification, it uses a fuzzy extension of RIPPER as a base learner within a round robin scheme, and thus, can be seen as a fuzzy variant of the R3 learner that has recently been introduced in the literature. A key feature of FR3, in comparison with its nonfuzzy counterpart, is its ability to represent different facets of uncertainty involved in a classification decision in a more faithful way. FR3 thus provides the basis for implementing ldquoreliable classifiersrdquo that may abstain from a decision when not being sure enough, or at least indicate that a classification is not fully supported by the empirical evidence at hand. Besides, our experimental results show that FR3 outperforms R3 in terms of classification accuracy, and therefore, suggest that it produces predictions that are not only more reliable but also more accurate. The superb classification performance of FR3 is furthermore confirmed by comparing it to other state-of-the-art (fuzzy) rule learners.

/pdf/fr3-a-fuzzy-rule-learner-for-inducing-reliable-classifiers-23jk86eep4.pdf

FR3: A Fuzzy Rule Learner for Inducing Reliable Classifiers

In recent years, a number of machine learning algorithms have been developed for the problem of ordinal classification. These algorithms try to exploit, in one way or the other, the order information of the problem, essentially relying on the assumption that the ordinal structure of the set of class labels is also reflected in the topology of the instance space. The purpose of this paper is to investigate, on an experimental basis, the validity of this assumption. Moreover, we seek to answer the question to what extent existing techniques and learning algorithms for ordinal classification are able to exploit order information and which properties of these techniques are important in this regard.

/pdf/is-an-ordinal-class-structure-useful-in-classifier-learning-klz21cxhlt.pdf

Is an ordinal class structure useful in classifier learning

This paper elaborates on a novel fuzzy rule-based classification method called FURIA, which is short for “Fuzzy Unordered Rule Induction Algorithm” FURIA has recently been developed as an extension of the well-known RIPPER algorithm It learns fuzzy rules instead of conventional rules and unordered rule sets instead of rule lists Moreover, to deal with uncovered examples, it makes use of an efficient rule stretching method First experimental results have shown that FURIA significantly outperforms the original RIPPER in terms of classification accuracy Elaborating on the advantages of a fuzzy approach, this paper makes an attempt to distill and quantify the influence of rule fuzzification on the performance of the algorithm Moreover, going beyond the conventional classification problem, we investigate the performance of FURIA in the context of bipartite ranking, in which a fuzzy approach appears to be even more appealing

Jens Christian Hühn

Papers

FURIA: an algorithm for unordered fuzzy rule induction

Decision tree and instance-based learning for label ranking

FR3: A Fuzzy Rule Learner for Inducing Reliable Classifiers

Is an ordinal class structure useful in classifier learning

An Analysis of the FURIA Algorithm for Fuzzy Rule Induction