Comparison of some classification algorithms based on deterministic and nondeterministic decision rules
Type
Conference PaperAuthors
Delimata, PawełMarszał-Paszek, Barbara
Moshkov, Mikhail

Paszek, Piotr
Skowron, Andrzej
Suraj, Zbigniew
KAUST Department
Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) DivisionApplied Mathematics and Computational Science Program
Extensions of Dynamic Programming, Machine Learning and Discrete Optimization Research Group
Date
2010Permanent link to this record
http://hdl.handle.net/10754/564258
Metadata
Show full item recordAbstract
We discuss two, in a sense extreme, kinds of nondeterministic rules in decision tables. The first kind of rules, called as inhibitory rules, are blocking only one decision value (i.e., they have all but one decisions from all possible decisions on their right hand sides). Contrary to this, any rule of the second kind, called as a bounded nondeterministic rule, can have on the right hand side only a few decisions. We show that both kinds of rules can be used for improving the quality of classification. In the paper, two lazy classification algorithms of polynomial time complexity are considered. These algorithms are based on deterministic and inhibitory decision rules, but the direct generation of rules is not required. Instead of this, for any new object the considered algorithms extract from a given decision table efficiently some information about the set of rules. Next, this information is used by a decision-making procedure. The reported results of experiments show that the algorithms based on inhibitory decision rules are often better than those based on deterministic decision rules. We also present an application of bounded nondeterministic rules in construction of rule based classifiers. We include the results of experiments showing that by combining rule based classifiers based on minimal decision rules with bounded nondeterministic rules having confidence close to 1 and sufficiently large support, it is possible to improve the classification quality. © 2010 Springer-Verlag.Citation
Delimata, P., Marszał-Paszek, B., Moshkov, M., Paszek, P., Skowron, A., & Suraj, Z. (2010). Comparison of Some Classification Algorithms Based on Deterministic and Nondeterministic Decision Rules. Transactions on Rough Sets XII, 90–105. doi:10.1007/978-3-642-14467-7_5Publisher
Springer NatureConference/Event name
Rough Set and Knowledge Technology Conference, RSKT 2008ISBN
3642144667; 9783642144660ae974a485f413a2113503eed53cd6c53
10.1007/978-3-642-14467-7_5
Scopus Count
Related items
Showing items related by title, author, creator and subject.
-
Greedy Algorithm for the Construction of Approximate Decision Rules for Decision Tables with Many-Valued DecisionsAzad, Mohammad; Moshkov, Mikhail; Zielosko, Beata (Lecture Notes in Computer Science, Springer Nature, 2016-10-21) [Book Chapter]The paper is devoted to the study of a greedy algorithm for construction of approximate decision rules. This algorithm is applicable to decision tables with many-valued decisions where each row is labeled with a set of decisions. For a given row, we should find a decision from the set attached to this row. We consider bounds on the precision of this algorithm relative to the length of rules. To illustrate proposed approach we study a problem of recognition of labels of points in the plain. This paper contains also results of experiments with modified decision tables from UCI Machine Learning Repository.
-
Decision and Inhibitory Trees for Decision Tables with Many-Valued DecisionsAzad, Mohammad (2018-06-06) [Dissertation]
Advisor: Moshkov, Mikhail
Committee members: Bajic, Vladimir B.; Zhang, Xiangliang; Boros, EndreDecision trees are one of the most commonly used tools in decision analysis, knowledge representation, machine learning, etc., for its simplicity and interpretability. We consider an extension of dynamic programming approach to process the whole set of decision trees for the given decision table which was previously only attainable by brute-force algorithms. We study decision tables with many-valued decisions (each row may contain multiple decisions) because they are more reasonable models of data in many cases. To address this problem in a broad sense, we consider not only decision trees but also inhibitory trees where terminal nodes are labeled with “̸= decision”. Inhibitory trees can sometimes describe more knowledge from datasets than decision trees. As for cost functions, we consider depth or average depth to minimize time complexity of trees, and the number of nodes or the number of the terminal, or nonterminal nodes to minimize the space complexity of trees. We investigate the multi-stage optimization of trees relative to some cost functions, and also the possibility to describe the whole set of strictly optimal trees. Furthermore, we study the bi-criteria optimization cost vs. cost and cost vs. uncertainty for decision trees, and cost vs. cost and cost vs. completeness for inhibitory trees. The most interesting application of the developed technique is the creation of multi-pruning and restricted multi-pruning approaches which are useful for knowledge representation and prediction. The experimental results show that decision trees constructed by these approaches can often outperform the decision trees constructed by the CART algorithm. Another application includes the comparison of 12 greedy heuristics for single- and bi-criteria optimization (cost vs. cost) of trees. We also study the three approaches (decision tables with many-valued decisions, decision tables with most common decisions, and decision tables with generalized decisions) to handle inconsistency of decision tables. We also analyze the time complexity of decision and inhibitory trees over arbitrary sets of attributes represented by information systems in the frameworks of local (when we can use in trees only attributes from problem description) and global (when we can use in trees arbitrary attributes from the information system) approaches. -
Decision and Inhibitory Trees and Rules for Decision Tables with Many-valued DecisionsAlsolami, Fawaz; Azad, Mohammad; Chikalov, Igor; Moshkov, Mikhail (Springer Nature, 2019-05-21) [Book]The results presented here (including the assessment of a new tool – inhibitory trees) offer valuable tools for researchers in the areas of data mining, knowledge discovery, and machine learning, especially those whose work involves decision tables with many-valued decisions. The authors consider various examples of problems and corresponding decision tables with many-valued decisions, discuss the difference between decision and inhibitory trees and rules, and develop tools for their analysis and design. Applications include the study of totally optimal (optimal in relation to a number of criteria simultaneously) decision and inhibitory trees and rules; the comparison of greedy heuristics for tree and rule construction as single-criterion and bi-criteria optimization algorithms; and the development of a restricted multi-pruning approach used in classification and knowledge representation.