Comparison of some classification algorithms based on deterministic and nondeterministic decision rules
KAUST DepartmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
Applied Mathematics and Computational Science Program
Extensions of Dynamic Programming, Machine Learning and Discrete Optimization Research Group
Permanent link to this recordhttp://hdl.handle.net/10754/564258
MetadataShow full item record
AbstractWe discuss two, in a sense extreme, kinds of nondeterministic rules in decision tables. The first kind of rules, called as inhibitory rules, are blocking only one decision value (i.e., they have all but one decisions from all possible decisions on their right hand sides). Contrary to this, any rule of the second kind, called as a bounded nondeterministic rule, can have on the right hand side only a few decisions. We show that both kinds of rules can be used for improving the quality of classification. In the paper, two lazy classification algorithms of polynomial time complexity are considered. These algorithms are based on deterministic and inhibitory decision rules, but the direct generation of rules is not required. Instead of this, for any new object the considered algorithms extract from a given decision table efficiently some information about the set of rules. Next, this information is used by a decision-making procedure. The reported results of experiments show that the algorithms based on inhibitory decision rules are often better than those based on deterministic decision rules. We also present an application of bounded nondeterministic rules in construction of rule based classifiers. We include the results of experiments showing that by combining rule based classifiers based on minimal decision rules with bounded nondeterministic rules having confidence close to 1 and sufficiently large support, it is possible to improve the classification quality. © 2010 Springer-Verlag.
PublisherSpringer Science + Business Media
Conference/Event nameRough Set and Knowledge Technology Conference, RSKT 2008
Showing items related by title, author, creator and subject.
Minimization of Decision Tree Average Depth for Decision Tables with Many-valued DecisionsAzad, Mohammad; Moshkov, Mikhail (Procedia Computer Science, Elsevier BV, 2014-09-13) [Conference Paper]The paper is devoted to the analysis of greedy algorithms for the minimization of average depth of decision trees for decision tables such that each row is labeled with a set of decisions. The goal is to find one decision from the set of decisions. When we compare with the optimal result obtained from dynamic programming algorithm, we found some greedy algorithms produces results which are close to the optimal result for the minimization of average depth of decision trees.
Optimization of decision rule complexity for decision tables with many-valued decisionsAzad, Mohammad; Chikalov, Igor; Moshkov, Mikhail (2013 IEEE International Conference on Systems, Man, and Cybernetics, Institute of Electrical and Electronics Engineers (IEEE), 2013-10) [Conference Paper]We describe new heuristics to construct decision rules for decision tables with many-valued decisions from the point of view of length and coverage which are enough good. We use statistical test to find leaders among the heuristics. After that, we compare our results with optimal result obtained by dynamic programming algorithms. The average percentage of relative difference between length (coverage) of constructed and optimal rules is at most 6.89% (15.89%, respectively) for leaders which seems to be a promising result. © 2013 IEEE.
Decision rules for decision tables with many-valued decisionsChikalov, Igor; Zielosko, Beata (Rough Sets and Knowledge Technology, Springer Science + Business Media, 2011) [Conference Paper]In the paper, authors presents a greedy algorithm for construction of exact and partial decision rules for decision tables with many-valued decisions. Exact decision rules can be 'over-fitted', so instead of exact decision rules with many attributes, it is more appropriate to work with partial decision rules with smaller number of attributes. Based on results for set cover problem authors study bounds on accuracy of greedy algorithm for exact and partial decision rule construction, and complexity of the problem of minimization of decision rule length. © 2011 Springer-Verlag.