Rule Based Data Mining Classifier: A Comprehensive Guide 101

Data Mining plays a significant role in understanding data. Data is so huge nowadays, so it is crucial to extract the required information. This article will learn a new Rule Based Data Mining classifier for classifying data and predicting class labels. This mining technique is widely used in various real-world business applications in machine learning. A rule-based classifier helps classify data and predict the possible outcome when rules scenarios are adequately defined.

Let’s dive into the Rule Based Data Mining Classifier in detail with examples.

Table of Contents

• What is a Rule Based Data Mining Classifier?
  • Properties of Rule Based Data Mining Classifiers
  • Assessment of Rule
  • Characteristics of Rule Based Data Mining Classifiers
  • Advantages of Rule Based Data Mining Classifiers
  • Example
• Direct Algorithms for Extracting Rules
  • Basic Algorithm
  • Sequential Covering Algorithm
• Conclusion

What is a Rule Based Data Mining Classifier?

The Rule Based Data Mining Classifier is a well-known technique used for data mining. Rules are a good way of representing information and can easily be read and understood. The efficiency of a rule-based classifier depends on factors such as the quality of the rules,  rule ordering, and properties of the set of rules.

The idea behind rule based Data Mining classifiers is to find regularities and different scenarios in data expressed in the IF-THEN rule. A collection of IF-THEN rules is used for classification and predicting the outcome.  IF-THEN rules are defined as

IF condition THEN conclusion

Properties of Rule Based Data Mining Classifiers

Let’s define the significant properties of the Rule Based Data Mining Classifier to understand it better.

• Rule Antecedent: The Left Hand Side(“IF” part) of a rule is called the rule antecedent or condition. The antecedent may have one or more conditions, which are logically ANDed. These conditions are nothing but splitting criteria that are logically ANDed. 

IF condition1 AND condition2 THEN conclusion

The first splitting criteria is a root node or start node.

• Rule Consequent: The Right Hand Side(“THEN” part) of a rule is called the rule consequent. Rule consequent consists of class prediction. The class prediction is the leaf node or end node.

Assessment of Rule

Rule can be accessed based on two factors. Let’s define a few parameters first.

n_a = number of records covered by the rule(R).

n_c = number of records correctly classified by rule(R).

n = Total number of records

• Coverage of a rule: Fraction of records that satisfy the rule’s antecedent describes rule coverage.

Coverage (R) = n_a  / |n| 

• Accuracy of a rule: Fraction of records that meet the antecedent and consequent value defines rule accuracy.

Accuracy (R) = n_c / n_a

Characteristics of Rule Based Data Mining Classifiers

Rule Based Data Mining classifiers possess two significant characteristics:

1) Rules may not be mutually exclusive.

Different rules are generated for data, so it is possible that many rules can cover the same record. That is why rules are called non-mutually exclusive.

The solution to make rules mutually exclusive

Two solutions are there such that the record is covered by at most one rule and make it mutually exclusive.

• Ordered rule set: Rank the rules according to their priority, and the class corresponding to the highest-ranked rule is taken as the final class.
• Unordered rule set: Votes are assigned to each class depending on their weights.

2) Rules may not be exhaustive.

It is possible that some of the data entries may not be covered by any of the rules; thus, rules are called not to be exhaustive.

The solution is to make rules exhaustive.

To make rules exhaustive such that the record is covered by at least one rule use the below solution.

• Use a default class: If none of the rules covered the record, assign it the default class.

Advantages of Rule Based Data Mining Classifiers

• Highly expressive.
• Easy to interpret.
• Easy to generate.
• Capability to classify new records rapidly.
• Performance is comparable to other classifiers.

Example

Let’s consider a simple data set to find gender based on height, weight, and foot size.

Apply the IF-THEN rule to split the data and gender prediction.

According to the above diagram, if height is greater than 5.9 ft or if height is less than or equal to 5.9 ft and weight is greater than 150 lbs, and foot size is greater than or equal to 10 inches, then Gender is classified as ‘male.’ And if height is less than 5.9 ft and weight is less than or equal to 150 lbs or if height is less than 5.9 ft and foot size is less than 10 inches then Gender is classified as ‘female.’ 

Assessment of Rule

Let’s assess the rule ‘height is greater than 5.9 ft’ and say it R1

Here, the total number of records, i.e., n = 8

Number of records covered by rule R1 is n_a= 3

Number of records correctly classified by rule R1 is n_c= 3

coverage(R) = n_a  / |n| = ⅜ = 37.5%

accuracy(R) = n_c / n_a  = 3/3 = 100%

Here, coverage of rule R1 is 37.5%, and accuracy is 100%.

This example shows that the optimal splitting attribute and splitting value need to be identified for an optimum rule-based classifier. It is used for classification as well as predicting class labels. If data spilling is done correctly and optimally, you can use it in various real-world business applications. 

Direct Algorithms for Extracting Rules

Let’s talk about a few direct algorithms that extract rules directly from data.

1) Basic Algorithm

The first algorithm is a fundamental algorithm called 1R Algorithm (Learn-One-Rule Algorithm)

1R Algorithm

1R is the easiest algorithm based on a simple classification rule. In this algorithm, rules are created to test each attribute/feature.

Pseudocode of 1R Algorithm

• For each attribute/feature like height/weight,
• For each categorical value or range interval for the numerical value of that attribute, make a rule as follows;
  • Count how often each value of class appears to
  • Find the most frequent class
  • Make the rule assign that class to this attribute-value pair
  • Calculate the error rate of the rules of each attribute
  • Choose the attribute with the lowest error rate

Problems with 1R algorithm

• Overfitting is likely to occur
• Noise sensitive

Solution to problems of the 1R algorithm is the Sequential Covering Algorithm.

2) Sequential Covering Algorithm

Sequential Covering Algorithms are the most widely used rule based Data Mining algorithms. In this kind of algorithm, rules are learned sequentially, one at a time. Ideally, Sequential Algorithms define rules to cover the maximum possible records of one class and none of the other classes. Once the rule is learned, the records covered by it are removed, and the process keeps on repeating the remaining data. 

There are many sequential algorithms like PRISM, FOIL, AQ, CN2, and RIPPER. 

Sequential Covering Algorithm Steps 

Step 1: Rule Growing

Start from an empty rule. Grow a rule using the 1R algorithm such that the rule covers the majority of records of the class.

Step 2: Instance Elimination

Remove the records covered by the previous rule. This step ensures that the following rule will differ from the previous one. It improves the accuracy of the rule as well.

Step 3: Rule Evaluation

Evaluate each rule’s accuracy. Repeat the above two steps until a stopping criterion is met.

Step 4: Stopping Criteria

If the accuracy of the rule is not up to mark, then discard that rule.

Step 5: Rule Pruning

Calculate the error rate at every step similar to the 1R algorithm. Suppose the error rate increases; prune that rule and again compare the error rate before and after pruning and take the best decision. If rule pruning is unnecessary, add that rule to the existing ruleset.

Conclusion

Rule Based Data Mining Classifier is a direct approach for data mining. This classifier is simple and more easily interpretable than regular data mining algorithms. These are learning sets of rules which are implemented using the IF-THEN clause. It works very well with both numerical as well as categorical data. Just try it with your dataset to get a real feel of it. In case you want to export data from a source of your choice into your desired Database/destination then Hevo Data is the right choice for you! 

Visit our Website to Explore Hevo

Hevo Data, a No-code Data Pipeline provides you with a consistent and reliable solution to manage data transfer between a variety of sources and a wide variety of Desired Destinations, with a few clicks. Hevo Data with its strong integration with 100+ sources (including 40+ free sources) allows you to not only export data from your desired data sources & load it to the destination of your choice, but also transform & enrich your data to make it analysis-ready so that you can focus on your key business needs and perform insightful analysis using BI tools.

Want to take Hevo for a spin? Sign Up for a 14-day free trial and experience the feature-rich Hevo suite first hand. You can also have a look at the unbeatable pricing that will help you choose the right plan for your business needs.

Share your experience of learning about the Rule Based Data Mining Classifier! Let us know in the comments section below!