<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Introduction
This IPython notebook illustrates how to perform matching using the rule-based matcher.
First, we need to import py_entitymatching package and other libraries as follows
Step1:  Then, read the (sample) input tables for matching purposes.
Step2:  Then, split the labeled data into development set and evaluation set. Use the development set to select the best learning-based matcher
Step3:  Creating and Using a Rule-Based Matcher
This, typically involves the following steps
Step4:  Creating Features
Next, we need to create a set of features for the development set. Magellan provides a way to automatically generate features based on the attributes in the input tables. For the purposes of this guide, we use the automatically generated features.
Step5:  We observe that there were 20 features generated. As a first step, lets say that we decide to use only 'year' related features.
Step6:  Adding Rules
Before we can use the rule-based matcher, we need to create rules to evaluate tuple pairs. Each rule is a list of strings. Each string specifies a conjunction of predicates. Each predicate has three parts
Step7:  Using the Matcher to Predict Results
Now that our rule-based matcher has some rules, we can use it to predict whether a tuple pair is actually a match. Each rule is is a conjunction of predicates and will return True only if all the predicates return True. The matcher is then a disjunction of rules and if any one of the rules return True, then the tuple pair will be a match.
<ASSISTANT_TASK:>
Python Code:
# Import py_entitymatching package
import py_entitymatching as em
import os
import pandas as pd
Explanation: Introduction
This IPython notebook illustrates how to perform matching using the rule-based matcher.
First, we need to import py_entitymatching package and other libraries as follows:
End of explanation
# Get the datasets directory
datasets_dir = em.get_install_path() + os.sep + 'datasets'
path_A = datasets_dir + os.sep + 'dblp_demo.csv'
path_B = datasets_dir + os.sep + 'acm_demo.csv'
path_labeled_data = datasets_dir + os.sep + 'labeled_data_demo.csv'
A = em.read_csv_metadata(path_A, key='id')
B = em.read_csv_metadata(path_B, key='id')
# Load the pre-labeled data
S = em.read_csv_metadata(path_labeled_data, 
                         key='_id',
                         ltable=A, rtable=B, 
                         fk_ltable='ltable_id', fk_rtable='rtable_id')
S.head()
Explanation: Then, read the (sample) input tables for matching purposes.
End of explanation
# Split S into I an J
IJ = em.split_train_test(S, train_proportion=0.5, random_state=0)
I = IJ['train']
J = IJ['test']
Explanation: Then, split the labeled data into development set and evaluation set. Use the development set to select the best learning-based matcher
End of explanation
brm = em.BooleanRuleMatcher()
Explanation: Creating and Using a Rule-Based Matcher
This, typically involves the following steps:
1. Creating the rule-based matcher
2. Creating features
3. Adding Rules
4. Using the Matcher to Predict Results
Creating the Rule-Based Matcher
End of explanation
# Generate a set of features
F = em.get_features_for_matching(A, B, validate_inferred_attr_types=False)
Explanation: Creating Features
Next, we need to create a set of features for the development set. Magellan provides a way to automatically generate features based on the attributes in the input tables. For the purposes of this guide, we use the automatically generated features.
End of explanation
F.feature_name
Explanation: We observe that there were 20 features generated. As a first step, lets say that we decide to use only 'year' related features.
End of explanation
# Add two rules to the rule-based matcher
# The first rule has two predicates, one comparing the titles and the other looking for an exact match of the years
brm.add_rule(['title_title_lev_sim(ltuple, rtuple) > 0.4', 'year_year_exm(ltuple, rtuple) == 1'], F)
# This second rule compares the authors
brm.add_rule(['authors_authors_lev_sim(ltuple, rtuple) > 0.4'], F)
brm.get_rule_names()
# Rules can also be deleted from the rule-based matcher
brm.delete_rule('_rule_1')
Explanation: Adding Rules
Before we can use the rule-based matcher, we need to create rules to evaluate tuple pairs. Each rule is a list of strings. Each string specifies a conjunction of predicates. Each predicate has three parts: (1) an expression, (2) a comparison operator, and (3) a value. The expression is evaluated over a tuple pair, producing a numeric value.
End of explanation
brm.predict(S, target_attr='pred_label', append=True)
S
Explanation: Using the Matcher to Predict Results
Now that our rule-based matcher has some rules, we can use it to predict whether a tuple pair is actually a match. Each rule is is a conjunction of predicates and will return True only if all the predicates return True. The matcher is then a disjunction of rules and if any one of the rules return True, then the tuple pair will be a match.
End of explanation
<END_TASK>