<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Mutual Information
In the feature selection context, Mutual Information between a label Y and a predictor X is the amount of entropy shared between the true distributions of P(X) and P(Y).
<img src="https
Step1:  Attribute Information
Step2:  Calculating Mututal Information Score Directly
Step3:  Mutual Information is defined by
$I(Y, X) = H(Y) - H(Y \mid X)$
First step is to calculate $H(Y) = - P(Y) \space log \space P(Y)$
Step4:  Next step is to calculate $H(Y \mid X)$.
$H(Y \mid X) = \sum_{x \in X} p(x) H(Y \mid X=x) = - \sum {x \in X} p(x) \sum{y \in Y} p(Y=y \mid X=x)\space log \space p(Y=y \mid X=x)$
<ASSISTANT_TASK:>
Python Code:
import pandas as pd
import numpy as np
lenses_data = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/lenses/lenses.data', sep='\s+', header=None
                         )
Explanation: Mutual Information
In the feature selection context, Mutual Information between a label Y and a predictor X is the amount of entropy shared between the true distributions of P(X) and P(Y).
<img src="https://upload.wikimedia.org/wikipedia/commons/thumb/d/d4/Entropy-mutual-information-relative-entropy-relation-diagram.svg/744px-Entropy-mutual-information-relative-entropy-relation-diagram.svg.png" width="400", height="400"/>
End of explanation
lenses_data.columns= ['index', 'age', 'spec_type', 'astigmatic', 'tear_prod_rate', 'lens_type']
lenses_data = lenses_data.set_index('index')
lenses_data.head()
lens_type_names = {1: 'hard', 2: 'soft', 3: 'no_lense'}
lenses_data = lenses_data.assign(lens_type=lenses_data.lens_type.map(lambda n: lens_type_names[n]))
type_names = {1: 'no', 2: 'yes'}
lenses_data = lenses_data.assign(astigmatic=lenses_data.astigmatic.map(lambda n: type_names[n]))
lenses_data.head()
Explanation: Attribute Information:   
 1 : the patient should be fitted with hard contact lenses,
 2 : the patient should be fitted with soft contact lenses,
 3 : the patient should not be fitted with contact lenses.
1. age of the patient: (1) young, (2) pre-presbyopic, (3) presbyopic
2. spectacle prescription:  (1) myope, (2) hypermetrope
3. astigmatic:     (1) no, (2) yes
4. tear production rate:  (1) reduced, (2) normal
End of explanation
from sklearn.metrics import mutual_info_score
mutual_info_score(lenses_data['lens_type'], lenses_data['astigmatic'])
Explanation: Calculating Mututal Information Score Directly
End of explanation
y_counts = lenses_data['astigmatic'].value_counts()
y_counts
P_y = lenses_data['astigmatic'].value_counts(normalize=True)
P_y
H_y = - P_y.dot(np.log(P_y))
H_y
Explanation: Mutual Information is defined by
$I(Y, X) = H(Y) - H(Y \mid X)$
First step is to calculate $H(Y) = - P(Y) \space log \space P(Y)$
End of explanation
# cont_table = pd.crosstab(lenses_data['astigmatic'], lenses_data['lens_type'])
cont_table = pd.crosstab(lenses_data['astigmatic'], lenses_data['lens_type'])
cont_table
n_elems = cont_table.sum(axis=0)
n_elems
P_cond = cont_table / n_elems
P_cond
P_x = lenses_data['lens_type'].value_counts(normalize=True)
P_x
P_cond_aug = P_cond.where(P_cond != 0, other=1.0)
P_cond_aug
H_temp = P_cond.mul(np.log(P_cond_aug)).sum(axis=0)
H_temp
H_Y_given_X = - H_temp.dot(P_x)
H_Y_given_X
mutual_info = H_y - H_Y_given_X
mutual_info
Explanation: Next step is to calculate $H(Y \mid X)$.
$H(Y \mid X) = \sum_{x \in X} p(x) H(Y \mid X=x) = - \sum {x \in X} p(x) \sum{y \in Y} p(Y=y \mid X=x)\space log \space p(Y=y \mid X=x)$
End of explanation
<END_TASK>