<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 <a href="https
Step1:  Classification with TensorFlow
By now you should be familiar with classification in scikit-learn. In this Colab we will explore another commonly used tool for classification and machine learning
Step2:  And then download the dataset.
Step3:  And load the data into a DataFrame and take a peek.
Step4:  We can see that all of the data is numeric, but varies a bit in scale.
Let's describe the data
Step5:  No missing data. Only 303 rows of data, though, so we aren't working with a huge dataset.
Now we'll dig deeper into the data in a few of the columns. If you were working with a dataset for a real-world model you would want to explore each column.
We'll start by mapping out the correlations in the data.
Step6:  There are no obviously strong correlations.
Let's now see how balanced our data is by sex
Step7:  In this data female maps to 0 and male maps to 1, so there are over twice as many men in the dataset.
Let's also check out the target.
Step8:  In this case the dataset looks more balanced.
And finally we'll look at age.
Step9:  The dataset seems to be pretty heavily skewed toward individuals in their 50s and 60s.
There isn't a lot of actionable information from our analysis. We might want to stratify our data by sex when we train and test our model, but there are no data repairs that seem to need to be done.
If you were building this model for a real world application, you would also want to ensure that the values in the numeric columns are realistic.
The Model
Let's build and train our model. We'll build a deep neural network that takes our input features and returns a 0 if it predicts that the patient doesn't have heart disease and a 1 if it predicts that the patient does have heart disease.
First let's create a list of features to make coding easier.
Step10:  We'll also want to normalize our feature data before feeding it into the model.
Step11:  We can also now split off a validation set from our data. Since we have so many more men than women in this dataset, we will stratify on sex.
Step12:  We'll use the TensorFlow Keras Sequential model. The input size needs to be equal to the number of input features that we have. The output size needs to be 1 since we are predicting a yes/no value. The number and width of layers in between are an area for experimentation, as are the activation functions.
We start with an initial hidden layer 64 nodes wide and funnel down to 32, 16, and finally, to the output layer of 1 node.
Step13:  We can now compile the model. We use 'binary_crossentropy' loss since this is a binary classification model.
Step14:  And finally, we can actually fit the model. We'll start with a run of 500 training epochs. Once we are done, we'll print out the final accuracy the model achieved.
Step15:  We got perfect accuracy in our model. Let's see how the accuracy improves and the loss is reduced over epochs.
Step16:  We seem to have kept training this model far too long. The accuracy reaches perfection, and the loss moves to 0.0 after a few hundred epochs.
Let's see if we overfit by using our validation holdout data. In order to do that, we need to convert our predictions back into a binary representation.
Step17:  As you can see, our predictions are continuous numbers, not the 1 or 0 values that we expected. These values are confidences that the value is 1. Let's look at them in a histogram.
Step18:  Here we can see that the model is highly confident yes or no in many cases, but there are some cases where the model was unsure.
How do we convert these confidences into a yes/no decision?
One way is to simply round
Step19:  This puts the cut-off threshold for a yes/no decision at 0.5. Let's think about the implications of this.
Also note that the choice of a sigmoid activation function was not coincidence. We wanted to use an activation function that would keep the output values between 0.0 and 1.0 for rounding purposes. 
Now let's check our accuracy.
Step20:  When we ran this model, our score was in the low 80s, which is not great. Yours is likely similar.
Exercise 2
Step21:  Exercise 2
<ASSISTANT_TASK:>
Python Code:
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
Explanation: <a href="https://colab.research.google.com/github/google/applied-machine-learning-intensive/blob/master/content/04_classification/03_classification_with_tensorflow/colab.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>
Copyright 2020 Google LLC.
End of explanation
! chmod 600 kaggle.json && (ls ~/.kaggle 2>/dev/null || mkdir ~/.kaggle) && mv kaggle.json ~/.kaggle/ && echo 'Done'
Explanation: Classification with TensorFlow
By now you should be familiar with classification in scikit-learn. In this Colab we will explore another commonly used tool for classification and machine learning: TensorFlow.
The Dataset
The dataset that we'll be using is the UCI Heart Disease dataset. The dataset contains health information about patients, as well as a "presence of heart disease" indicator.
The original dataset contains over 70 different attributes and five heart disease classifications. For this lab we'll use a simplified version of the dataset hosted on Kaggle.
This simplified version of the dataset contains 13 attributes and a yes/no indicator for the presence or absence of heart disease.
The columns are below:
Feature | Description
--------|--------------
age     | age in years
sex     | sex<br>0 = female<br>1 = male
cp      | chest pain type<br>1 = typical angina<br>2 = atypical angina<br>3 = non-anginal pain<br>4 = asymptomatic
trestbps  | resting blood pressure in Hg
chol      | serum cholesterol in mg/dl
fbs       | is fasting blood sugar > 120 mg/dl<br>0 = false<br>1 = true
restecg   | results of a resting electrocardiograph<br>0 = normal<br>1 = ST-T wave abnormality<br>2 = left ventricular hypertrophy
thalach   | max heart rate
exang     | exercise induced angina<br>0 = no<br>1 = yes
oldpeak   | measurement of an abnormal ST depression
slope     | slope of peak of exercise ST segment<br>1 = upslope<br>2 = flat<br>3 = downslope
ca        | count of major blood vessels colored by fluoroscopy<br>0, 1, 2, 3, or 4
thal      | presence heart condition<br>0 = unknown<br>1 = normal<br>2 = fixed defect<br>3 = reversible defect
The heart disease indicator is a 0 for no disease and a 1 for heart disease.
Let's assume we have been given this dataset by the Cleveland Clinic and have been asked to build a model that can predict if their patients have heart disease or not. The purpose of the model is to assist doctors in making diagnostic decisions faster.
Exercise 1: Ethical Considerations
Before we dive in, let's take a moment to think about the dataset and the larger problem that we are trying to solve. We have 17 data attributes related to an individual's health, as well as an indicator that determines if the patient has heart disease.
Question 1
Are there any attributes in the data that we should pay special attention to? Imagine a case where the data is unbalanced in some way. How might that affect the model and the doctor/patient experience?
Student Solution
Your answer goes here
Question 2
Assuming we can get a reasonably well-performing model deployed, is there potential for problems with how the predictions from this model are interpreted and used?
Student Solution
Your answer goes here
Exploratory Data Analysis
Let's download the data and take a look at what we are working with.
Upload your kaggle.json file and run the code below.
End of explanation
!kaggle datasets download ronitf/heart-disease-uci
!ls
Explanation: And then download the dataset.
End of explanation
import pandas as pd
df = pd.read_csv('heart-disease-uci.zip')
df.sample(5)
Explanation: And load the data into a DataFrame and take a peek.
End of explanation
df.describe()
Explanation: We can see that all of the data is numeric, but varies a bit in scale.
Let's describe the data:
End of explanation
import matplotlib.pyplot as plt
import seaborn as sns
plt.figure(figsize=(10,10))
_ = sns.heatmap(df.corr(), cmap='coolwarm', annot=True)
Explanation: No missing data. Only 303 rows of data, though, so we aren't working with a huge dataset.
Now we'll dig deeper into the data in a few of the columns. If you were working with a dataset for a real-world model you would want to explore each column.
We'll start by mapping out the correlations in the data.
End of explanation
df['sex'].hist()
Explanation: There are no obviously strong correlations.
Let's now see how balanced our data is by sex:
End of explanation
df['target'].hist()
Explanation: In this data female maps to 0 and male maps to 1, so there are over twice as many men in the dataset.
Let's also check out the target.
End of explanation
df['age'].hist()
Explanation: In this case the dataset looks more balanced.
And finally we'll look at age.
End of explanation
FEATURES = df.columns.values[:-1]
TARGET = df.columns.values[-1]
FEATURES, TARGET
Explanation: The dataset seems to be pretty heavily skewed toward individuals in their 50s and 60s.
There isn't a lot of actionable information from our analysis. We might want to stratify our data by sex when we train and test our model, but there are no data repairs that seem to need to be done.
If you were building this model for a real world application, you would also want to ensure that the values in the numeric columns are realistic.
The Model
Let's build and train our model. We'll build a deep neural network that takes our input features and returns a 0 if it predicts that the patient doesn't have heart disease and a 1 if it predicts that the patient does have heart disease.
First let's create a list of features to make coding easier.
End of explanation
df.loc[:, FEATURES] = ((df[FEATURES] - df[FEATURES].min()) / (df[FEATURES].max() - df[FEATURES].min()))
df.describe()
Explanation: We'll also want to normalize our feature data before feeding it into the model.
End of explanation
from sklearn.model_selection import train_test_split
X_train, X_validate, y_train, y_validate = train_test_split(
    df[FEATURES], df[TARGET], test_size=0.2, stratify=df['sex'])
X_train.shape, X_validate.shape
Explanation: We can also now split off a validation set from our data. Since we have so many more men than women in this dataset, we will stratify on sex.
End of explanation
import tensorflow as tf
model = tf.keras.Sequential([
    tf.keras.layers.Dense(64, activation=tf.nn.relu, 
                          input_shape=(FEATURES.size,)),
    tf.keras.layers.Dense(32, activation=tf.nn.relu),
    tf.keras.layers.Dense(16, activation=tf.nn.relu),
    tf.keras.layers.Dense(1, activation=tf.nn.sigmoid)
])
model.summary()
Explanation: We'll use the TensorFlow Keras Sequential model. The input size needs to be equal to the number of input features that we have. The output size needs to be 1 since we are predicting a yes/no value. The number and width of layers in between are an area for experimentation, as are the activation functions.
We start with an initial hidden layer 64 nodes wide and funnel down to 32, 16, and finally, to the output layer of 1 node.
End of explanation
model.compile(
    loss='binary_crossentropy',
    optimizer='Adam',
    metrics=['accuracy']
)
Explanation: We can now compile the model. We use 'binary_crossentropy' loss since this is a binary classification model.
End of explanation
history = model.fit(X_train, y_train, epochs=500, verbose=0)
history.history['accuracy'][-1]
Explanation: And finally, we can actually fit the model. We'll start with a run of 500 training epochs. Once we are done, we'll print out the final accuracy the model achieved.
End of explanation
import matplotlib.pyplot as plt
plt.figure(figsize=(16,5))
plt.subplot(1,2,1)
plt.plot(history.history['accuracy'])
plt.title('Training Accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train_accuracy'], loc='best')
plt.subplot(1,2,2)
plt.plot(history.history['loss'])
plt.title('Training Loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train_loss'], loc='best')
Explanation: We got perfect accuracy in our model. Let's see how the accuracy improves and the loss is reduced over epochs.
End of explanation
predictions = model.predict(X_validate)
predictions[:10]
Explanation: We seem to have kept training this model far too long. The accuracy reaches perfection, and the loss moves to 0.0 after a few hundred epochs.
Let's see if we overfit by using our validation holdout data. In order to do that, we need to convert our predictions back into a binary representation.
End of explanation
import matplotlib.pyplot as plt
_ = plt.hist(predictions)
Explanation: As you can see, our predictions are continuous numbers, not the 1 or 0 values that we expected. These values are confidences that the value is 1. Let's look at them in a histogram.
End of explanation
predictions = [round(x[0]) for x in predictions]
_ = plt.hist(predictions)
Explanation: Here we can see that the model is highly confident yes or no in many cases, but there are some cases where the model was unsure.
How do we convert these confidences into a yes/no decision?
One way is to simply round:
End of explanation
from sklearn.metrics import accuracy_score
accuracy_score(y_validate, predictions)
Explanation: This puts the cut-off threshold for a yes/no decision at 0.5. Let's think about the implications of this.
Also note that the choice of a sigmoid activation function was not coincidence. We wanted to use an activation function that would keep the output values between 0.0 and 1.0 for rounding purposes. 
Now let's check our accuracy.
End of explanation
# Your Code Goes Here
Explanation: When we ran this model, our score was in the low 80s, which is not great. Yours is likely similar.
Exercise 2: Adjusting the Threshold
Question 1
We decided to round for our classification, which puts the threshold for the decision at 0.5. This decision was made somewhat arbitrarily. Let's think about our problem space a bit more. We are making a model that predicts if an individual has heart disease. Would it be better if we set the threshold for predicting heart disease higher or lower than 0.5? Or is 0.5 okay? Explain your reasoning.
Student Solution
Your solution goes here
Question 2
Write code to make yes/no predictions using a higher or lower threshold based on the argument you made in the first question of this exercise. If you chose to keep the threshold at 0.5, then just pick higher or lower and write the code for that. Print out the accuracy for the new threshold.
Student Solution
End of explanation
# Your code goes here
Explanation: Exercise 2: Early Stopping
Five hundred epochs turned out to be a bit too many. Use the EarlyStopping class to stop when the loss doesn't improve over the course of five epochs. Print your accuracy score so you can see if it stayed reasonably close to your earlier model. Be sure to also make model fitting verbosity 1 or 2 so you can see at which epoch your model stopped.
Student Solution
End of explanation
<END_TASK>