<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 In this exercise, you'll apply what you learned in the Handling missing values tutorial.
Setup
The questions below will give you feedback on your work. Run the following cell to set up the feedback system.
Step1:  1) Take a first look at the data
Run the next code cell to load in the libraries and dataset you'll use to complete the exercise.
Step2:  Use the code cell below to print the first five rows of the sf_permits DataFrame.
Step3:  Does the dataset have any missing values?  Once you have an answer, run the code cell below to get credit for your work.
Step4:  2) How many missing data points do we have?
What percentage of the values in the dataset are missing?  Your answer should be a number between 0 and 100.  (If 1/4 of the values in the dataset are missing, the answer is 25.)
Step5:  3) Figure out why the data is missing
Look at the columns "Street Number Suffix" and "Zipcode" from the San Francisco Building Permits dataset. Both of these contain missing values. 
- Which, if either, are missing because they don't exist? 
- Which, if either, are missing because they weren't recorded?  
Once you have an answer, run the code cell below.
Step6:  4) Drop missing values
Step7:  Once you have an answer, run the code cell below.
Step8:  5) Drop missing values
Step9:  6) Fill in missing values automatically
Try replacing all the NaN's in the sf_permits data with the one that comes directly after it and then replacing any remaining NaN's with 0.  Set the result to a new DataFrame sf_permits_with_na_imputed.
<ASSISTANT_TASK:>
Python Code:
from learntools.core import binder
binder.bind(globals())
from learntools.data_cleaning.ex1 import *
print("Setup Complete")
Explanation: In this exercise, you'll apply what you learned in the Handling missing values tutorial.
Setup
The questions below will give you feedback on your work. Run the following cell to set up the feedback system.
End of explanation
# modules we'll use
import pandas as pd
import numpy as np
# read in all our data
sf_permits = pd.read_csv("../input/building-permit-applications-data/Building_Permits.csv")
# set seed for reproducibility
np.random.seed(0) 
Explanation: 1) Take a first look at the data
Run the next code cell to load in the libraries and dataset you'll use to complete the exercise.
End of explanation
# TODO: Your code here!
#%%RM_IF(PROD)%%
sf_permits.head()
Explanation: Use the code cell below to print the first five rows of the sf_permits DataFrame.
End of explanation
# Check your answer (Run this code cell to receive credit!)
q1.check()
# Line below will give you a hint
#_COMMENT_IF(PROD)_
q1.hint()
Explanation: Does the dataset have any missing values?  Once you have an answer, run the code cell below to get credit for your work.
End of explanation
# TODO: Your code here!
percent_missing = ____
# Check your answer
q2.check()
# Lines below will give you a hint or solution code
#_COMMENT_IF(PROD)_
q2.hint()
#_COMMENT_IF(PROD)_
q2.solution()
#%%RM_IF(PROD)%%
# get the number of missing data points per column
percent_missing = sf_permits.isnull().sum().sum()
q2.assert_check_failed()
#%%RM_IF(PROD)%%
# get the number of missing data points per column
missing_values_count = sf_permits.isnull().sum()
# how many total missing values do we have?
total_cells = np.product(sf_permits.shape)
total_missing = missing_values_count.sum()
# percent of data that is missing
percent_missing = (total_missing/total_cells) * 100
q2.assert_check_passed()
Explanation: 2) How many missing data points do we have?
What percentage of the values in the dataset are missing?  Your answer should be a number between 0 and 100.  (If 1/4 of the values in the dataset are missing, the answer is 25.)
End of explanation
# Check your answer (Run this code cell to receive credit!)
q3.check()
# Line below will give you a hint
#_COMMENT_IF(PROD)_
q3.hint()
Explanation: 3) Figure out why the data is missing
Look at the columns "Street Number Suffix" and "Zipcode" from the San Francisco Building Permits dataset. Both of these contain missing values. 
- Which, if either, are missing because they don't exist? 
- Which, if either, are missing because they weren't recorded?  
Once you have an answer, run the code cell below.
End of explanation
# TODO: Your code here!
#%%RM_IF(PROD)%%
sf_permits.dropna()
Explanation: 4) Drop missing values: rows
If you removed all of the rows of sf_permits with missing values, how many rows are left?
Note: Do not change the value of sf_permits when checking this.
End of explanation
# Check your answer (Run this code cell to receive credit!)
q4.check()
# Line below will give you a hint
#_COMMENT_IF(PROD)_
q4.hint()
Explanation: Once you have an answer, run the code cell below.
End of explanation
# TODO: Your code here
sf_permits_with_na_dropped = ____
dropped_columns = ____
# Check your answer
q5.check()
#%%RM_IF(PROD)%%
# remove all columns with at least one missing value
sf_permits_with_na_dropped = sf_permits.dropna(axis=1)
# calculate number of dropped columns
cols_in_original_dataset = sf_permits.shape[1]
cols_in_na_dropped = sf_permits_with_na_dropped.shape[1]
dropped_columns = cols_in_original_dataset - cols_in_na_dropped 
q5.assert_check_passed()
# Lines below will give you a hint or solution code
#_COMMENT_IF(PROD)_
q5.hint()
#_COMMENT_IF(PROD)_
q5.solution()
Explanation: 5) Drop missing values: columns
Now try removing all the columns with empty values.
- Create a new DataFrame called sf_permits_with_na_dropped that has all of the columns with empty values removed.
- How many columns were removed from the original sf_permits DataFrame? Use this number to set the value of the dropped_columns variable below.
End of explanation
# TODO: Your code here
sf_permits_with_na_imputed = ____
# Check your answer
q6.check()
#%%RM_IF(PROD)%%
sf_permits_with_na_imputed = sf_permits_with_na_dropped.fillna(method='bfill', axis=0).fillna(0)
q6.assert_check_failed()
#%%RM_IF(PROD)%%
sf_permits_with_na_imputed = sf_permits.fillna(method='bfill', axis=0).fillna(0)
q6.assert_check_passed()
# Lines below will give you a hint or solution code
#_COMMENT_IF(PROD)_
q6.hint()
#_COMMENT_IF(PROD)_
q6.solution()
Explanation: 6) Fill in missing values automatically
Try replacing all the NaN's in the sf_permits data with the one that comes directly after it and then replacing any remaining NaN's with 0.  Set the result to a new DataFrame sf_permits_with_na_imputed.
End of explanation
<END_TASK>