<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Homework 8
Step1:  Now let's look at the columns in the database.
Step2:  So there are the following things in the dataset. 
* Name of the Slums
The Municipal Circle and Ward Number
The number of Households and Population slumwise.
The number of Households that are Below Poverty Line (BPL) - It means there are people in slums who are not poor as defined by the government's definition. This becomes crucial because - it is often assumed that the slum is a proxy for urban poverty. Gautam Bhan and Arindam Jana's paper argues otherwise (http
Step3:  A quarter of the city's population approximately lives in slums.
2. What is the average number of households in a slum? What is the average family size?
Step4:  3. Which circles have the maximum number of slums?
Circle here refers to a Municipal Subdivision. The city is divided into 18 circles, with each circle being further subdivided into wards. The city has 150 of these wards.
Step5:  Circle 4 - which is the old city quarter of Hyderabad has the maximum number of slums in terms of numbers. 
4. Which wards have the maximum number of slums?
Step6:  Because the data is faulty, there are slums whose ward numbers is empty, ie 0.The remainder of the list are the ward numbers with maximum slums and on the right are the number of slums in them. That makes ward 108 have the maximum number of slums. On introspecting on a map(in CartoDB) - one is curious to know why is it that this ward has a highly fragmented number of slums, often not more than a couple of houses. 
5. What is the total literacy level in the slums?
We count the total number of literates in each slum and compare it with the total population.
Step7:  6. What percentage of households in the slum are BPL(Below Poverty Level)?
As argued above, there are valid concerns that the slum might not be a great proxy for urban powerty. But if the percentage of households that are Below Poverty Line is high enough, then the slum can be considered as a proxy for urban poverty.
Step8:  That is high enough for us to consider the slum as a decent proxy for urban poverty in Hyderabad. 
The remaining data in its absolute number forms make it impossible for us to compare slums among each other. It would be useful to convert the caste, tenure, age, structures into percentages, making it easy to compare.
Now, we create new columns in the dataframe using the existing columns and check.
Step9:  On closer introspection, it is found that the other datasets are per household and not population wise like tha literacy percentage. So we do the addition of the following columns into the dataframe. So we repeat the above calculation for percentage of households below poverty line.
Step10:  Now we repeat this for the Caste Parameters.
Step11:  And for the type of structures..
Step12:  .. And for the kind of tenure
Step13:  and finally for the number of years the residents have stayed..
Step14:  Now, let us see how the slums dataframe looks like
Step15:  We now have the data with all the calculations necessary made for analysis and now we begin. 
CASTE PARAMETERS
7. Which slums have the highest percentage of households that belong to the General Caste?
Step16:  8. Which slums have the highest percentage of households that belong to the SC Caste?
Step17:  9. Which slums have the highest percentage of households that belong to the ST Caste?
Step18:  10. Which slums have the highest percentage of households that belong to the OBC Caste?
Step19:  Now, we see what percentage of each of these castes live in slums.
Step20:  STRUCTURES OF HOUSES
Now let us see the number of households that are proper pucca structures
Step21:  That's roughly about half of them. What about the remaining half?
12. What percentage of Households are Semi-pucca Structures?
Step22:  That still leaves out a little bit more... and the remaining structures are made out of..
13. What percentage of Households are Neither Semipucca or Pucca Structures?
Step23:  Kaccha houses. 
LAND TENURE DETAILS
14. What percentage of Households have access to Pattas?
Step24:  15. What percentage of Households have access to Possession Certificates?
Step25:  16. What percentage of Households are on Private Land?
Step26:  17. What percentage of Households are on Public Land?
Step27:  18. What percentage of Households are Renters?
Step28:  19. What percentage of Households don't fit in any of the other categories?
Step29:  WHEN HAVE PEOPLE MOVED IN
20. What percentage of Households have moved in the last one year?
Step30:  21. What percentage of Households have moved in the last one to three years?
Step31:  22. What percentage of Households have moved in the last three to five years?
Step32:  23. What percentage of Households have moved more than five years ago?
Step33:  INCOME, EXPENDITURE, DEBT
24. What is the average monthly income of a slum household? (in rupees, 70 rupees make a dollar)
Step34:  25. What is the average monthly expenditure of a slum household? (in rupees, 70 rupees make a dollar)
Step35:  26. What is the average debt per slum?(in rupees, 70 rupees make a dollar)
<ASSISTANT_TASK:>
Python Code:
#Starting out the basics. 
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
slums= pd.read_csv("hyderabad_slum_master.csv")
slums.head() #The dataset is a spatialised one, hence the_geom column. 
Explanation: Homework 8: Dataset: Slums of Hyderabad
The following dataset is a heavily cleaned up version of the official data on slums from the Government's Rajiv Awas Yojana Scheme's Slum Free City Action Plan of 2011. This dataset is the only geocoded data for slums in India. While the quality of data could be questionable, it still provides an insightful understanding of the spatial patterns of poverty in Indian cities.
End of explanation
slums.columns
Explanation: Now let's look at the columns in the database.
End of explanation
totalpopulation=slums['population'].sum()
print("The total number of people who live in slums in Hyderabad are",int(totalpopulation))
print("")
populationofhyderabad=6731790 #According to the 2011 census. 
percentageofpopulation=(totalpopulation/populationofhyderabad)*100
print("The percentage of people who live in slums is",(percentageofpopulation))
Explanation: So there are the following things in the dataset. 
* Name of the Slums
The Municipal Circle and Ward Number
The number of Households and Population slumwise.
The number of Households that are Below Poverty Line (BPL) - It means there are people in slums who are not poor as defined by the government's definition. This becomes crucial because - it is often assumed that the slum is a proxy for urban poverty. Gautam Bhan and Arindam Jana's paper argues otherwise (http://epw.yodasoft.com/journal/2015/22/review-urban-affairs-review-issues/reading-spatial-inequality-urban-india.html)
Caste Parameters : In Indian cities, Caste is an important index. The households that are "General", "Scheduled Caste", "Scheduled Tribe" or "Other Backward Castes" are all counted and have their own columns.
Tenure Parameters: Because the dataset is primarily about eliminating slums with the idea of providing Government built houses, the land tenure status of the slum households is essential to know. They are Patta, Possession Certificate, Slums on Private Land, Slums on Public Land, Households that are renters and Other tenures that do not fit under this framework. 
Structure of the House : This pertains to what kind of a house the slum dweller lives on. Pucca means a stable house made out of concrete, Semipucca could mean a not so stable house and Kuccha house means a house made out of mud/wood/etc. 
Average Monthly Income, Expenditure and Debt.
The number of years have the dwellers stayed in the slums 
Number of Female Headed Households. 
1. What are the total number of slums in Hyderabad? What is the number of people who live in slums? How much percentage of the city's population lives in slums?
End of explanation
print("The total number of slum households is",slums['households'].sum())
print("")
print("The average number of households in a slum is",slums['households'].mean())
print("")
print("The average family size is", (slums['population'].sum()/slums['households'].sum()))
Explanation: A quarter of the city's population approximately lives in slums.
2. What is the average number of households in a slum? What is the average family size?
End of explanation
circle_count=slums['circle_number'].value_counts()
circle_count
plt.style.use('ggplot')
circle_count.plot(kind='bar',x='Circle Number',y='Number of Slums',legend=False, figsize=(10,10))
Explanation: 3. Which circles have the maximum number of slums?
Circle here refers to a Municipal Subdivision. The city is divided into 18 circles, with each circle being further subdivided into wards. The city has 150 of these wards.
End of explanation
slums['ward_number'].value_counts().head(20)
Explanation: Circle 4 - which is the old city quarter of Hyderabad has the maximum number of slums in terms of numbers. 
4. Which wards have the maximum number of slums?
End of explanation
print("The total percentage of literacy in slums is",(slums['literacy_literates'].sum())/(slums['population'].sum())*100)
Explanation: Because the data is faulty, there are slums whose ward numbers is empty, ie 0.The remainder of the list are the ward numbers with maximum slums and on the right are the number of slums in them. That makes ward 108 have the maximum number of slums. On introspecting on a map(in CartoDB) - one is curious to know why is it that this ward has a highly fragmented number of slums, often not more than a couple of houses. 
5. What is the total literacy level in the slums?
We count the total number of literates in each slum and compare it with the total population.
End of explanation
(slums['number_of_bpl_households'].sum())/(slums['households'].sum())*100 
Explanation: 6. What percentage of households in the slum are BPL(Below Poverty Level)?
As argued above, there are valid concerns that the slum might not be a great proxy for urban powerty. But if the percentage of households that are Below Poverty Line is high enough, then the slum can be considered as a proxy for urban poverty.
End of explanation
slums['percentageofliterates']=slums['literacy_literates']/slums['population']*100
slums['percentageofliterates'].mean()
Explanation: That is high enough for us to consider the slum as a decent proxy for urban poverty in Hyderabad. 
The remaining data in its absolute number forms make it impossible for us to compare slums among each other. It would be useful to convert the caste, tenure, age, structures into percentages, making it easy to compare.
Now, we create new columns in the dataframe using the existing columns and check.
End of explanation
slums['percentageofbplhouseholds']=slums['number_of_bpl_households']/slums['households']*100
Explanation: On closer introspection, it is found that the other datasets are per household and not population wise like tha literacy percentage. So we do the addition of the following columns into the dataframe. So we repeat the above calculation for percentage of households below poverty line.
End of explanation
slums['percentageofgeneral'] = slums['caste_general']/slums['households']*100
slums['percentageofminority']= slums['minority']/slums['households']*100
slums['percentageofobc'] = slums['caste_obc']/slums['households']*100
slums['percentageofsc'] = slums['caste_sc']/slums['households']*100
slums['percentageofst'] = slums['caste_st']/slums['households']*100
Explanation: Now we repeat this for the Caste Parameters.
End of explanation
slums['percentageofpuccastructures'] = slums['structure_pucca']/slums['households']*100
slums['percentageofsemipuccastructures'] = slums['structure_semipucca']/slums['households']*100
slums['percentageofkucchastructures'] = slums['structure_kuccha']/slums['households']*100
Explanation: And for the type of structures..
End of explanation
slums['percentageoftenure_patta'] = slums['tenure_patta']/slums['households']*100
slums['percentageoftenure_pc'] = slums['tenure_possession_certificate']/slums['households']*100
slums['percentageoftenure_private'] = slums['tenure_private_land']/slums['households']*100
slums['percentageoftenure_public'] = slums['tenure_public']/slums['households']*100
slums['percentageoftenure_renters'] = slums['tenure_rented']/slums['households']*100
slums['percentageoftenure_other'] = slums['tenure_others']/slums['households']*100
Explanation: .. And for the kind of tenure
End of explanation
slums['percentageoftenure_0to1'] = slums['zerotoone_years_of_stay']/slums['households']*100
slums['percentageoftenure_1to3'] = slums['onetothree_years_of_stay']/slums['households']*100
slums['percentageoftenure_3to5'] = slums['threetofive_years_of_stay']/slums['households']*100
slums['percentageoftenure_morethan5'] = slums['morethanfive_years_of_stay']/slums['households']*100
Explanation: and finally for the number of years the residents have stayed..
End of explanation
slums.head()
slums.columns
Explanation: Now, let us see how the slums dataframe looks like
End of explanation
slums.sort_values(by='percentageofgeneral').head(5)
Explanation: We now have the data with all the calculations necessary made for analysis and now we begin. 
CASTE PARAMETERS
7. Which slums have the highest percentage of households that belong to the General Caste?
End of explanation
slums.sort_values(by='percentageofsc').head(5)
Explanation: 8. Which slums have the highest percentage of households that belong to the SC Caste?
End of explanation
slums.sort_values(by='percentageofst').head(5)
Explanation: 9. Which slums have the highest percentage of households that belong to the ST Caste?
End of explanation
slums.sort_values(by='percentageofobc').head(5)
Explanation: 10. Which slums have the highest percentage of households that belong to the OBC Caste?
End of explanation
a=slums['percentageofgeneral'].mean()
b=slums['percentageofsc'].mean()
c=slums['percentageofst'].mean()
d=slums['percentageofobc'].mean()
print(a,b,c,d)
plt.style.use('ggplot')
series=pd.Series([a,b,c,d], index=['General', 'SC', 'ST','OBC'], name='Caste Percentages')
series.plot.pie(figsize=(6, 6))
Explanation: Now, we see what percentage of each of these castes live in slums.
End of explanation
a=slums['percentageofpuccastructures'].mean()
Explanation: STRUCTURES OF HOUSES
Now let us see the number of households that are proper pucca structures:
11. What percentage of Households are Pucca Structures?
End of explanation
b=slums['percentageofsemipuccastructures'].mean()
Explanation: That's roughly about half of them. What about the remaining half?
12. What percentage of Households are Semi-pucca Structures?
End of explanation
c=slums['percentageofkucchastructures'].mean()
plt.style.use('ggplot')
series=pd.Series([a,b,c], index=['Pucca Houses', 'Semipucca Houses', 'Kuccha Houses'], name='Household Structure Type')
series.plot.pie(figsize=(6, 6))
Explanation: That still leaves out a little bit more... and the remaining structures are made out of..
13. What percentage of Households are Neither Semipucca or Pucca Structures?
End of explanation
a=slums['percentageoftenure_patta'].mean()
Explanation: Kaccha houses. 
LAND TENURE DETAILS
14. What percentage of Households have access to Pattas?
End of explanation
b=slums['percentageoftenure_pc'].mean()
Explanation: 15. What percentage of Households have access to Possession Certificates?
End of explanation
c=slums['percentageoftenure_private'].mean()
Explanation: 16. What percentage of Households are on Private Land?
End of explanation
d=slums['percentageoftenure_public'].mean()
Explanation: 17. What percentage of Households are on Public Land?
End of explanation
e=slums['percentageoftenure_renters'].mean()
Explanation: 18. What percentage of Households are Renters?
End of explanation
f=slums['percentageoftenure_other'].mean()
plt.style.use('ggplot')
series=pd.Series([a,b,c,d,e,f], index=['Pattas', 'PC', 'Private Land','Public Land','Renters','Others'], name='Tenuree Type')
series.plot.pie(figsize=(6, 6))
Explanation: 19. What percentage of Households don't fit in any of the other categories?
End of explanation
a=slums['percentageoftenure_0to1'].mean()
Explanation: WHEN HAVE PEOPLE MOVED IN
20. What percentage of Households have moved in the last one year?
End of explanation
b=slums['percentageoftenure_1to3'].mean()
Explanation: 21. What percentage of Households have moved in the last one to three years?
End of explanation
c=slums['percentageoftenure_3to5'].mean()
Explanation: 22. What percentage of Households have moved in the last three to five years?
End of explanation
d=slums['percentageoftenure_morethan5'].mean()
plt.style.use('ggplot')
series=pd.Series([a,b,c,d], index=['Zero to One Years', 'One to Three Years', 'Three to Five Years','More than Five Years'])
series.plot.pie(figsize=(6, 6))
Explanation: 23. What percentage of Households have moved more than five years ago?
End of explanation
slums['avg_monthly_income'].mean()
Explanation: INCOME, EXPENDITURE, DEBT
24. What is the average monthly income of a slum household? (in rupees, 70 rupees make a dollar)
End of explanation
slums['avg_monthly_expenditure'].mean()
Explanation: 25. What is the average monthly expenditure of a slum household? (in rupees, 70 rupees make a dollar)
End of explanation
slums['debts_outstanding'].mean()
Explanation: 26. What is the average debt per slum?(in rupees, 70 rupees make a dollar)
End of explanation
<END_TASK>