<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Lets load in the dataset
Step1:  Lets see the types that were imported on our behalf
Step2:  See the shape of the dataset
Step3:  Here we can see the that it has 891 rows of data and 12 attributes worth of imformation.
Step4:  Observations
Step5:  Since we know that Parch is the number of parents or children onboard, and SibSp is the number of siblings or spouses, we could collect those together as a FamilySize
<ASSISTANT_TASK:>
Python Code:
df = pd.read_csv("data/train.csv")
df.head()
df.tail()
Explanation: Lets load in the dataset
End of explanation
df.dtypes
df.info()
df.describe()
Explanation: Lets see the types that were imported on our behalf
End of explanation
df.shape
Explanation: See the shape of the dataset
End of explanation
len(df)
len(df.columns)
# where df.columns is 
df.columns
df["Name"]
my_famous_passenger = df[df["Name"] == "Guggenheim, Mr. Benjamin"]
print(my_famous_passenger)
###Lets get some information about a column
### Like mean age on the boat
df["Age"].mean()
### Fare
df["Fare"].mean()
df["Fare"].describe()
my_rich_passenger = df[df["Fare"] == 512.3292]
print(my_rich_passenger)
####Lets rearrange some columns. This would be very hard to do using a csv library and doing this by hand.
####Panda allows us to do this very intuitively
cols = list(df.columns.values)
print(cols)
#Using that list above, we can create a new list, with the values rearranged.
cols = ['Survived', 'Pclass', 'Name', 'Sex', 'Age', 'SibSp', 'Parch', 'Ticket', 'Fare', 'Cabin', 'Embarked', 'PassengerId']
new_df = df[cols]
new_df.head()
#### We can create new dataframes from a few attributes
new_df = df[["Sex","Age"]]
new_df.head()
df_of_women = df[df["Sex"] == "female"] #### Elaborate and find out what it means to df a df? Look up panda series
df_of_men = df[df["Sex"] == "male"]
df_of_women.head()
df_of_women.head()
# Excercise 
# Create three data frames. Capture them by passenger class 'PClass'. 
# There are three of them. Then figure out the size of each one.
df_pclass_1 = df[df["Pclass"] == 1]
df_pclass_1.head()
df_pclass_1.shape
df_pclass_2 = df[df["Pclass"] == 2]
df_pclass_2.head()
df_pclass_2.shape
df_pclass_3 = df[df["Pclass"] == 3]
df_pclass_3.head()
df_pclass_3.shape
Explanation: Here we can see the that it has 891 rows of data and 12 attributes worth of imformation.
End of explanation
df['FamilySize'] = df['SibSp'] + df['Parch']
df.head()
Explanation: Observations: There were many people in third class. More so than the other two class of passengers combined.
We can create new attributes from other attributes!
End of explanation
df["Age"].hist()
df["Age"].dropna().hist(bins=16, range=(0,80))
df["Fare"].hist()
plt.scatter(df['Fare'], df['Survived'])
plt.show()
### Side step
# Lets create a scatter plot
d = {'one' : np.random.rand(10),
     'two' : np.random.rand(10)}
print(d)
df_scrap = pd.DataFrame(d)
df_scrap.plot(style=['ro','bx'])
##### Back to the titanic. So we have our original dataset
df.head()
#### Lets group them by gender
grouped_by_sex = df.groupby(["Sex"])
grouped_by_sex.describe()
#### Lets group them by gender
grouped_by_sex_and_pclass = df.groupby(["Sex", "Pclass"])
grouped_by_sex_and_pclass.describe()
#### Lets group them by gender
grouped_by_sex_and_pclass_survived = df.groupby(["Sex", "Pclass", "Survived"])
grouped_by_sex_and_pclass_survived.describe()
df.groupby(['Sex', 'Pclass',"Survived"]).count() ### Count of records in each group throughout a dataset
# Logistic Regression Time!
import statsmodels.api as sm
import pylab as pl
print(df.columns)
# Create a new temp data frame
new_df = df
def gender_to_numeric(x):
    if x == "male":
        return 0
    else:
        return 1
new_df['Sex'] = new_df['Sex'].apply(gender_to_numeric)
new_df = new_df[["Survived", "Age","Sex", "Pclass"]]
new_df = new_df.dropna()
train_cols = new_df.columns[1:]
train_cols
logit = sm.Logit(new_df['Survived'], new_df[train_cols])
#Fit the model
result = logit.fit()
print(result.summary())
print(result.conf_int())
Explanation: Since we know that Parch is the number of parents or children onboard, and SibSp is the number of siblings or spouses, we could collect those together as a FamilySize
End of explanation
<END_TASK>