<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Example
Step1:  Get the Data
Step2:  Add a Computed Column
Some of the columns in the data are problematic for modeling
Step3:  Group By Analysis
Step4:  Sample the Data
Step5:  Check that Sampling is Even
As long as each partition has approximately .38 for the mean, then survivor rows are distributed evenly in the partitions.
Step6:  Train a Model
The casOut parameter that is shown in the example is used to store the model as an in-memory table. The next step of this example is to show how to score data with the model.
Step7:  Use the Model for Scoring
In this example, both the training data and the validation data are scored. This is done so that we can assess the effectiveness of the model for predicting whether someone survives on the Titanic.
The in-memory table, forestModel, is used as the model. The scoring output is stored in an in-memory table that is named forestScored.
Step8:  Assess the Model
The assess action is part of the percentile action set. You can run the loadActionSet action to ensure that the action is available to your session.
Step9:  Plot ROC
Step10:  Plot Lift
<ASSISTANT_TASK:>
Python Code:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from IPython.core.display import display, HTML
%matplotlib inline
import swat
s = swat.CAS('cloud.example.com', 5570)
Explanation: Example: Model Titanic Survival
This example is provided to demonstrate some of the typical programming activities for working with Python in a SAS Viya environment to run actions in SAS Cloud Analytic Services. The actions that are used in the example require SAS Visual Data Mining and Machine Learning.
For more information, see http://support.sas.com/documentation/prod-p/vdmml/index.html.
End of explanation
ulresult = s.upload('http://biostat.mc.vanderbilt.edu/wiki/pub/Main/DataSets/titanic3.csv')
titanic3 = ulresult.casTable
type(titanic3)
titanic3.table.columnInfo()
Explanation: Get the Data
End of explanation
# Create a computed variable.
titanic3.computedVars = ['deck']                         # 1
titanic3.computedVarsProgram = \
   "if cabin ne '' then deck = ksubstr(cabin,1,1); else deck = '';"
numeric=['pclass', 'survived', 'age', 'sibsp', 'parch', 'fare']
# Remove boat and body because they are proxies for survived.
# Remove ticket and cabin. Use the computed column, deck, instead.
char = ['sex', 'deck', 'embarked', 'home.dest']
all = numeric + char
Explanation: Add a Computed Column
Some of the columns in the data are problematic for modeling:
* The name column should not have a bearing on the analysis.
* The boat and body columns are proxies for the response variable, survived.
* The cabin column is similar to name in that it is too specific to be generalized. A computed column, deck, is created because it is slightly more general than cabin.
End of explanation
# The numeric variable was defined earlier.
results = titanic3[numeric].groupby("survived").simple.summary()
resultColumns = ['Column', 'Min', 'Max', 'N', 'NMiss', 'Mean', 'Sum', 'Std', 'StdErr'];
display(HTML('<h3>Perished</h3>'))
display(results['ByGroup1.Summary'][resultColumns])       # 1
display(HTML('<h3>Survived</h3>'))
display(results['ByGroup2.Summary'][resultColumns])
Explanation: Group By Analysis: Descriptive Statistics
The simple.summary action is used to provide some descriptive statistics. The groupBy parameter is set on the Titanic3 object so that the statistics are shown for survivors and those that did not survive.
End of explanation
s.builtins.loadActionSet('sampling')
# The sampling.stratified action does not accept the vars parameter.
# Instead, copyVars is used to select the columns to copy to the output table.
if 'vars' in titanic3.params:
    del titanic3.vars
# Temporarily set a groupBy parameter.
with titanic3:
    titanic3.groupBy={'survived'}
    titanic3.sampling.stratified(
       partInd=True,                                     # 1
       samppct=40,                                       # 2
       seed=1234,
       output={
          'casout':{'name':'titanic3part', 'replace':True},
          'copyVars':all
       }
    )
titanic3.table.dropTable()                               # 3
titanic3part = s.CASTable('titanic3part')                # 4
ci = titanic3part.columnInfo()
display(ci)
Explanation: Sample the Data
End of explanation
survSummary = titanic3part['survived'].groupby('_partind_').simple.summary()
 
resultColumns = ['Column', 'N', 'NMiss', 'Mean', 'Sum', 'Std', 'StdErr']
display(survSummary['ByGroupInfo'])
display(survSummary['ByGroup1.Summary'][resultColumns])
display(survSummary['ByGroup2.Summary'][resultColumns])
Explanation: Check that Sampling is Even
As long as each partition has approximately .38 for the mean, then survivor rows are distributed evenly in the partitions.
End of explanation
s.builtins.loadActionSet('decisionTree')                 # 1
training = titanic3part.query('0 = _partind_')           # 2
trainingResults = training.forestTrain(
        target='survived',
        inputs=all,
        nominals=char + ['pclass', 'survived'],
        casOut={'name':'forestModel', 'replace':True},
        seed=1234,
        binOrder=True,
        varImp=True
    )
display(trainingResults)
Explanation: Train a Model
The casOut parameter that is shown in the example is used to store the model as an in-memory table. The next step of this example is to show how to score data with the model.
End of explanation
forestModel = s.CASTable('forestModel')
titanic3part.forestScore(
    modelTable=forestModel, 
    copyVars=['survived', '_partind_'],
    casOut={'name':'forestScored', 'replace':True}, 
    vote='prob'
)
Explanation: Use the Model for Scoring
In this example, both the training data and the validation data are scored. This is done so that we can assess the effectiveness of the model for predicting whether someone survives on the Titanic.
The in-memory table, forestModel, is used as the model. The scoring output is stored in an in-memory table that is named forestScored.
End of explanation
s.builtins.loadActionSet('percentile')
forestScored = s.CASTable('forestScored')                # 1
forestScored.groupBy='_PartInd_'                         # 2
forestScored.computedVars=['P1', 'P0']                   # 3
forestScored.computedVarsProgram='''                     
   if '1' eq strip(_RF_PredName_) then do;
     P1 = _RF_PredP_;
     P0 = 1 - _RF_PredP_;
   end;
   else do;
     P1 = 1 - _RF_PredP_;
     P0 = _RF_PredP_;
   end;
'''                                                      # 4
forestScored.percentile.assess(
    casOut={'name':'forestAssess', 'replace':True},
    nbins=10,
    cutStep = 0.01,
    inputs=['P1'],
    response='survived', 
    event='1', 
    pVar=['P0'], 
    pEvent='0'
)
Explanation: Assess the Model
The assess action is part of the percentile action set. You can run the loadActionSet action to ensure that the action is available to your session.
End of explanation
forestAssess_ROC = \
    s.CASTable('forestAssess_ROC', where='1 = _partind_') # 1
out2 = forestAssess_ROC.to_frame()
plt.figure(figsize=(8,8))
plt.plot(out2._FPR_,out2._Sensitivity_,'bo-',linewidth=2)
plt.plot(pd.Series(range(0,11,1))/10,pd.Series(range(0,11,1))/10,'k--',linewidth=1)
plt.xlabel('False Positive Rate')
plt.ylabel('Correct Classification Rate')
plt.grid(True)
plt.title('ROC Curve')
plt.show()
Explanation: Plot ROC
End of explanation
forestAssess = \
     s.CASTable('forestAssess', where='1 = _partind_')  # 1
lift = forestAssess.to_frame()
plt.figure(figsize=(8,8))
plt.plot(lift._Depth_, lift._Lift_,'bo-',linewidth=2)
plt.xlabel('Percentile')
plt.ylabel('Lift')
plt.grid(True)
plt.title('Lift Chart')
plt.show()
s.close()
Explanation: Plot Lift
End of explanation
<END_TASK>