<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 13 - Model Deployment
by Alejandro Correa Bahnsen
version 0.1, May 2016
Part of the class Machine Learning for Security Informatics
This notebook is licensed under a [Creative Commons Attribution-ShareAlike 3.0 Unported License]
Agenda
Step1:  Creating features
Step2:  Contain any of the following
Step3:  Lenght of the url
Lenght of domain
is IP?
Number of .com
Step4:  Create Model
Step5:  Save model
Step6:  Part 2
Step7:  Part 3
Step8:  Create api
Step9:  Load model and create function that predicts an URL
Step10:  Run API
<ASSISTANT_TASK:>
Python Code:
import pandas as pd
import zipfile
with zipfile.ZipFile('../datasets/phishing.csv.zip', 'r') as z:
    f = z.open('phishing.csv')
    data = pd.read_csv(f, index_col=False)
data.head()
data.phishing.value_counts()
Explanation: 13 - Model Deployment
by Alejandro Correa Bahnsen
version 0.1, May 2016
Part of the class Machine Learning for Security Informatics
This notebook is licensed under a [Creative Commons Attribution-ShareAlike 3.0 Unported License]
Agenda:
Creating and saving a model
Running the model in batch
Exposing the model as an API
Part 1: Phishing Detection
Phishing, by definition, is the act of defrauding an online user in order to obtain personal information by posing as a trustworthy institution or entity. Users usually have a hard time differentiating between legitimate and malicious sites because they are made to look exactly the same. Therefore, there is a need to create better tools to combat attackers.
End of explanation
data.url[data.phishing==1].sample(50, random_state=1).tolist()
Explanation: Creating features
End of explanation
keywords = ['https', 'login', '.php', '.html', '@', 'sign']
for keyword in keywords:
    data['keyword_' + keyword] = data.url.str.contains(keyword).astype(int)
Explanation: Contain any of the following:
* https
* login
* .php
* .html
* @
* sign
* ?
End of explanation
data['lenght'] = data.url.str.len() - 2
domain = data.url.str.split('/', expand=True).iloc[:, 2]
data['lenght_domain'] = domain.str.len()
domain.head(12)
data['isIP'] = (domain.str.replace('.', '') * 1).str.isnumeric().astype(int)
data['count_com'] = data.url.str.count('com')
data.sample(15, random_state=4)
Explanation: Lenght of the url
Lenght of domain
is IP?
Number of .com
End of explanation
X = data.drop(['url', 'phishing'], axis=1)
y = data.phishing
from sklearn.ensemble import RandomForestClassifier
from sklearn.cross_validation import cross_val_score
clf = RandomForestClassifier(n_jobs=-1, n_estimators=100)
cross_val_score(clf, X, y, cv=10)
clf.fit(X, y)
Explanation: Create Model
End of explanation
from sklearn.externals import joblib
joblib.dump(clf, '22_clf_rf.pkl', compress=3)
Explanation: Save model
End of explanation
from m22_model_deployment import predict_proba
predict_proba('http://www.vipturismolondres.com/com.br/?atendimento=Cliente&/LgSgkszm64/B8aNzHa8Aj.php')
Explanation: Part 2: Model in batch
See 22_model_deployment.py
End of explanation
from flask import Flask
from flask.ext.restplus import Api
from flask.ext.restplus import fields
from sklearn.externals import joblib
from flask.ext.restplus import Resource
from sklearn.externals import joblib
import pandas as pd
Explanation: Part 3: API
Flask is considered more Pythonic than Django because Flask web application code is in most cases more explicit. Flask is easy to get started with as a beginner because there is little boilerplate code for getting a simple app up and running.
First we need to install some libraries 
pip install flask-restplus
Load Flask
End of explanation
app = Flask(__name__)
api = Api(
    app, 
    version='1.0', 
    title='Phishing Prediction API',
    description='Phishing Prediction API')
ns = api.namespace('predict', 
     description='Phishing Classifier')
   
parser = api.parser()
parser.add_argument(
    'URL', 
    type=str, 
    required=True, 
    help='URL to be analyzed', 
    location='args')
resource_fields = api.model('Resource', {
    'result': fields.String,
})
Explanation: Create api
End of explanation
clf = joblib.load('22_clf_rf.pkl') 
@ns.route('/')
class PhishingApi(Resource):
    @api.doc(parser=parser)
    @api.marshal_with(resource_fields)
    def get(self):
        args = parser.parse_args()
        result = self.predict_proba(args)
        return result, 200
    def predict_proba(self, args):
        url = args['URL']
        
        url_ = pd.DataFrame([url], columns=['url'])
        
        # Create features
        keywords = ['https', 'login', '.php', '.html', '@', 'sign']
        for keyword in keywords:
            url_['keyword_' + keyword] = url_.url.str.contains(keyword).astype(int)
        
        url_['lenght'] = url_.url.str.len() - 2
        domain = url_.url.str.split('/', expand=True).iloc[:, 2]
        url_['lenght_domain'] = domain.str.len()
        url_['isIP'] = (url_.url.str.replace('.', '') * 1).str.isnumeric().astype(int)
        url_['count_com'] = url_.url.str.count('com')
        # Make prediction
        p1 = clf.predict_proba(url_.drop('url', axis=1))[0,1]
        print('url=', url,'| p1=', p1)
        return {
         "result": p1
        }
Explanation: Load model and create function that predicts an URL
End of explanation
app.run(debug=True, use_reloader=False, host='0.0.0.0', port=5000)
Explanation: Run API
End of explanation
<END_TASK>