<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 notes
Now I know I should think in column vector, and Tensorflow is very picky about the shape of data. But in numpy, the normal 1D ndarray is represented as column vector already. If I reshape $\mathbb{R}^n$ as $\mathbb{R}^{n\times1}$, It's not the same as column vector anymore. It's Matrix with 1 column. And I got troubles with scipy optimizer. 
So I should just treat tensorflow's data as special case. Keep using the convention of numpy world.
Step1:  sigmoid function
Step2:  cost function
$max(\ell(\theta)) = min(-\ell(\theta))$  
choose $-\ell(\theta)$ as the cost function
<img style="float
Step3:  looking good, be careful of the data shape
gradient
this is batch gradient  
translate this into vector computation $\frac{1}{m} X^T( Sigmoid(X\theta) - y )$
<img style="float
Step4:  fit the parameter
here I'm using scipy.optimize.minimize to find the parameters  
and I use this model without understanding.... what is Jacobian ...
Step5:  predict and validate from training set
now we are using training set to evaluate the model, this is not the best practice, but the course just begin, I guess Andrew will cover how to do model validation properlly later
Step6:  find the decision boundary
http
Step7:  you know the intercept would be around 125 for both x and y
<ASSISTANT_TASK:>
Python Code:
%reload_ext autoreload
%autoreload 2
%matplotlib inline
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
plt.style.use('fivethirtyeight')
import sys
sys.path.append('..')
from helper import logistic_regression as lr  # my own module
from helper import general as general
from sklearn.metrics import classification_report
# prepare data
data = pd.read_csv('ex2data1.txt', names=['exam1', 'exam2', 'admitted'])
data.head()
X = general.get_X(data)
print(X.shape)
y = general.get_y(data)
print(y.shape)
Explanation: notes
Now I know I should think in column vector, and Tensorflow is very picky about the shape of data. But in numpy, the normal 1D ndarray is represented as column vector already. If I reshape $\mathbb{R}^n$ as $\mathbb{R}^{n\times1}$, It's not the same as column vector anymore. It's Matrix with 1 column. And I got troubles with scipy optimizer. 
So I should just treat tensorflow's data as special case. Keep using the convention of numpy world.
End of explanation
fig, ax = plt.subplots(figsize=(8, 6))
ax.plot(np.arange(-10, 10, step=0.01),
        lr.sigmoid(np.arange(-10, 10, step=0.01)))
ax.set_ylim((-0.1,1.1))
ax.set_xlabel('z', fontsize=18)
ax.set_ylabel('g(z)', fontsize=18)
ax.set_title('sigmoid function', fontsize=18)
Explanation: sigmoid function
End of explanation
theta = theta=np.zeros(3) # X(m*n) so theta is n*1
theta
lr.cost(theta, X, y)
Explanation: cost function
$max(\ell(\theta)) = min(-\ell(\theta))$  
choose $-\ell(\theta)$ as the cost function
<img style="float: left;" src="../img/logistic_cost.png">
End of explanation
lr.gradient(theta, X, y)
Explanation: looking good, be careful of the data shape
gradient
this is batch gradient  
translate this into vector computation $\frac{1}{m} X^T( Sigmoid(X\theta) - y )$
<img style="float: left;" src="../img/logistic_gradient.png">
End of explanation
import scipy.optimize as opt
res = opt.minimize(fun=lr.cost, x0=theta, args=(X, y), method='Newton-CG', jac=lr.gradient)
print(res)
Explanation: fit the parameter
here I'm using scipy.optimize.minimize to find the parameters  
and I use this model without understanding.... what is Jacobian ...
End of explanation
final_theta = res.x
y_pred = lr.predict(X, final_theta)
print(classification_report(y, y_pred))
Explanation: predict and validate from training set
now we are using training set to evaluate the model, this is not the best practice, but the course just begin, I guess Andrew will cover how to do model validation properlly later
End of explanation
print(res.x) # this is final theta
coef = -(res.x / res.x[2])  # find the equation
print(coef)
x = np.arange(130, step=0.1)
y = coef[0] + coef[1]*x
data.describe()  # find the range of x and y
Explanation: find the decision boundary
http://stats.stackexchange.com/questions/93569/why-is-logistic-regression-a-linear-classifier
$X \times \theta = 0$  (this is the line)
End of explanation
sns.set(context="notebook", style="ticks", font_scale=1.5)
sns.lmplot('exam1', 'exam2', hue='admitted', data=data, 
           size=6, 
           fit_reg=False, 
           scatter_kws={"s": 25}
          )
plt.plot(x, y, 'grey')
plt.xlim(0, 130)
plt.ylim(0, 130)
plt.title('Decision Boundary')
Explanation: you know the intercept would be around 125 for both x and y
End of explanation
<END_TASK>