<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Regressions
Author
Step1:  Underfitting
<ASSISTANT_TASK:>
Python Code:
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
import math
x = np.linspace(0,4*math.pi,100)
y = map(math.sin,x)
plt.plot(x,y)
plt.plot(x,y,'o')
plt.show()
Explanation: Regressions
Author: Yang Long
Email:  longyang_123@yeah.net
Linear Regression
Logistic Regression
Softmax Regression
Regularization
Optimization Objective
Linear Regression
The most common form of linear regression can be written as:
$$ y = \theta X $$
which tell us that the value of $y$ is determined by the linear combination of components of $X$. But in the mind, the vector $X$ not only contains the all features ${x_i}$, but also contains the bias $x_0$. 
In some case, this linear regression can fit and predict the value of $y$ well, but unfortunately, it doesn't often behave as our wish. So for describing the bias between the practical value and analytical one, we define Cost Function J:
$$J = \frac{1}{m} \sum_{i=1}^{m} (\theta X^{(i)} - y^{(i)})^2 $$
Best value for parameters $\theta$ can minimize the cost function. 
Logistic Regression
The common form:
$$ y = h_{\theta}(X) $$
$$ h_{\theta}(X) = g(\theta X) $$
The sigmoid function:
$$ g(z) = \frac {1} {1+e^{-z}}$$
The cost function J for logistic regression would be described by Cross-Entropy as the following form:
$$J = \frac{1}{m} \sum_{i=1}^{m} -y^{(i)}log(h_{\theta}(X^{(i)})) - (1-y^{(i)})log(1-h_{\theta}(X^{(i)}))$$
Softmax Regression
$$p(y^{(i)}=j|x^{(i)};\theta)=\frac{e^{\theta^T_j x^{(i)}}} {\sum_{l=1}^{k} {\theta^T_l x^{(i)}}}$$
$$J(\theta) = -\frac{1}{m}(\sum_{i=1}^{m} \sum_{j=1}^{k}1{y^{(i)}=j} log\frac{e^{\theta^T_j x^{(i)}}} {\sum_{l=1}^{k} {\theta^T_l x^{(i)}}})+\frac {\lambda}{2} \sum_{i=1}^{m} \sum_{j=0}^{k} \theta_{ij}^2$$
Fitting Error
As we know, simple linear regression can not fit some specific function very well, especially for regular function, i.e. $sin(x)$. So we will add or extract more features to minimize the difference between fitting one and practical one.
Overfitting and underfitting
Two kinds of errors frequently happen when fitting the training data -- Overfit and Underfit.
Overfitting
End of explanation
import matplotlib.pyplot as plt
import numpy as np
import math
x = np.linspace(0,4*math.pi,100)
y = map(math.sin,x)
plt.plot(x,y)
plt.plot(x,y,'o')
plt.show()
Explanation: Underfitting
End of explanation
<END_TASK>