<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 linear regression
解析式直接求解
Step1:  $y = Xw$
$ w = (X^TX)^[-1]X^T*y$
Step2:  Results
w1 = 2.97396653
w2 = -0.54139002
w3 = 0.97132913
b = 2.03076198
Step3:  梯度下降法求解
目标函数选取要合适一些, 前边乘以适当的系数.
注意检验梯度的计算是否正确...
Step4:  随机梯度下降法求解
Stochastic gradient descent
使用了固定步长
一开始用的0.1, 始终达不到给定的精度
于是添加了判定条件用来更新步长.
<ASSISTANT_TASK:>
Python Code:
df['x4'] = 1
X = df.iloc[:,(0,1,2,4)].values
y = df.y.values
Explanation: linear regression
解析式直接求解
End of explanation
inv_XX_T = inv(X.T.dot(X))
w = inv_XX_T.dot(X.T).dot(df.y.values)
w
Explanation: $y = Xw$
$ w = (X^TX)^[-1]X^T*y$
End of explanation
qr(inv_XX_T)
X.shape
#solve(X,y)##只能解方阵
Explanation: Results
w1 = 2.97396653
w2 = -0.54139002
w3 = 0.97132913
b = 2.03076198
End of explanation
def f(w,X,y):    
    return ((X.dot(w)-y)**2/(2*1000)).sum()
def grad_f(w,X,y):
    return  (X.dot(w) - y).dot(X)/1000
w0 = np.array([100.0,100.0,100.0,100.0])
epsilon = 1e-10
alpha = 0.1
check_condition = 1
while  check_condition > epsilon:
    w0 += -alpha*grad_f(w0,X,y)
    check_condition = abs(grad_f(w0,X,y)).sum()
print w0
Explanation: 梯度下降法求解
目标函数选取要合适一些, 前边乘以适当的系数.
注意检验梯度的计算是否正确...
End of explanation
def cost_function(w,X,y):    
    return (X.dot(w)-y)**2/2
def grad_cost_f(w,X,y):
    return  (np.dot(X, w) - y)*X
w0 = np.array([1.0, 1.0, 1.0, 1.0])
epsilon = 1e-3
alpha = 0.01
# 生成随机index,用来随机索引数据.
random_index = np.arange(1000)
np.random.shuffle(random_index)
cost_value = np.inf #初始化目标函数值
while abs(grad_f(w0,X,y)).sum() > epsilon:
    for i in range(1000):
        w0 += -alpha*grad_cost_f(w0,X[random_index[i]],y[random_index[i]])
    
    #检查目标函数变化趋势, 如果趋势变化达到临界值, 更新更小的步长继续计算
    difference = cost_value - f(w0, X, y) 
    if difference < 1e-10:
        alpha *= 0.9
    cost_value = f(w0, X, y)
print w0
Explanation: 随机梯度下降法求解
Stochastic gradient descent
使用了固定步长
一开始用的0.1, 始终达不到给定的精度
于是添加了判定条件用来更新步长.
End of explanation
<END_TASK>