Penalty function method
<p>I am trying to implement penalty function method for minimizing function. I need to find the minimum of <a href="https://en.wikipedia.org/wiki/Rosenbrock_function" rel="nofollow noreferrer">Rosenbrok's function</a>.</p>

<p>I am using this penalty function:</p>

<p><a href="https://i.stack.imgur.com/UgHju.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/UgHju.png" alt="Penalty Function"></a></p>

<p><a href="https://i.stack.imgur.com/nDMvR.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/nDMvR.png" alt="Penalty Function"></a></p>

<p>First of all, I have found the minimum using <code>scipy.optimize.minimize</code>:</p>

<pre><code>from scipy.optimize import minimize, rosen
rz = lambda x: (1-x[0])**2 + 100*(x[1] - x[0]**2)**2;
h_1 = lambda x: (x[0] - 2 * x[1] + 2);
h_2 = lambda x: (-x[0] - 2 * x[1] + 6);
h_3 = lambda x: (-x[0] + 2 * x[1] + 2);

x0 = [2.3, 5];
cons = ({'type': 'ineq', 'fun': h_1},
       {'type': 'ineq', 'fun': h_2},
       {'type': 'ineq', 'fun': h_3}) 
minimize(rz, x0, constraints=cons)
</code></pre>

<p>The answer is <code>x</code>: <code>array([ 0.99971613,  0.99942073])</code></p>

<p>Then I am trying to find the minimum using my implementation of penalty method:</p>

<pre><code>x_c = [2.3, 3];
i = 1;
while i &lt; 1000:
    curr_func = lambda x: rz(x) + i*(h_1(x)**2 + h_2(x)**2 + h_3(x)**2)
    x_c = minimize(curr_func, x_c).x;
    i  *= 1.2;
print(answer.x);
</code></pre>

<p>Which gives me <code>[ 2.27402022  1.4157964 ]</code> (if I increase the number of iterations, final values are even greater).</p>

<p>Where is the mistake in my implementation?
Thanks.</p>

<p>P.S. Function <code>curr_func</code> is specific for my constraints, of course, when they are all 'inequals' type.</p>