Is there any "standard" way to calculate the numerical gradient?
<p>I am trying to calculate the numerical gradient of a smooth function in c++. And the parameter value could vary from zero to a very large number(maybe 1e10 to 1e20?)</p>

<p>I used the function f(x,y) = 10*x^3 + y^3 as a testbench, but I found that if x or y is too large, I can't get correct gradient.</p>

<p>Here is my code to calculate the graidient:</p>

<pre><code>#include &lt;iostream&gt;
#include &lt;cmath&gt;
#include &lt;cassert&gt;
using namespace std;
double f(double x, double y)
{
    // black box expensive function
    return 10 * pow(x, 3) + pow(y, 3);
}
int main()
{
    // double x = -5897182590.8347721;
    // double y = 269857217.0017581;
    double x = 1.13041e+19;
    double y = -5.49756e+14;
    const double epsi = 1e-4;

    double f1 = f(x, y);
    double f2 = f(x, y+epsi);
    double f3 = f(x, y-epsi);
    cout &lt;&lt; f1 &lt;&lt; endl;
    cout &lt;&lt; f2 &lt;&lt; endl;
    cout &lt;&lt; f3 &lt;&lt; endl;
    cout &lt;&lt; f1 - f2 &lt;&lt; endl; // 0
    cout &lt;&lt; f2 - f3 &lt;&lt; endl; // 0
    return 0;
}
</code></pre>

<p>If I use the above code to calculate the gradient, the gradient would be zero!</p>

<p>The testbench function, 10*x^3 + y^3, is just a demo, the real problem I need to solve is actually a black box function.</p>

<p>So, is there any "standard" way to calculate the numerical gradient?</p>