
<!DOCTYPE html>

<html lang="en">
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.18.1: http://docutils.sourceforge.net/" />

    <title>Overview and Tutorial &#8212; lsqfit 13.0.1 documentation</title>
    <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
    <link rel="stylesheet" type="text/css" href="_static/pyramid.css" />
    <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
    <script src="_static/jquery.js"></script>
    <script src="_static/underscore.js"></script>
    <script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
    <script src="_static/doctools.js"></script>
    <link rel="index" title="Index" href="genindex.html" />
    <link rel="search" title="Search" href="search.html" />
    <link rel="next" title="Non-Gaussian Behavior; Testing Fits" href="testing.html" />
    <link rel="prev" title="lsqfit Documentation" href="index.html" />
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Neuton&amp;subset=latin" type="text/css" media="screen" charset="utf-8" />
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Nobile:regular,italic,bold,bolditalic&amp;subset=latin" type="text/css" media="screen" charset="utf-8" />
<!--[if lte IE 6]>
<link rel="stylesheet" href="_static/ie6.css" type="text/css" media="screen" charset="utf-8" />
<![endif]-->

  </head><body>

    <div class="related" role="navigation" aria-label="related navigation">
      <h3>Navigation</h3>
      <ul>
        <li class="right" style="margin-right: 10px">
          <a href="genindex.html" title="General Index"
             accesskey="I">index</a></li>
        <li class="right" >
          <a href="py-modindex.html" title="Python Module Index"
             >modules</a> |</li>
        <li class="right" >
          <a href="testing.html" title="Non-Gaussian Behavior; Testing Fits"
             accesskey="N">next</a> |</li>
        <li class="right" >
          <a href="index.html" title="lsqfit Documentation"
             accesskey="P">previous</a> |</li>
        <li class="nav-item nav-item-0"><a href="index.html">lsqfit 13.0.1 documentation</a> &#187;</li>
        <li class="nav-item nav-item-this"><a href="">Overview and Tutorial</a></li> 
      </ul>
    </div>  

    <div class="document">
      <div class="documentwrapper">
        <div class="bodywrapper">
          <div class="body" role="main">
            
  <section id="overview-and-tutorial">
<h1>Overview and Tutorial<a class="headerlink" href="#overview-and-tutorial" title="Permalink to this heading">¶</a></h1>
<section id="introduction">
<h2>Introduction<a class="headerlink" href="#introduction" title="Permalink to this heading">¶</a></h2>
<p>The  <a class="reference internal" href="lsqfit.html#module-lsqfit" title="lsqfit: Nonlinear least squares fitting."><code class="xref py py-mod docutils literal notranslate"><span class="pre">lsqfit</span></code></a> module is designed to facilitate least-squares fitting of
noisy data by multi-dimensional, nonlinear functions of arbitrarily many
parameters, each with a (Bayesian) prior.  <a class="reference internal" href="lsqfit.html#module-lsqfit" title="lsqfit: Nonlinear least squares fitting."><code class="xref py py-mod docutils literal notranslate"><span class="pre">lsqfit</span></code></a> makes heavy use of
another module, <code class="xref py py-mod docutils literal notranslate"><span class="pre">gvar</span></code> (distributed separately), which provides tools
that simplify the analysis of error propagation, and also the creation of
complicated multi-dimensional Gaussian distributions.
This technology also allows <a class="reference internal" href="lsqfit.html#module-lsqfit" title="lsqfit: Nonlinear least squares fitting."><code class="xref py py-mod docutils literal notranslate"><span class="pre">lsqfit</span></code></a>
to calculate exact derivatives of fit functions from the fit functions
themselves, using automatic differentiation, thereby avoiding the need to
code these by hand (the fitters use the derivatives).
The power of the
<code class="xref py py-mod docutils literal notranslate"><span class="pre">gvar</span></code> module, particularly for correlated distributions, enables
a variety of unusual fitting strategies, as we illustrate below;
it is a feature that distinguishes <a class="reference internal" href="lsqfit.html#module-lsqfit" title="lsqfit: Nonlinear least squares fitting."><code class="xref py py-mod docutils literal notranslate"><span class="pre">lsqfit</span></code></a> from
standard fitting packages.</p>
<p>The following (complete) code illustrates basic usage of <a class="reference internal" href="lsqfit.html#module-lsqfit" title="lsqfit: Nonlinear least squares fitting."><code class="xref py py-mod docutils literal notranslate"><span class="pre">lsqfit</span></code></a>:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">import</span> <span class="nn">gvar</span> <span class="k">as</span> <span class="nn">gv</span>
<span class="kn">import</span> <span class="nn">lsqfit</span>

<span class="n">y</span> <span class="o">=</span> <span class="p">{</span>                                 <span class="c1"># data for the dependent variable</span>
   <span class="s1">&#39;data1&#39;</span> <span class="p">:</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">([</span><span class="mf">1.376</span><span class="p">,</span> <span class="mf">2.010</span><span class="p">],</span> <span class="p">[[</span> <span class="mf">0.0047</span><span class="p">,</span> <span class="mf">0.01</span><span class="p">],</span> <span class="p">[</span> <span class="mf">0.01</span><span class="p">,</span> <span class="mf">0.056</span><span class="p">]]),</span>
   <span class="s1">&#39;data2&#39;</span> <span class="p">:</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">([</span><span class="mf">1.329</span><span class="p">,</span> <span class="mf">1.582</span><span class="p">],</span> <span class="p">[[</span> <span class="mf">0.0047</span><span class="p">,</span> <span class="mf">0.0067</span><span class="p">],</span> <span class="p">[</span><span class="mf">0.0067</span><span class="p">,</span> <span class="mf">0.0136</span><span class="p">]]),</span>
   <span class="s1">&#39;b/a&#39;</span>   <span class="p">:</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="mf">2.0</span><span class="p">,</span> <span class="mf">0.5</span><span class="p">)</span>
   <span class="p">}</span>
<span class="n">x</span> <span class="o">=</span> <span class="p">{</span>                                 <span class="c1"># independent variable</span>
   <span class="s1">&#39;data1&#39;</span> <span class="p">:</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">0.1</span><span class="p">,</span> <span class="mf">1.0</span><span class="p">]),</span>
   <span class="s1">&#39;data2&#39;</span> <span class="p">:</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">0.1</span><span class="p">,</span> <span class="mf">0.5</span><span class="p">])</span>
   <span class="p">}</span>
<span class="n">prior</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">prior</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="mf">0.5</span><span class="p">,</span> <span class="mf">0.5</span><span class="p">)</span>
<span class="n">prior</span><span class="p">[</span><span class="s1">&#39;b&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="mf">0.5</span><span class="p">,</span> <span class="mf">0.5</span><span class="p">)</span>

<span class="k">def</span> <span class="nf">fcn</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">p</span><span class="p">):</span>                        <span class="c1"># fit function of x and parameters p</span>
  <span class="n">ans</span> <span class="o">=</span> <span class="p">{}</span>
  <span class="k">for</span> <span class="n">k</span> <span class="ow">in</span> <span class="p">[</span><span class="s1">&#39;data1&#39;</span><span class="p">,</span> <span class="s1">&#39;data2&#39;</span><span class="p">]:</span>
     <span class="n">ans</span><span class="p">[</span><span class="n">k</span><span class="p">]</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]</span> <span class="o">+</span> <span class="n">x</span><span class="p">[</span><span class="n">k</span><span class="p">]</span> <span class="o">*</span> <span class="n">p</span><span class="p">[</span><span class="s1">&#39;b&#39;</span><span class="p">])</span>
  <span class="n">ans</span><span class="p">[</span><span class="s1">&#39;b/a&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">p</span><span class="p">[</span><span class="s1">&#39;b&#39;</span><span class="p">]</span> <span class="o">/</span> <span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]</span>
  <span class="k">return</span> <span class="n">ans</span>

<span class="c1"># do the fit</span>
<span class="n">fit</span> <span class="o">=</span> <span class="n">lsqfit</span><span class="o">.</span><span class="n">nonlinear_fit</span><span class="p">(</span><span class="n">data</span><span class="o">=</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">),</span> <span class="n">prior</span><span class="o">=</span><span class="n">prior</span><span class="p">,</span> <span class="n">fcn</span><span class="o">=</span><span class="n">fcn</span><span class="p">,</span> <span class="n">debug</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">fit</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">maxline</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span>       <span class="c1"># print standard summary of fit</span>

<span class="n">p</span> <span class="o">=</span> <span class="n">fit</span><span class="o">.</span><span class="n">p</span>                             <span class="c1"># best-fit values for parameters</span>
<span class="n">outputs</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">a</span><span class="o">=</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">],</span> <span class="n">b</span><span class="o">=</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;b&#39;</span><span class="p">])</span>
<span class="n">outputs</span><span class="p">[</span><span class="s1">&#39;b/a&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">p</span><span class="p">[</span><span class="s1">&#39;b&#39;</span><span class="p">]</span><span class="o">/</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]</span>
<span class="n">inputs</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">y</span><span class="o">=</span><span class="n">y</span><span class="p">,</span> <span class="n">prior</span><span class="o">=</span><span class="n">prior</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">gv</span><span class="o">.</span><span class="n">fmt_values</span><span class="p">(</span><span class="n">outputs</span><span class="p">))</span>              <span class="c1"># tabulate outputs</span>
<span class="nb">print</span><span class="p">(</span><span class="n">gv</span><span class="o">.</span><span class="n">fmt_errorbudget</span><span class="p">(</span><span class="n">outputs</span><span class="p">,</span> <span class="n">inputs</span><span class="p">))</span> <span class="c1"># print error budget for outputs</span>
</pre></div>
</div>
<p>This code fits the function <code class="docutils literal notranslate"><span class="pre">f(x,a,b)=</span> <span class="pre">exp(a+b*x)</span></code> (see <code class="docutils literal notranslate"><span class="pre">fcn(x,p)</span></code>)
to two sets of data, labeled <code class="docutils literal notranslate"><span class="pre">data1</span></code> and <code class="docutils literal notranslate"><span class="pre">data2</span></code>, by varying parameters
<code class="docutils literal notranslate"><span class="pre">a</span></code> and <code class="docutils literal notranslate"><span class="pre">b</span></code> until <code class="docutils literal notranslate"><span class="pre">f(x['data1'],a,b)</span></code> and <code class="docutils literal notranslate"><span class="pre">f(x['data2'],a,b)</span></code>
equal <code class="docutils literal notranslate"><span class="pre">y['data1']</span></code> and <code class="docutils literal notranslate"><span class="pre">y['data2']</span></code>, respectively, to within the
<code class="docutils literal notranslate"><span class="pre">y</span></code>s’ errors.</p>
<p>The means and covariance matrices for the <code class="docutils literal notranslate"><span class="pre">y</span></code>s are
specified in the <code class="docutils literal notranslate"><span class="pre">gv.gvar(...)</span></code>s used to create them: thus, for example,</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">y</span><span class="p">[</span><span class="s1">&#39;data1&#39;</span><span class="p">])</span>
<span class="go">[1.376(69) 2.01(24)]</span>
<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">y</span><span class="p">[</span><span class="s1">&#39;data1&#39;</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">mean</span><span class="p">,</span> <span class="s2">&quot;+-&quot;</span><span class="p">,</span> <span class="n">y</span><span class="p">[</span><span class="s1">&#39;data1&#39;</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">sdev</span><span class="p">)</span>
<span class="go">1.376 +- 0.068556546004</span>
<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">gv</span><span class="o">.</span><span class="n">evalcov</span><span class="p">(</span><span class="n">y</span><span class="p">[</span><span class="s1">&#39;data1&#39;</span><span class="p">]))</span>   <span class="c1"># covariance matrix</span>
<span class="go">[[ 0.0047  0.01  ]</span>
<span class="go"> [ 0.01    0.056 ]]</span>
</pre></div>
</div>
<p>shows the means, standard deviations and covariance matrix for the data in
the first data set (0.0685565 is the square root of the 0.0047 in
the covariance matrix).</p>
<p>The dictionary <code class="docutils literal notranslate"><span class="pre">prior</span></code> gives <em>a priori</em> estimates
for the two parameters, <code class="docutils literal notranslate"><span class="pre">a</span></code> and <code class="docutils literal notranslate"><span class="pre">b</span></code>: each is assumed to be 0.5±0.5
before fitting. The parameters <code class="docutils literal notranslate"><span class="pre">p[k]</span></code> in the fit function <code class="docutils literal notranslate"><span class="pre">fcn(x,</span> <span class="pre">p)</span></code>
are stored in a dictionary having the same keys and layout as
<code class="docutils literal notranslate"><span class="pre">prior</span></code> (since <code class="docutils literal notranslate"><span class="pre">prior</span></code> specifies the fit parameters for
the fitter).</p>
<p>In addition to the <code class="docutils literal notranslate"><span class="pre">data1</span></code> and <code class="docutils literal notranslate"><span class="pre">data2</span></code> data sets,
there is an extra piece of input data,
<code class="docutils literal notranslate"><span class="pre">y['b/a']</span></code>, which indicates that <code class="docutils literal notranslate"><span class="pre">b/a</span></code> is 2±0.5. The fit
function for this data is simply the ratio <code class="docutils literal notranslate"><span class="pre">b/a</span></code> (represented by
<code class="docutils literal notranslate"><span class="pre">p['b']/p['a']</span></code> in fit function <code class="docutils literal notranslate"><span class="pre">fcn(x,p)</span></code>). The fit function returns
a dictionary having the same keys and layout as the input data <code class="docutils literal notranslate"><span class="pre">y</span></code>.</p>
<p>The output from the code sample above is:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.17</span> <span class="p">[</span><span class="mi">5</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.97</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">0.65538</span>

<span class="n">Parameters</span><span class="p">:</span>
              <span class="n">a</span>   <span class="mf">0.253</span> <span class="p">(</span><span class="mi">32</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="n">b</span>   <span class="mf">0.449</span> <span class="p">(</span><span class="mi">65</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Fit</span><span class="p">:</span>
        <span class="n">key</span>        <span class="n">y</span><span class="p">[</span><span class="n">key</span><span class="p">]</span>     <span class="n">f</span><span class="p">(</span><span class="n">p</span><span class="p">)[</span><span class="n">key</span><span class="p">]</span>
<span class="o">---------------------------------------</span>
        <span class="n">b</span><span class="o">/</span><span class="n">a</span>     <span class="mf">2.00</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span>     <span class="mf">1.78</span> <span class="p">(</span><span class="mi">30</span><span class="p">)</span>  
    <span class="n">data1</span> <span class="mi">0</span>    <span class="mf">1.376</span> <span class="p">(</span><span class="mi">69</span><span class="p">)</span>    <span class="mf">1.347</span> <span class="p">(</span><span class="mi">46</span><span class="p">)</span>  
          <span class="mi">1</span>     <span class="mf">2.01</span> <span class="p">(</span><span class="mi">24</span><span class="p">)</span>     <span class="mf">2.02</span> <span class="p">(</span><span class="mi">16</span><span class="p">)</span>  
    <span class="n">data2</span> <span class="mi">0</span>    <span class="mf">1.329</span> <span class="p">(</span><span class="mi">69</span><span class="p">)</span>    <span class="mf">1.347</span> <span class="p">(</span><span class="mi">46</span><span class="p">)</span>  
          <span class="mi">1</span>     <span class="mf">1.58</span> <span class="p">(</span><span class="mi">12</span><span class="p">)</span>    <span class="mf">1.612</span> <span class="p">(</span><span class="mi">82</span><span class="p">)</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">8</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="n">Values</span><span class="p">:</span>
                  <span class="n">a</span><span class="p">:</span> <span class="mf">0.253</span><span class="p">(</span><span class="mi">32</span><span class="p">)</span>           
                <span class="n">b</span><span class="o">/</span><span class="n">a</span><span class="p">:</span> <span class="mf">1.78</span><span class="p">(</span><span class="mi">30</span><span class="p">)</span>            
                  <span class="n">b</span><span class="p">:</span> <span class="mf">0.449</span><span class="p">(</span><span class="mi">65</span><span class="p">)</span>           

<span class="n">Partial</span> <span class="o">%</span> <span class="n">Errors</span><span class="p">:</span>
                   <span class="n">a</span>       <span class="n">b</span><span class="o">/</span><span class="n">a</span>         <span class="n">b</span>
<span class="o">----------------------------------------</span>
        <span class="n">y</span><span class="p">:</span>     <span class="mf">12.75</span>     <span class="mf">16.72</span>     <span class="mf">14.30</span>
    <span class="n">prior</span><span class="p">:</span>      <span class="mf">0.92</span>      <span class="mf">1.58</span>      <span class="mf">1.88</span>
<span class="o">----------------------------------------</span>
    <span class="n">total</span><span class="p">:</span>     <span class="mf">12.78</span>     <span class="mf">16.80</span>     <span class="mf">14.42</span>

</pre></div>
</div>
<p>The best-fit values for <code class="docutils literal notranslate"><span class="pre">a</span></code> and <code class="docutils literal notranslate"><span class="pre">b</span></code> are 0.253(32) and
0.449(65), respectively; and the best-fit result for <code class="docutils literal notranslate"><span class="pre">b/a</span></code> is
1.78(30), which, because of correlations, is slightly more accurate
than might be expected from the separate errors for <code class="docutils literal notranslate"><span class="pre">a</span></code> and <code class="docutils literal notranslate"><span class="pre">b</span></code>. The
error budget for each of these three quantities is tabulated at the end and
shows that the bulk of the error in each case comes from uncertainties in
the <code class="docutils literal notranslate"><span class="pre">y</span></code> data, with only small contributions from uncertainties in the
priors <code class="docutils literal notranslate"><span class="pre">prior</span></code>. The fit results corresponding to each piece of input data
are also tabulated (<code class="docutils literal notranslate"><span class="pre">Fit:</span> <span class="pre">...</span></code>); the agreement is excellent, as expected
given that the <code class="docutils literal notranslate"><span class="pre">chi**2</span></code> per degree of freedom is only 0.17.</p>
<p>Note that the constraint in <code class="docutils literal notranslate"><span class="pre">y</span></code> on <code class="docutils literal notranslate"><span class="pre">b/a</span></code> in this example is much tighter
than the constraints on <code class="docutils literal notranslate"><span class="pre">a</span></code> and <code class="docutils literal notranslate"><span class="pre">b</span></code> separately. This suggests a variation
on the previous code, where the tight restriction on <code class="docutils literal notranslate"><span class="pre">b/a</span></code> is built into the
prior rather than <code class="docutils literal notranslate"><span class="pre">y</span></code>:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="o">...</span> <span class="k">as</span> <span class="n">before</span> <span class="o">...</span>

<span class="n">y</span> <span class="o">=</span> <span class="p">{</span>                      <span class="c1"># data for the dependent variable</span>
    <span class="s1">&#39;data1&#39;</span> <span class="p">:</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">([</span><span class="mf">1.376</span><span class="p">,</span> <span class="mf">2.010</span><span class="p">],</span> <span class="p">[[</span> <span class="mf">0.0047</span><span class="p">,</span> <span class="mf">0.01</span><span class="p">],</span> <span class="p">[</span> <span class="mf">0.01</span><span class="p">,</span> <span class="mf">0.056</span><span class="p">]]),</span>
    <span class="s1">&#39;data2&#39;</span> <span class="p">:</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">([</span><span class="mf">1.329</span><span class="p">,</span> <span class="mf">1.582</span><span class="p">],</span> <span class="p">[[</span> <span class="mf">0.0047</span><span class="p">,</span> <span class="mf">0.0067</span><span class="p">],</span> <span class="p">[</span><span class="mf">0.0067</span><span class="p">,</span> <span class="mf">0.0136</span><span class="p">]])</span>
    <span class="p">}</span>
<span class="n">x</span> <span class="o">=</span> <span class="p">{</span>                      <span class="c1"># independent variable</span>
    <span class="s1">&#39;data1&#39;</span> <span class="p">:</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">0.1</span><span class="p">,</span> <span class="mf">1.0</span><span class="p">]),</span>
    <span class="s1">&#39;data2&#39;</span> <span class="p">:</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">0.1</span><span class="p">,</span> <span class="mf">0.5</span><span class="p">])</span>
    <span class="p">}</span>
<span class="n">prior</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">prior</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="mf">0.5</span><span class="p">,</span> <span class="mf">0.5</span><span class="p">)</span>
<span class="n">prior</span><span class="p">[</span><span class="s1">&#39;b&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">prior</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]</span> <span class="o">*</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="mf">2.0</span><span class="p">,</span> <span class="mf">0.5</span><span class="p">)</span>

<span class="k">def</span> <span class="nf">fcn</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">p</span><span class="p">):</span>             <span class="c1"># fit function of x and parameters p[k]</span>
   <span class="n">ans</span> <span class="o">=</span> <span class="p">{}</span>
   <span class="k">for</span> <span class="n">k</span> <span class="ow">in</span> <span class="p">[</span><span class="s1">&#39;data1&#39;</span><span class="p">,</span> <span class="s1">&#39;data2&#39;</span><span class="p">]:</span>
      <span class="n">ans</span><span class="p">[</span><span class="n">k</span><span class="p">]</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]</span> <span class="o">+</span> <span class="n">x</span><span class="p">[</span><span class="n">k</span><span class="p">]</span><span class="o">*</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;b&#39;</span><span class="p">])</span>
   <span class="k">return</span> <span class="n">ans</span>

<span class="o">...</span> <span class="k">as</span> <span class="n">before</span> <span class="o">...</span>
</pre></div>
</div>
<p>Here the dependent data <code class="docutils literal notranslate"><span class="pre">y</span></code> no longer has an entry for <code class="docutils literal notranslate"><span class="pre">b/a</span></code>, and neither
do results from the fit function; but the prior for <code class="docutils literal notranslate"><span class="pre">b</span></code> is now 2±0.5
times the prior for <code class="docutils literal notranslate"><span class="pre">a</span></code>, thereby introducing a correlation that
limits the ratio <code class="docutils literal notranslate"><span class="pre">b/a</span></code> to be 2±0.5 in the fit. This code gives almost
identical results to the first one — very slightly less accurate, since
there is slightly less input data. We can often move information
from the <code class="docutils literal notranslate"><span class="pre">y</span></code> data to
the prior or back since both are forms of input information.</p>
<p>There are several things worth noting from this example:</p>
<blockquote>
<div><ul>
<li><p>The input data (<code class="docutils literal notranslate"><span class="pre">y</span></code>) is expressed in terms of Gaussian random
variables — quantities with means and a covariance matrix. These are
represented by objects of type <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code> in the code; module
<code class="xref py py-mod docutils literal notranslate"><span class="pre">gvar</span></code> has a variety of tools for creating and manipulating
Gaussian random variables (also see below).</p></li>
<li><p>The input data is stored in a dictionary (<code class="docutils literal notranslate"><span class="pre">y</span></code>) whose values can
be <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s or arrays of <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s. The use of a dictionary allows for
far greater flexibility than, say, an array. The fit function
(<code class="docutils literal notranslate"><span class="pre">fcn(x,</span> <span class="pre">p)</span></code>) has to return a dictionary with the same layout as
that of <code class="docutils literal notranslate"><span class="pre">y</span></code> (that is, with the same keys and where the value for
each key has the same shape as the corresponding value in <code class="docutils literal notranslate"><span class="pre">y</span></code>).
<a class="reference internal" href="lsqfit.html#module-lsqfit" title="lsqfit: Nonlinear least squares fitting."><code class="xref py py-mod docutils literal notranslate"><span class="pre">lsqfit</span></code></a> allows <code class="docutils literal notranslate"><span class="pre">y</span></code> to be an array instead of a dictionary,
which might be preferable for simple fits (but usually not
otherwise).</p></li>
<li><p>The independent data (<code class="docutils literal notranslate"><span class="pre">x</span></code>) can be anything; it is simply passed
through the fit code to the fit function <code class="docutils literal notranslate"><span class="pre">fcn(x,p)</span></code>. It can
also be omitted altogether, in which case the fit function
depends only upon the parameters: <code class="docutils literal notranslate"><span class="pre">fcn(p)</span></code>.</p></li>
<li><p>The fit parameters (<code class="docutils literal notranslate"><span class="pre">p</span></code> in <code class="docutils literal notranslate"><span class="pre">fcn(x,p)</span></code>) are also stored in a
dictionary whose values are <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s or arrays of <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s. Again this
allows for great flexibility. The layout of the parameter dictionary
is copied from that of the prior (<code class="docutils literal notranslate"><span class="pre">prior</span></code>). Again <code class="docutils literal notranslate"><span class="pre">p</span></code> can be a
single array instead of a dictionary, if that simplifies the code.</p></li>
<li><p>The best-fit values of the fit parameters (<code class="docutils literal notranslate"><span class="pre">fit.p[k]</span></code>) are also
<code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s and these capture statistical correlations between different
parameters that are indicated by the fit. These output parameters can
be combined in arithmetic expressions, using standard operators and
standard functions, to obtain derived quantities. These operations
take account of and track statistical correlations.</p></li>
<li><p>Function <code class="xref py py-func docutils literal notranslate"><span class="pre">gvar.fmt_errorbudget()</span></code> is a useful tool for assessing
the origins (<code class="docutils literal notranslate"><span class="pre">inputs</span></code>) of the statistical errors obtained in various
final results (<code class="docutils literal notranslate"><span class="pre">outputs</span></code>). It is particularly useful for analyzing
the impact of the <em>a priori</em> uncertainties encoded in the prior
(<code class="docutils literal notranslate"><span class="pre">prior</span></code>).</p></li>
<li><p>Parameter <code class="docutils literal notranslate"><span class="pre">debug=True</span></code> is set in <a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a>. This is a good idea,
particularly in the early stages of a project, because it causes the
code to check for various common errors and give more intelligible
error messages than would otherwise arise. This parameter can be dropped
once code development is over.</p></li>
<li><p>The priors for the fit parameters specify Gaussian distributions,
characterized by the means and standard deviations given
<code class="docutils literal notranslate"><span class="pre">gv.gvar(...)</span></code>. Some other distributions are available, and
new ones can be created.  The
distribution for parameter <code class="docutils literal notranslate"><span class="pre">a</span></code>, for example, can be switched to a
log-normal distribution by replacing <code class="docutils literal notranslate"><span class="pre">prior['a']=gv.gvar(0.5,</span> <span class="pre">0.5)</span></code>
with:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">prior</span><span class="p">[</span><span class="s1">&#39;log(a)&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="mf">0.5</span><span class="p">,</span><span class="mf">0.5</span><span class="p">))</span>
</pre></div>
</div>
<p>in the code. This change would
be desirable, for example, if we knew <em>a priori</em> that
parameter <code class="docutils literal notranslate"><span class="pre">a</span></code> is positive
since this is guaranteed with a log-normal distribution. Only the
prior need be changed. (In particular, the fit function <code class="docutils literal notranslate"><span class="pre">fcn(x,p)</span></code>
need <em>not</em> be changed.)</p>
</li>
</ul>
</div></blockquote>
<p>What follows is a tutorial that demonstrates in greater detail how to use
these modules in a selection of variations on the data fitting problem. As
above, code for the examples is specified completely (with one exception) and
so can be copied into a file, and run as is. It can also be modified, allowing
for experimentation.</p>
<p>Another way to learn about the modules is to examine the case studies
that follow this section. Each focuses on a single problem, again with
the full code and data to allow for experimentation.</p>
<p><em>About Printing:</em> The examples in this tutorial use the <code class="docutils literal notranslate"><span class="pre">print</span></code> function
as it is used in Python 3. Drop the outermost parenthesis in each <code class="docutils literal notranslate"><span class="pre">print</span></code>
statement if using Python 2; or add</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">__future__</span> <span class="kn">import</span> <span class="n">print_function</span>
</pre></div>
</div>
<p>at the start of your file.</p>
</section>
<section id="gaussian-random-variables-and-error-propagation">
<h2>Gaussian Random Variables and Error Propagation<a class="headerlink" href="#gaussian-random-variables-and-error-propagation" title="Permalink to this heading">¶</a></h2>
<p>The inputs and outputs of a nonlinear least squares analysis are probability
distributions, and these distributions will be Gaussian provided the input
data are sufficiently accurate. <a class="reference internal" href="lsqfit.html#module-lsqfit" title="lsqfit: Nonlinear least squares fitting."><code class="xref py py-mod docutils literal notranslate"><span class="pre">lsqfit</span></code></a> assumes this to be the case.
(It also provides tests for non-Gaussian behavior, together with
methods for dealing with such behavior. See: <a class="reference internal" href="testing.html#non-gaussian-behavior"><span class="std std-ref">Non-Gaussian Behavior; Testing Fits</span></a>.)</p>
<p>One of the most distinctive features of <a class="reference internal" href="lsqfit.html#module-lsqfit" title="lsqfit: Nonlinear least squares fitting."><code class="xref py py-mod docutils literal notranslate"><span class="pre">lsqfit</span></code></a> is that it is
built around a class, <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>, of objects that can be used to
represent arbitrarily complicated Gaussian distributions
— that is, they represent <em>Gaussian random variables</em> that specify the means and
covariance matrix of the probability distributions.
The input data for a fit are represented
by a collection of <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s that specify both the values and possible
errors in the input values. The result of a fit is a collection of
<code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s specifying the best-fit values for the fit parameters and the
estimated uncertainties in those values.</p>
<p><code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s are defined in the <code class="xref py py-mod docutils literal notranslate"><span class="pre">gvar</span></code> module.
There are five important things to know about them (see the
<code class="xref py py-mod docutils literal notranslate"><span class="pre">gvar</span></code> documentation for more details):</p>
<blockquote>
<div><ol class="arabic">
<li><p><code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s are created by <code class="xref py py-meth docutils literal notranslate"><span class="pre">gvar.gvar()</span></code>, individually or in
groups: for example,</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="kn">import</span> <span class="nn">gvar</span> <span class="k">as</span> <span class="nn">gv</span>
<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="mf">1.0</span><span class="p">,</span> <span class="mf">0.1</span><span class="p">),</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="s1">&#39;1.0 +- 0.2&#39;</span><span class="p">),</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="s1">&#39;1.0(4)&#39;</span><span class="p">))</span>
<span class="go">1.00(10) 1.00(20) 1.00(40)</span>

<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">([</span><span class="mf">1.0</span><span class="p">,</span> <span class="mf">1.0</span><span class="p">,</span> <span class="mf">1.0</span><span class="p">],</span> <span class="p">[</span><span class="mf">0.1</span><span class="p">,</span> <span class="mf">0.2</span><span class="p">,</span> <span class="mf">0.41</span><span class="p">]))</span>
<span class="go">[1.00(10) 1.00(20) 1.00(41)]</span>

<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">([</span><span class="s1">&#39;1.0(1)&#39;</span><span class="p">,</span> <span class="s1">&#39;1.0(2)&#39;</span><span class="p">,</span> <span class="s1">&#39;1.00(41)&#39;</span><span class="p">]))</span>
<span class="go">[1.00(10) 1.00(20) 1.00(41)]</span>

<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="nb">dict</span><span class="p">(</span><span class="n">a</span><span class="o">=</span><span class="s1">&#39;1.0(1)&#39;</span><span class="p">,</span> <span class="n">b</span><span class="o">=</span><span class="p">[</span><span class="s1">&#39;1.0(2)&#39;</span><span class="p">,</span> <span class="s1">&#39;1.0(4)&#39;</span><span class="p">])))</span>
<span class="go">{&#39;a&#39;: 1.00(10),&#39;b&#39;: array([1.00(20), 1.00(40)], dtype=object)}</span>
</pre></div>
</div>
<p><code class="xref py py-mod docutils literal notranslate"><span class="pre">gvar</span></code> uses the compact notation 1.234(22) to represent
1.234±0.022 — the digits in parentheses indicate the
uncertainty in the rightmost corresponding digits quoted for the
mean value. Very large (or small) numbers use a notation like
1.234(22)e+10.</p>
</li>
<li><p><code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s describe not only means and standard deviations, but also
statistical correlations between different objects. For example, the
<code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s created by</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="kn">import</span> <span class="nn">gvar</span> <span class="k">as</span> <span class="nn">gv</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">a</span><span class="p">,</span> <span class="n">b</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">],</span> <span class="p">[[</span><span class="mf">0.01</span><span class="p">,</span> <span class="mf">0.01</span><span class="p">],</span> <span class="p">[</span><span class="mf">0.01</span><span class="p">,</span> <span class="mf">0.010001</span><span class="p">]])</span>
<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">)</span>
<span class="go">1.00(10) 1.00(10)</span>
</pre></div>
</div>
<p>both have means of <code class="docutils literal notranslate"><span class="pre">1</span></code> and standard deviations equal to or
very close to <code class="docutils literal notranslate"><span class="pre">0.1</span></code>, but the ratio <code class="docutils literal notranslate"><span class="pre">b/a</span></code> has a
standard deviation that is 100x smaller:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">b</span> <span class="o">/</span> <span class="n">a</span><span class="p">)</span>
<span class="go">1.0000(10)</span>
</pre></div>
</div>
<p>This is because the covariance matrix specified for <code class="docutils literal notranslate"><span class="pre">a</span></code> and <code class="docutils literal notranslate"><span class="pre">b</span></code>
when they were created has large, positive off-diagonal elements:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">gv</span><span class="o">.</span><span class="n">evalcov</span><span class="p">([</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">]))</span>         <span class="c1"># covariance matrix</span>
<span class="go">[[ 0.01      0.01    ]</span>
<span class="go"> [ 0.01      0.010001]]</span>
</pre></div>
</div>
<p>These off-diagonal elements imply that <code class="docutils literal notranslate"><span class="pre">a</span></code> and <code class="docutils literal notranslate"><span class="pre">b</span></code> are strongly
correlated, which means that <code class="docutils literal notranslate"><span class="pre">b/a</span></code> or <code class="docutils literal notranslate"><span class="pre">b-a</span></code> will have
much smaller uncertainties than <code class="docutils literal notranslate"><span class="pre">a</span></code> or <code class="docutils literal notranslate"><span class="pre">b</span></code> separately. The
correlation coefficient for <code class="docutils literal notranslate"><span class="pre">a</span></code> and <code class="docutils literal notranslate"><span class="pre">b</span></code> is 0.99995:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">gv</span><span class="o">.</span><span class="n">evalcorr</span><span class="p">([</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">]))</span>        <span class="c1"># correlation matrix</span>
<span class="go">[[ 1.       0.99995]</span>
<span class="go"> [ 0.99995  1.     ]]</span>
</pre></div>
</div>
</li>
<li><p><code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s can be used in arithmetic expressions or as arguments
to pure-Python functions. The results are also <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s. Covariances
are propagated through these expressions following the usual rules,
(automatically) preserving information about correlations. For
example, the <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s <code class="docutils literal notranslate"><span class="pre">a</span></code> and <code class="docutils literal notranslate"><span class="pre">b</span></code> above could have been created
using the following code:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="kn">import</span> <span class="nn">gvar</span> <span class="k">as</span> <span class="nn">gv</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">a</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mf">0.1</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">b</span> <span class="o">=</span> <span class="n">a</span> <span class="o">+</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mf">0.001</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">)</span>
<span class="go">1.00(10) 1.00(10)</span>

<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">b</span> <span class="o">/</span> <span class="n">a</span><span class="p">)</span>
<span class="go">1.0000(10)</span>

<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">gv</span><span class="o">.</span><span class="n">evalcov</span><span class="p">([</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">]))</span>
<span class="go">[[ 0.01      0.01    ]</span>
<span class="go"> [ 0.01      0.010001]]</span>
</pre></div>
</div>
<p>The correlation is obvious from this code: <code class="docutils literal notranslate"><span class="pre">b</span></code> is equal to <code class="docutils literal notranslate"><span class="pre">a</span></code>
plus a very small correction. From these variables we can
create new variables that are also highly correlated:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="n">x</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="mi">1</span> <span class="o">+</span> <span class="n">a</span> <span class="o">**</span> <span class="mi">2</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">y</span> <span class="o">=</span> <span class="n">b</span> <span class="o">*</span> <span class="n">gv</span><span class="o">.</span><span class="n">cosh</span><span class="p">(</span><span class="n">a</span> <span class="o">/</span> <span class="mi">2</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">y</span> <span class="o">/</span> <span class="n">x</span><span class="p">)</span>
<span class="go">0.69(10) 1.13(14) 1.627(34)</span>

<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span> <span class="n">gv</span><span class="o">.</span><span class="n">evalcov</span><span class="p">([</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">])</span>
<span class="go">[[ 0.01        0.01388174]</span>
<span class="go"> [ 0.01388174  0.01927153]]</span>
</pre></div>
</div>
<p>The <code class="xref py py-mod docutils literal notranslate"><span class="pre">gvar</span></code> module defines versions of the standard Python
functions (<code class="docutils literal notranslate"><span class="pre">sin</span></code>, <code class="docutils literal notranslate"><span class="pre">cos</span></code>, …) that work with <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s. Most any
numeric pure-Python function will work with them as well. Numeric
functions that are compiled in C or other low-level languages
generally do not work with <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s; they should be replaced by
equivalent pure-Python functions if they are needed for <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>-valued
arguments. See the <code class="xref py py-mod docutils literal notranslate"><span class="pre">gvar</span></code> documentation for more
information.</p>
<p>The fact that correlation information is preserved <em>automatically</em>
through arbitrarily complicated arithmetic is what makes
<code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s particularly useful. This is accomplished using <em>automatic
differentiation</em> to compute the derivatives of any <em>derived</em> <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>
with respect to the <em>primary</em> <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s (those defined using
<code class="xref py py-func docutils literal notranslate"><span class="pre">gvar.gvar()</span></code>) from which it was created. As a result, for example,
we need not provide derivatives of fit functions for <a class="reference internal" href="lsqfit.html#module-lsqfit" title="lsqfit: Nonlinear least squares fitting."><code class="xref py py-mod docutils literal notranslate"><span class="pre">lsqfit</span></code></a>
(which are needed for the fit) since they are computed implicitly
by the fitter from the fit function itself. Also
it becomes trivial to build correlations into the priors used
in fits, and to analyze the propagation of errors through
complicated functions of the parameters after the fit.</p>
</li>
<li><p>The uncertainties in derived <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s come from the
uncertainties in the primary <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s from which they were created.
It is easy to create an “error budget” that decomposes the
uncertainty in a derived <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code> into components coming from each
of the primary <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s involved in its creation. For example,</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="kn">import</span> <span class="nn">gvar</span> <span class="k">as</span> <span class="nn">gv</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">a</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="s1">&#39;1.0(1)&#39;</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">b</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="s1">&#39;0.9(2)&#39;</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">x</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="mi">1</span> <span class="o">+</span> <span class="n">a</span> <span class="o">**</span> <span class="mi">2</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">y</span> <span class="o">=</span> <span class="n">b</span> <span class="o">*</span> <span class="n">gv</span><span class="o">.</span><span class="n">cosh</span><span class="p">(</span><span class="n">a</span> <span class="o">/</span> <span class="mi">2</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">outputs</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">x</span><span class="o">=</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="o">=</span><span class="n">y</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">gv</span><span class="o">.</span><span class="n">fmt_values</span><span class="p">(</span><span class="n">outputs</span><span class="p">))</span>
<span class="go">Values:</span>
<span class="go">                  y: 1.01(23)</span>
<span class="go">                  x: 0.69(10)</span>

<span class="gp">&gt;&gt;&gt; </span><span class="n">inputs</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">a</span><span class="o">=</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="o">=</span><span class="n">b</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">gv</span><span class="o">.</span><span class="n">fmt_errorbudget</span><span class="p">(</span><span class="n">outputs</span><span class="o">=</span><span class="n">outputs</span><span class="p">,</span> <span class="n">inputs</span><span class="o">=</span><span class="n">inputs</span><span class="p">))</span>
<span class="go">Partial % Errors:</span>
<span class="go">                   y         x</span>
<span class="go">------------------------------</span>
<span class="go">        a:      2.31     14.43</span>
<span class="go">        b:     22.22      0.00</span>
<span class="go">------------------------------</span>
<span class="go">    total:     22.34     14.43</span>
</pre></div>
</div>
<p>The error budget shows that most of <code class="docutils literal notranslate"><span class="pre">y</span></code>’s 22.34% uncertainty comes
from <code class="docutils literal notranslate"><span class="pre">b</span></code>, with just 2.3% coming from <code class="docutils literal notranslate"><span class="pre">a</span></code>. The total uncertainty is
the sum in quadrature of the two separate uncertainties. The uncertainty
in <code class="docutils literal notranslate"><span class="pre">x</span></code> is entirely from <code class="docutils literal notranslate"><span class="pre">a</span></code>, of course.</p>
</li>
<li><p>Storing <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s in a file for later use is somewhat complicated
because one generally wants to hold onto their correlations as well
as their mean values and standard deviations. One easy way to do
this is to put all of the <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s to be saved into a single
array or dictionary that is saved using function <code class="xref py py-func docutils literal notranslate"><span class="pre">gvar.dump()</span></code>:
for example,  use</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="n">gv</span><span class="o">.</span><span class="n">dump</span><span class="p">([</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">],</span> <span class="s1">&#39;outputfile.p&#39;</span><span class="p">)</span>
</pre></div>
</div>
<p>to save the variables defined above in a file named <code class="docutils literal notranslate"><span class="pre">'outputfile.p'</span></code>.
Loading the file into a Python code later, with <code class="xref py py-func docutils literal notranslate"><span class="pre">gvar.load()</span></code>,
recovers the array with standard deviations and correlations intact:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="n">a</span><span class="p">,</span><span class="n">b</span><span class="p">,</span><span class="n">x</span><span class="p">,</span><span class="n">y</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="s1">&#39;outputfile.p&#39;</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span>
<span class="go">1.00(10) 0.90(20) 0.69(10) 1.01(23)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">y</span> <span class="o">/</span> <span class="n">b</span><span class="p">,</span> <span class="n">gv</span><span class="o">.</span><span class="n">sqrt</span><span class="p">(</span><span class="n">gv</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span> <span class="o">/</span> <span class="n">a</span><span class="p">)</span>
<span class="go">1.128(26) 1(0)</span>
</pre></div>
</div>
<p><code class="xref py py-func docutils literal notranslate"><span class="pre">gvar.dump()</span></code> and <code class="xref py py-func docutils literal notranslate"><span class="pre">gvar.load()</span></code> are similar to the same
functions in Python’s <code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> module, except that they
can deal properly with <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s. They can be used to archive
(possibly nested) containers like dictionaries and lists that
contain <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s (together with other data types), as well as
many other types of object containing <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s. In particular,
they can be used to save the best-fit parameters
from a fit,</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="n">gv</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">fit</span><span class="o">.</span><span class="n">p</span><span class="p">,</span> <span class="s1">&#39;fitpfile.p&#39;</span><span class="p">)</span>
</pre></div>
</div>
<p>or the entire <a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a> object:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="n">gv</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">fit</span><span class="p">,</span> <span class="s1">&#39;fitfile.p&#39;</span><span class="p">)</span>
</pre></div>
</div>
<p>The archived fit preserves all
or most of the fit object’s functionality (depending upon
whether or not the fit function can be pickled): for
example,</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="n">fit</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="s1">&#39;fitfile.p&#39;</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">fit</span><span class="p">)</span>
<span class="go">Least Square Fit:</span>
<span class="go">  chi2/dof [dof] = 0.17 [5]    Q = 0.97    logGBF = 0.65538</span>

<span class="go">Parameters:</span>
<span class="go">              a   0.253 (32)     [  0.50 (50) ]</span>
<span class="go">              b   0.449 (65)     [  0.50 (50) ]</span>

<span class="go">Settings:</span>
<span class="go">  svdcut/n = 1e-12/0    tol = (1e-08*,1e-10,1e-10)    (itns/time = 8/0.0)</span>

<span class="gp">&gt;&gt;&gt; </span><span class="n">p</span> <span class="o">=</span> <span class="n">fit</span><span class="o">.</span><span class="n">p</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">outputs</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">a</span><span class="o">=</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">],</span> <span class="n">b</span><span class="o">=</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;b&#39;</span><span class="p">])</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">outputs</span><span class="p">[</span><span class="s1">&#39;b/a&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">p</span><span class="p">[</span><span class="s1">&#39;b&#39;</span><span class="p">]</span><span class="o">/</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">inputs</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">y</span><span class="o">=</span><span class="n">y</span><span class="p">,</span> <span class="n">prior</span><span class="o">=</span><span class="n">prior</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">gv</span><span class="o">.</span><span class="n">fmt_errorbudget</span><span class="p">(</span><span class="n">outputs</span><span class="p">,</span> <span class="n">inputs</span><span class="p">))</span>
<span class="go">Partial % Errors:</span>
<span class="go">                  a       b/a         b</span>
<span class="go">----------------------------------------</span>
<span class="go">        y:     12.75     16.72     14.30</span>
<span class="go">    prior:      0.92      1.58      1.88</span>
<span class="go">----------------------------------------</span>
<span class="go">    total:     12.78     16.80     14.42</span>
</pre></div>
</div>
</li>
</ol>
</div></blockquote>
<p>There is considerably more information about <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s in the <a class="reference external" href="https://gvar.readthedocs.io/en/latest/">documentation</a>
for module <code class="xref py py-mod docutils literal notranslate"><span class="pre">gvar</span></code>.</p>
</section>
<section id="basic-fits">
<span id="id1"></span><h2>Basic Fits<a class="headerlink" href="#basic-fits" title="Permalink to this heading">¶</a></h2>
<p>A fit analysis typically requires three types of input:</p>
<ol class="arabic simple">
<li><p>fit data <code class="docutils literal notranslate"><span class="pre">x,y</span></code> (or possibly just <code class="docutils literal notranslate"><span class="pre">y</span></code>);</p></li>
<li><p>a function <code class="docutils literal notranslate"><span class="pre">y</span> <span class="pre">=</span> <span class="pre">f(x,p)</span></code> relating
values of <code class="docutils literal notranslate"><span class="pre">y</span></code> to values of <code class="docutils literal notranslate"><span class="pre">x</span></code> and a set of fit parameters <code class="docutils literal notranslate"><span class="pre">p</span></code>;
if there is no <code class="docutils literal notranslate"><span class="pre">x</span></code>, then <code class="docutils literal notranslate"><span class="pre">y</span> <span class="pre">=</span> <span class="pre">f(p)</span></code>;</p></li>
<li><p>some <em>a priori</em> idea about the fit parameters’ values (possibly
quite imprecise — for example, that a
particular parameter is of order 1).</p></li>
</ol>
<p>The point of
the fit is to improve our knowledge of the parameter values, beyond
our <em>a priori</em> impressions, by analyzing the fit data. We now show how
to do this using the <a class="reference internal" href="lsqfit.html#module-lsqfit" title="lsqfit: Nonlinear least squares fitting."><code class="xref py py-mod docutils literal notranslate"><span class="pre">lsqfit</span></code></a> module for a more realistic
problem, one that is
familiar from numerical simulations of quantum chromodynamics (QCD).</p>
<p>We need code for each of the three fit inputs. The fit data in our example
is assembled by the following function:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">import</span> <span class="nn">gvar</span> <span class="k">as</span> <span class="nn">gv</span>

<span class="k">def</span> <span class="nf">make_data</span><span class="p">():</span>
    <span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span>  <span class="mf">5.</span><span class="p">,</span>   <span class="mf">6.</span><span class="p">,</span>   <span class="mf">7.</span><span class="p">,</span>   <span class="mf">8.</span><span class="p">,</span>   <span class="mf">9.</span><span class="p">,</span>  <span class="mf">10.</span><span class="p">,</span>  <span class="mf">12.</span><span class="p">,</span>  <span class="mf">14.</span><span class="p">])</span>
    <span class="n">ymean</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">(</span>
        <span class="p">[</span>  <span class="mf">4.5022829417e-03</span><span class="p">,</span>   <span class="mf">1.8170543788e-03</span><span class="p">,</span>   <span class="mf">7.3618847843e-04</span><span class="p">,</span>
           <span class="mf">2.9872730036e-04</span><span class="p">,</span>   <span class="mf">1.2128831367e-04</span><span class="p">,</span>   <span class="mf">4.9256559129e-05</span><span class="p">,</span>
           <span class="mf">8.1263644483e-06</span><span class="p">,</span>   <span class="mf">1.3415253536e-06</span><span class="p">]</span>
        <span class="p">)</span>
    <span class="n">ycov</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">(</span>
        <span class="p">[[</span> <span class="mf">2.1537808808e-09</span><span class="p">,</span>   <span class="mf">8.8161794696e-10</span><span class="p">,</span>   <span class="mf">3.6237356558e-10</span><span class="p">,</span>
           <span class="mf">1.4921344875e-10</span><span class="p">,</span>   <span class="mf">6.1492842463e-11</span><span class="p">,</span>   <span class="mf">2.5353714617e-11</span><span class="p">,</span>
           <span class="mf">4.3137593878e-12</span><span class="p">,</span>   <span class="mf">7.3465498888e-13</span><span class="p">],</span>
        <span class="p">[</span>  <span class="mf">8.8161794696e-10</span><span class="p">,</span>   <span class="mf">3.6193461816e-10</span><span class="p">,</span>   <span class="mf">1.4921610813e-10</span><span class="p">,</span>
           <span class="mf">6.1633547703e-11</span><span class="p">,</span>   <span class="mf">2.5481570082e-11</span><span class="p">,</span>   <span class="mf">1.0540958082e-11</span><span class="p">,</span>
           <span class="mf">1.8059692534e-12</span><span class="p">,</span>   <span class="mf">3.0985581496e-13</span><span class="p">],</span>
        <span class="p">[</span>  <span class="mf">3.6237356558e-10</span><span class="p">,</span>   <span class="mf">1.4921610813e-10</span><span class="p">,</span>   <span class="mf">6.1710468826e-11</span><span class="p">,</span>
           <span class="mf">2.5572230776e-11</span><span class="p">,</span>   <span class="mf">1.0608148954e-11</span><span class="p">,</span>   <span class="mf">4.4036448945e-12</span><span class="p">,</span>
           <span class="mf">7.6008881270e-13</span><span class="p">,</span>   <span class="mf">1.3146405310e-13</span><span class="p">],</span>
        <span class="p">[</span>  <span class="mf">1.4921344875e-10</span><span class="p">,</span>   <span class="mf">6.1633547703e-11</span><span class="p">,</span>   <span class="mf">2.5572230776e-11</span><span class="p">,</span>
           <span class="mf">1.0632830128e-11</span><span class="p">,</span>   <span class="mf">4.4264622187e-12</span><span class="p">,</span>   <span class="mf">1.8443245513e-12</span><span class="p">,</span>
           <span class="mf">3.2087725578e-13</span><span class="p">,</span>   <span class="mf">5.5986403288e-14</span><span class="p">],</span>
        <span class="p">[</span>  <span class="mf">6.1492842463e-11</span><span class="p">,</span>   <span class="mf">2.5481570082e-11</span><span class="p">,</span>   <span class="mf">1.0608148954e-11</span><span class="p">,</span>
           <span class="mf">4.4264622187e-12</span><span class="p">,</span>   <span class="mf">1.8496194125e-12</span><span class="p">,</span>   <span class="mf">7.7369196122e-13</span><span class="p">,</span>
           <span class="mf">1.3576009069e-13</span><span class="p">,</span>   <span class="mf">2.3914810594e-14</span><span class="p">],</span>
        <span class="p">[</span>  <span class="mf">2.5353714617e-11</span><span class="p">,</span>   <span class="mf">1.0540958082e-11</span><span class="p">,</span>   <span class="mf">4.4036448945e-12</span><span class="p">,</span>
           <span class="mf">1.8443245513e-12</span><span class="p">,</span>   <span class="mf">7.7369196122e-13</span><span class="p">,</span>   <span class="mf">3.2498644263e-13</span><span class="p">,</span>
           <span class="mf">5.7551104112e-14</span><span class="p">,</span>   <span class="mf">1.0244738582e-14</span><span class="p">],</span>
        <span class="p">[</span>  <span class="mf">4.3137593878e-12</span><span class="p">,</span>   <span class="mf">1.8059692534e-12</span><span class="p">,</span>   <span class="mf">7.6008881270e-13</span><span class="p">,</span>
           <span class="mf">3.2087725578e-13</span><span class="p">,</span>   <span class="mf">1.3576009069e-13</span><span class="p">,</span>   <span class="mf">5.7551104112e-14</span><span class="p">,</span>
           <span class="mf">1.0403917951e-14</span><span class="p">,</span>   <span class="mf">1.8976295583e-15</span><span class="p">],</span>
        <span class="p">[</span>  <span class="mf">7.3465498888e-13</span><span class="p">,</span>   <span class="mf">3.0985581496e-13</span><span class="p">,</span>   <span class="mf">1.3146405310e-13</span><span class="p">,</span>
           <span class="mf">5.5986403288e-14</span><span class="p">,</span>   <span class="mf">2.3914810594e-14</span><span class="p">,</span>   <span class="mf">1.0244738582e-14</span><span class="p">,</span>
           <span class="mf">1.8976295583e-15</span><span class="p">,</span>   <span class="mf">3.5672355835e-16</span><span class="p">]]</span>
        <span class="p">)</span>
    <span class="k">return</span> <span class="n">x</span><span class="p">,</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="n">ymean</span><span class="p">,</span> <span class="n">ycov</span><span class="p">)</span>
</pre></div>
</div>
<p>The function call <code class="docutils literal notranslate"><span class="pre">x,y</span> <span class="pre">=</span> <span class="pre">make_data()</span></code> returns eight <code class="docutils literal notranslate"><span class="pre">x[i]</span></code>, and the
corresponding values <code class="docutils literal notranslate"><span class="pre">y[i]</span></code> that we will fit. The <code class="docutils literal notranslate"><span class="pre">y[i]</span></code> are <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s
(Gaussian random variables — see previous section)
built from the mean values in <code class="docutils literal notranslate"><span class="pre">ymean</span></code> and the covariance matrix <code class="docutils literal notranslate"><span class="pre">ycov</span></code>, which shows strong correlations:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">y</span><span class="p">)</span>                      <span class="c1"># fit data</span>
<span class="go">[0.004502(46) 0.001817(19) 0.0007362(79) ... 1.342(19)e-06]</span>

<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">gv</span><span class="o">.</span><span class="n">evalcorr</span><span class="p">(</span><span class="n">y</span><span class="p">))</span>         <span class="c1"># correlation matrix</span>
<span class="go">[[ 1.          0.99853801  0.99397698  ... 0.83814041]</span>
<span class="go"> [ 0.99853801  1.          0.99843828  ... 0.86234032]</span>
<span class="go"> [ 0.99397698  0.99843828  1.          ... 0.88605708]</span>
<span class="go"> ...</span>
<span class="go"> ...</span>
<span class="go"> ...</span>
<span class="go"> [ 0.83814041  0.86234032  0.88605708  ... 1.        ]]</span>
</pre></div>
</div>
<p>These particular data were generated numerically. They come from a
function that is a sum of a very large number of decaying exponentials,</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">a</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">*</span> <span class="n">np</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="o">-</span><span class="n">E</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">*</span> <span class="n">x</span><span class="p">)</span>
</pre></div>
</div>
<p>with coefficients <code class="docutils literal notranslate"><span class="pre">a[i]</span></code> of order 0.5±0.4 and exponents <code class="docutils literal notranslate"><span class="pre">E[i]</span></code> of
order i+1±0.4. The function was evaluated with a particular set of
parameters <code class="docutils literal notranslate"><span class="pre">a[i]</span></code> and <code class="docutils literal notranslate"><span class="pre">E[i]</span></code>, and then noise was added to create
this data. Our challenge is to find estimates for the values of the
parameters <code class="docutils literal notranslate"><span class="pre">a[i]</span></code> and <code class="docutils literal notranslate"><span class="pre">E[i]</span></code> that were used to create the data.</p>
<p>Next we need code for the fit function. Here we know
that a sum of decaying exponentials is appropriate,
and therefore we define the following
Python function:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>

<span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">p</span><span class="p">):</span>         <span class="c1"># function used to fit x, y data</span>
    <span class="n">a</span> <span class="o">=</span> <span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]</span>       <span class="c1"># array of a[i]s</span>
    <span class="n">E</span> <span class="o">=</span> <span class="n">p</span><span class="p">[</span><span class="s1">&#39;E&#39;</span><span class="p">]</span>       <span class="c1"># array of E[i]s</span>
    <span class="k">return</span> <span class="nb">sum</span><span class="p">(</span><span class="n">ai</span> <span class="o">*</span> <span class="n">np</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="o">-</span><span class="n">Ei</span> <span class="o">*</span> <span class="n">x</span><span class="p">)</span> <span class="k">for</span> <span class="n">ai</span><span class="p">,</span> <span class="n">Ei</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">E</span><span class="p">))</span>
</pre></div>
</div>
<p>The fit parameters, <code class="docutils literal notranslate"><span class="pre">a[i]</span></code> and <code class="docutils literal notranslate"><span class="pre">E[i]</span></code>, are stored as arrays in a
dictionary, using labels <code class="docutils literal notranslate"><span class="pre">a</span></code> and <code class="docutils literal notranslate"><span class="pre">E</span></code> to access them. These parameters
are varied in the fit to find the best-fit values <code class="docutils literal notranslate"><span class="pre">p=fit.p</span></code> for which
<code class="docutils literal notranslate"><span class="pre">f(x,fit.p)</span></code> most closely approximates the <code class="docutils literal notranslate"><span class="pre">y</span></code>s in our fit data. The
number of exponentials included in the sum is specified implicitly in this
function, by the lengths of the <code class="docutils literal notranslate"><span class="pre">p['a']</span></code> and <code class="docutils literal notranslate"><span class="pre">p['E']</span></code> arrays. In
principle there are infinitely many exponentials; in practice, given the
finite precision of our data, we will need only a few.</p>
<p>Finally we need to define priors that encapsulate our <em>a priori</em> knowledge
about the fit-parameter values. In practice we almost always have <em>a priori</em>
knowledge about parameters; it is usually impossible to design a fit
function without some sense of the parameter sizes. Given such knowledge
it is important (often essential) to include it in the fit. This is
done by designing priors for the fit, which are probability distributions
for each parameter that describe the <em>a priori</em> uncertainty in that
parameter. As discussed in the previous section, we use objects of type
<code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code> to describe (Gaussian) probability distributions.
Here we know that each <code class="docutils literal notranslate"><span class="pre">a[i]</span></code> is of order
0.5±0.4, while <code class="docutils literal notranslate"><span class="pre">E[i]</span></code> is of order 1+i±0.4. A prior
that represents this information is built using the following code:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">lsqfit</span>
<span class="kn">import</span> <span class="nn">gvar</span> <span class="k">as</span> <span class="nn">gv</span>

<span class="k">def</span> <span class="nf">make_prior</span><span class="p">(</span><span class="n">nexp</span><span class="p">):</span>               <span class="c1"># make priors for fit parameters</span>
    <span class="n">prior</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">BufferDict</span><span class="p">()</span>         <span class="c1"># any dictionary works</span>
    <span class="n">prior</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span><span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="mf">0.5</span><span class="p">,</span> <span class="mf">0.4</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">nexp</span><span class="p">)]</span>
    <span class="n">prior</span><span class="p">[</span><span class="s1">&#39;E&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span><span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="n">i</span><span class="o">+</span><span class="mi">1</span><span class="p">,</span> <span class="mf">0.4</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">nexp</span><span class="p">)]</span>
    <span class="k">return</span> <span class="n">prior</span>
</pre></div>
</div>
<p>where <code class="docutils literal notranslate"><span class="pre">nexp</span></code> is the number of exponential terms that will be used (and
therefore the number of <code class="docutils literal notranslate"><span class="pre">a[i]</span></code>s and <code class="docutils literal notranslate"><span class="pre">E[i]</span></code>s). With <code class="docutils literal notranslate"><span class="pre">nexp=3</span></code>,
for example, we have:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">prior</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">])</span>
<span class="go">[0.50(40) 0.50(40) 0.50(40)]</span>

<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">prior</span><span class="p">[</span><span class="s1">&#39;E&#39;</span><span class="p">])</span>
<span class="go">[1.00(40), 2.00(40), 3.00(40)]</span>
</pre></div>
</div>
<p>We habitually
use dictionary-like class <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.BufferDict</span></code> for the prior because it
allows for a variety of non-Gaussian priors (see <a class="reference internal" href="#positive-parameters"><span class="std std-ref">Positive Parameters; Non-Gaussian Priors</span></a>).
If non-Gaussian priors are unnecessary, <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.BufferDict</span></code> can be replaced by
<code class="docutils literal notranslate"><span class="pre">dict()</span></code> or most any other Python dictionary class.</p>
<p>With fit data, a fit function, and a prior for the fit parameters, we are
finally ready to do the fit, which is now easy:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">fit</span> <span class="o">=</span> <span class="n">lsqfit</span><span class="o">.</span><span class="n">nonlinear_fit</span><span class="p">(</span><span class="n">data</span><span class="o">=</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">),</span> <span class="n">fcn</span><span class="o">=</span><span class="n">f</span><span class="p">,</span> <span class="n">prior</span><span class="o">=</span><span class="n">prior</span><span class="p">)</span>
</pre></div>
</div>
<p>Our complete Python program is, therefore:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">lsqfit</span>
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">import</span> <span class="nn">gvar</span> <span class="k">as</span> <span class="nn">gv</span>

<span class="k">def</span> <span class="nf">main</span><span class="p">():</span>
    <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="o">=</span> <span class="n">make_data</span><span class="p">()</span>              <span class="c1"># collect fit data</span>
    <span class="n">p0</span> <span class="o">=</span> <span class="kc">None</span>                       <span class="c1"># make larger fits go faster (opt.)</span>
    <span class="k">for</span> <span class="n">nexp</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">7</span><span class="p">):</span>
        <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;************************************* nexp =&#39;</span><span class="p">,</span> <span class="n">nexp</span><span class="p">)</span>
        <span class="n">prior</span> <span class="o">=</span> <span class="n">make_prior</span><span class="p">(</span><span class="n">nexp</span><span class="p">)</span>
        <span class="n">fit</span> <span class="o">=</span> <span class="n">lsqfit</span><span class="o">.</span><span class="n">nonlinear_fit</span><span class="p">(</span><span class="n">data</span><span class="o">=</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">),</span> <span class="n">fcn</span><span class="o">=</span><span class="n">f</span><span class="p">,</span> <span class="n">prior</span><span class="o">=</span><span class="n">prior</span><span class="p">,</span> <span class="n">p0</span><span class="o">=</span><span class="n">p0</span><span class="p">)</span>
        <span class="nb">print</span><span class="p">(</span><span class="n">fit</span><span class="p">)</span>                  <span class="c1"># print the fit results</span>
        <span class="k">if</span> <span class="n">nexp</span> <span class="o">&gt;</span> <span class="mi">2</span><span class="p">:</span>
            <span class="n">E</span> <span class="o">=</span> <span class="n">fit</span><span class="o">.</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;E&#39;</span><span class="p">]</span>          <span class="c1"># best-fit parameters</span>
            <span class="n">a</span> <span class="o">=</span> <span class="n">fit</span><span class="o">.</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]</span>
            <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;E1/E0 =&#39;</span><span class="p">,</span> <span class="n">E</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">/</span> <span class="n">E</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="s1">&#39;  E2/E0 =&#39;</span><span class="p">,</span> <span class="n">E</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">/</span> <span class="n">E</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
            <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;a1/a0 =&#39;</span><span class="p">,</span> <span class="n">a</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">/</span> <span class="n">a</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="s1">&#39;  a2/a0 =&#39;</span><span class="p">,</span> <span class="n">a</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">/</span> <span class="n">a</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
        <span class="k">if</span> <span class="n">fit</span><span class="o">.</span><span class="n">chi2</span> <span class="o">/</span> <span class="n">fit</span><span class="o">.</span><span class="n">dof</span> <span class="o">&lt;</span> <span class="mf">1.</span><span class="p">:</span>
            <span class="n">p0</span> <span class="o">=</span> <span class="n">fit</span><span class="o">.</span><span class="n">pmean</span>          <span class="c1"># starting point for next fit (opt.)</span>
        <span class="nb">print</span><span class="p">()</span>

    <span class="c1"># error budget analysis</span>
    <span class="n">outputs</span> <span class="o">=</span> <span class="p">{</span>
        <span class="s1">&#39;E1/E0&#39;</span><span class="p">:</span><span class="n">E</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">/</span><span class="n">E</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="s1">&#39;E2/E0&#39;</span><span class="p">:</span><span class="n">E</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span><span class="o">/</span><span class="n">E</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span>
        <span class="s1">&#39;a1/a0&#39;</span><span class="p">:</span><span class="n">a</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">/</span><span class="n">a</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="s1">&#39;a2/a0&#39;</span><span class="p">:</span><span class="n">a</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span><span class="o">/</span><span class="n">a</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
        <span class="p">}</span>
    <span class="n">inputs</span> <span class="o">=</span> <span class="p">{</span><span class="s1">&#39;E&#39;</span><span class="p">:</span><span class="n">fit</span><span class="o">.</span><span class="n">prior</span><span class="p">[</span><span class="s1">&#39;E&#39;</span><span class="p">],</span> <span class="s1">&#39;a&#39;</span><span class="p">:</span><span class="n">fit</span><span class="o">.</span><span class="n">prior</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">],</span> <span class="s1">&#39;y&#39;</span><span class="p">:</span><span class="n">y</span><span class="p">}</span>
    <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;================= Error Budget Analysis&#39;</span><span class="p">)</span>
    <span class="nb">print</span><span class="p">(</span><span class="n">gv</span><span class="o">.</span><span class="n">fmt_values</span><span class="p">(</span><span class="n">outputs</span><span class="p">))</span>
    <span class="nb">print</span><span class="p">(</span><span class="n">gv</span><span class="o">.</span><span class="n">fmt_errorbudget</span><span class="p">(</span><span class="n">outputs</span><span class="p">,</span><span class="n">inputs</span><span class="p">))</span>

<span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">p</span><span class="p">):</span>                        <span class="c1"># function used to fit x, y data</span>
    <span class="n">a</span> <span class="o">=</span> <span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]</span>                      <span class="c1"># array of a[i]s</span>
    <span class="n">E</span> <span class="o">=</span> <span class="n">p</span><span class="p">[</span><span class="s1">&#39;E&#39;</span><span class="p">]</span>                      <span class="c1"># array of E[i]s</span>
    <span class="k">return</span> <span class="nb">sum</span><span class="p">(</span><span class="n">ai</span> <span class="o">*</span> <span class="n">np</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="o">-</span><span class="n">Ei</span> <span class="o">*</span> <span class="n">x</span><span class="p">)</span> <span class="k">for</span> <span class="n">ai</span><span class="p">,</span> <span class="n">Ei</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">E</span><span class="p">))</span>

<span class="k">def</span> <span class="nf">make_prior</span><span class="p">(</span><span class="n">nexp</span><span class="p">):</span>               <span class="c1"># make priors for fit parameters</span>
    <span class="n">prior</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">BufferDict</span><span class="p">()</span>         <span class="c1"># any dictionary works</span>
    <span class="n">prior</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span><span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="mf">0.5</span><span class="p">,</span> <span class="mf">0.4</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">nexp</span><span class="p">)]</span>
    <span class="n">prior</span><span class="p">[</span><span class="s1">&#39;E&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span><span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="n">i</span><span class="o">+</span><span class="mi">1</span><span class="p">,</span> <span class="mf">0.4</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">nexp</span><span class="p">)]</span>
    <span class="k">return</span> <span class="n">prior</span>

<span class="k">def</span> <span class="nf">make_data</span><span class="p">():</span>                     <span class="c1"># assemble fit data</span>
   <span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span>  <span class="mf">5.</span><span class="p">,</span>   <span class="mf">6.</span><span class="p">,</span>   <span class="mf">7.</span><span class="p">,</span>   <span class="mf">8.</span><span class="p">,</span>   <span class="mf">9.</span><span class="p">,</span>  <span class="mf">10.</span><span class="p">,</span>  <span class="mf">12.</span><span class="p">,</span>  <span class="mf">14.</span><span class="p">])</span>
   <span class="n">ymean</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">(</span>
       <span class="p">[</span>  <span class="mf">4.5022829417e-03</span><span class="p">,</span>   <span class="mf">1.8170543788e-03</span><span class="p">,</span>   <span class="mf">7.3618847843e-04</span><span class="p">,</span>
          <span class="mf">2.9872730036e-04</span><span class="p">,</span>   <span class="mf">1.2128831367e-04</span><span class="p">,</span>   <span class="mf">4.9256559129e-05</span><span class="p">,</span>
          <span class="mf">8.1263644483e-06</span><span class="p">,</span>   <span class="mf">1.3415253536e-06</span><span class="p">]</span>
       <span class="p">)</span>
   <span class="n">ycov</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">(</span>
       <span class="p">[[</span> <span class="mf">2.1537808808e-09</span><span class="p">,</span>   <span class="mf">8.8161794696e-10</span><span class="p">,</span>   <span class="mf">3.6237356558e-10</span><span class="p">,</span>
          <span class="mf">1.4921344875e-10</span><span class="p">,</span>   <span class="mf">6.1492842463e-11</span><span class="p">,</span>   <span class="mf">2.5353714617e-11</span><span class="p">,</span>
          <span class="mf">4.3137593878e-12</span><span class="p">,</span>   <span class="mf">7.3465498888e-13</span><span class="p">],</span>
       <span class="p">[</span>  <span class="mf">8.8161794696e-10</span><span class="p">,</span>   <span class="mf">3.6193461816e-10</span><span class="p">,</span>   <span class="mf">1.4921610813e-10</span><span class="p">,</span>
          <span class="mf">6.1633547703e-11</span><span class="p">,</span>   <span class="mf">2.5481570082e-11</span><span class="p">,</span>   <span class="mf">1.0540958082e-11</span><span class="p">,</span>
          <span class="mf">1.8059692534e-12</span><span class="p">,</span>   <span class="mf">3.0985581496e-13</span><span class="p">],</span>
       <span class="p">[</span>  <span class="mf">3.6237356558e-10</span><span class="p">,</span>   <span class="mf">1.4921610813e-10</span><span class="p">,</span>   <span class="mf">6.1710468826e-11</span><span class="p">,</span>
          <span class="mf">2.5572230776e-11</span><span class="p">,</span>   <span class="mf">1.0608148954e-11</span><span class="p">,</span>   <span class="mf">4.4036448945e-12</span><span class="p">,</span>
          <span class="mf">7.6008881270e-13</span><span class="p">,</span>   <span class="mf">1.3146405310e-13</span><span class="p">],</span>
       <span class="p">[</span>  <span class="mf">1.4921344875e-10</span><span class="p">,</span>   <span class="mf">6.1633547703e-11</span><span class="p">,</span>   <span class="mf">2.5572230776e-11</span><span class="p">,</span>
          <span class="mf">1.0632830128e-11</span><span class="p">,</span>   <span class="mf">4.4264622187e-12</span><span class="p">,</span>   <span class="mf">1.8443245513e-12</span><span class="p">,</span>
          <span class="mf">3.2087725578e-13</span><span class="p">,</span>   <span class="mf">5.5986403288e-14</span><span class="p">],</span>
       <span class="p">[</span>  <span class="mf">6.1492842463e-11</span><span class="p">,</span>   <span class="mf">2.5481570082e-11</span><span class="p">,</span>   <span class="mf">1.0608148954e-11</span><span class="p">,</span>
          <span class="mf">4.4264622187e-12</span><span class="p">,</span>   <span class="mf">1.8496194125e-12</span><span class="p">,</span>   <span class="mf">7.7369196122e-13</span><span class="p">,</span>
          <span class="mf">1.3576009069e-13</span><span class="p">,</span>   <span class="mf">2.3914810594e-14</span><span class="p">],</span>
       <span class="p">[</span>  <span class="mf">2.5353714617e-11</span><span class="p">,</span>   <span class="mf">1.0540958082e-11</span><span class="p">,</span>   <span class="mf">4.4036448945e-12</span><span class="p">,</span>
          <span class="mf">1.8443245513e-12</span><span class="p">,</span>   <span class="mf">7.7369196122e-13</span><span class="p">,</span>   <span class="mf">3.2498644263e-13</span><span class="p">,</span>
          <span class="mf">5.7551104112e-14</span><span class="p">,</span>   <span class="mf">1.0244738582e-14</span><span class="p">],</span>
       <span class="p">[</span>  <span class="mf">4.3137593878e-12</span><span class="p">,</span>   <span class="mf">1.8059692534e-12</span><span class="p">,</span>   <span class="mf">7.6008881270e-13</span><span class="p">,</span>
          <span class="mf">3.2087725578e-13</span><span class="p">,</span>   <span class="mf">1.3576009069e-13</span><span class="p">,</span>   <span class="mf">5.7551104112e-14</span><span class="p">,</span>
          <span class="mf">1.0403917951e-14</span><span class="p">,</span>   <span class="mf">1.8976295583e-15</span><span class="p">],</span>
       <span class="p">[</span>  <span class="mf">7.3465498888e-13</span><span class="p">,</span>   <span class="mf">3.0985581496e-13</span><span class="p">,</span>   <span class="mf">1.3146405310e-13</span><span class="p">,</span>
          <span class="mf">5.5986403288e-14</span><span class="p">,</span>   <span class="mf">2.3914810594e-14</span><span class="p">,</span>   <span class="mf">1.0244738582e-14</span><span class="p">,</span>
          <span class="mf">1.8976295583e-15</span><span class="p">,</span>   <span class="mf">3.5672355835e-16</span><span class="p">]]</span>
       <span class="p">)</span>
   <span class="k">return</span> <span class="n">x</span><span class="p">,</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="n">ymean</span><span class="p">,</span> <span class="n">ycov</span><span class="p">)</span>

<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">&#39;__main__&#39;</span><span class="p">:</span>
    <span class="n">main</span><span class="p">()</span>
</pre></div>
</div>
<p>We are not sure <em>a priori</em> how many exponentials are needed to fit our
data. Consequently
we write our code to try fitting with each of <code class="docutils literal notranslate"><span class="pre">nexp=1,2,3..6</span></code> terms.
(The pieces of the code involving <code class="docutils literal notranslate"><span class="pre">p0</span></code> are optional; they make the
more complicated fits go about 30 times faster since the output from one
fit is used as the starting point for the next fit — see the discussion
of the <code class="docutils literal notranslate"><span class="pre">p0</span></code> parameter for <a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a>.) Running
this code produces the following output, which is reproduced here in some
detail in order to illustrate a variety of features:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="o">*************************************</span> <span class="n">nexp</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">1.2e+03</span> <span class="p">[</span><span class="mi">8</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mi">0</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="o">-</span><span class="mf">4837.2</span>

<span class="n">Parameters</span><span class="p">:</span>
            <span class="n">a</span> <span class="mi">0</span>   <span class="mf">0.00735</span> <span class="p">(</span><span class="mi">59</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  <span class="o">*</span>
            <span class="n">E</span> <span class="mi">0</span>    <span class="mf">1.1372</span> <span class="p">(</span><span class="mi">49</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">1.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">1</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">12</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="o">*************************************</span> <span class="n">nexp</span> <span class="o">=</span> <span class="mi">2</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">2.2</span> <span class="p">[</span><span class="mi">8</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.024</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">111.69</span>

<span class="n">Parameters</span><span class="p">:</span>
            <span class="n">a</span> <span class="mi">0</span>    <span class="mf">0.4024</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>    <span class="mf">0.4471</span> <span class="p">(</span><span class="mi">46</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
            <span class="n">E</span> <span class="mi">0</span>   <span class="mf">0.90104</span> <span class="p">(</span><span class="mi">51</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">1.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>    <span class="mf">1.8282</span> <span class="p">(</span><span class="mi">14</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">2.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">1</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">9</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="o">*************************************</span> <span class="n">nexp</span> <span class="o">=</span> <span class="mi">3</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.63</span> <span class="p">[</span><span class="mi">8</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.76</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">116.29</span>

<span class="n">Parameters</span><span class="p">:</span>
            <span class="n">a</span> <span class="mi">0</span>    <span class="mf">0.4019</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>     <span class="mf">0.406</span> <span class="p">(</span><span class="mi">14</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">2</span>      <span class="mf">0.61</span> <span class="p">(</span><span class="mi">36</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
            <span class="n">E</span> <span class="mi">0</span>   <span class="mf">0.90039</span> <span class="p">(</span><span class="mi">54</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">1.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>    <span class="mf">1.8026</span> <span class="p">(</span><span class="mi">82</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">2.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">2</span>      <span class="mf">2.83</span> <span class="p">(</span><span class="mi">19</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">3.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">1</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">28</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="n">E1</span><span class="o">/</span><span class="n">E0</span> <span class="o">=</span> <span class="mf">2.0020</span><span class="p">(</span><span class="mi">86</span><span class="p">)</span>   <span class="n">E2</span><span class="o">/</span><span class="n">E0</span> <span class="o">=</span> <span class="mf">3.14</span><span class="p">(</span><span class="mi">21</span><span class="p">)</span>
<span class="n">a1</span><span class="o">/</span><span class="n">a0</span> <span class="o">=</span> <span class="mf">1.011</span><span class="p">(</span><span class="mi">32</span><span class="p">)</span>   <span class="n">a2</span><span class="o">/</span><span class="n">a0</span> <span class="o">=</span> <span class="mf">1.52</span><span class="p">(</span><span class="mi">89</span><span class="p">)</span>
<span class="o">*************************************</span> <span class="n">nexp</span> <span class="o">=</span> <span class="mi">4</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.63</span> <span class="p">[</span><span class="mi">8</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.76</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">116.3</span>

<span class="n">Parameters</span><span class="p">:</span>
            <span class="n">a</span> <span class="mi">0</span>    <span class="mf">0.4019</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>     <span class="mf">0.406</span> <span class="p">(</span><span class="mi">14</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">2</span>      <span class="mf">0.61</span> <span class="p">(</span><span class="mi">36</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">3</span>      <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
            <span class="n">E</span> <span class="mi">0</span>   <span class="mf">0.90039</span> <span class="p">(</span><span class="mi">54</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">1.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>    <span class="mf">1.8026</span> <span class="p">(</span><span class="mi">82</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">2.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">2</span>      <span class="mf">2.83</span> <span class="p">(</span><span class="mi">19</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">3.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">3</span>      <span class="mf">4.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">4.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">1</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">9</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="n">E1</span><span class="o">/</span><span class="n">E0</span> <span class="o">=</span> <span class="mf">2.0020</span><span class="p">(</span><span class="mi">86</span><span class="p">)</span>   <span class="n">E2</span><span class="o">/</span><span class="n">E0</span> <span class="o">=</span> <span class="mf">3.14</span><span class="p">(</span><span class="mi">21</span><span class="p">)</span>
<span class="n">a1</span><span class="o">/</span><span class="n">a0</span> <span class="o">=</span> <span class="mf">1.011</span><span class="p">(</span><span class="mi">32</span><span class="p">)</span>   <span class="n">a2</span><span class="o">/</span><span class="n">a0</span> <span class="o">=</span> <span class="mf">1.52</span><span class="p">(</span><span class="mi">89</span><span class="p">)</span>
<span class="o">*************************************</span> <span class="n">nexp</span> <span class="o">=</span> <span class="mi">5</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.63</span> <span class="p">[</span><span class="mi">8</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.76</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">116.3</span>

<span class="n">Parameters</span><span class="p">:</span>
            <span class="n">a</span> <span class="mi">0</span>    <span class="mf">0.4019</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>     <span class="mf">0.406</span> <span class="p">(</span><span class="mi">14</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">2</span>      <span class="mf">0.61</span> <span class="p">(</span><span class="mi">36</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">3</span>      <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">4</span>      <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
            <span class="n">E</span> <span class="mi">0</span>   <span class="mf">0.90039</span> <span class="p">(</span><span class="mi">54</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">1.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>    <span class="mf">1.8026</span> <span class="p">(</span><span class="mi">82</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">2.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">2</span>      <span class="mf">2.83</span> <span class="p">(</span><span class="mi">19</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">3.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">3</span>      <span class="mf">4.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">4.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">4</span>      <span class="mf">5.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">5.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">1</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">4</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="n">E1</span><span class="o">/</span><span class="n">E0</span> <span class="o">=</span> <span class="mf">2.0020</span><span class="p">(</span><span class="mi">86</span><span class="p">)</span>   <span class="n">E2</span><span class="o">/</span><span class="n">E0</span> <span class="o">=</span> <span class="mf">3.14</span><span class="p">(</span><span class="mi">21</span><span class="p">)</span>
<span class="n">a1</span><span class="o">/</span><span class="n">a0</span> <span class="o">=</span> <span class="mf">1.011</span><span class="p">(</span><span class="mi">32</span><span class="p">)</span>   <span class="n">a2</span><span class="o">/</span><span class="n">a0</span> <span class="o">=</span> <span class="mf">1.52</span><span class="p">(</span><span class="mi">89</span><span class="p">)</span>
<span class="o">*************************************</span> <span class="n">nexp</span> <span class="o">=</span> <span class="mi">6</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.63</span> <span class="p">[</span><span class="mi">8</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.76</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">116.3</span>

<span class="n">Parameters</span><span class="p">:</span>
            <span class="n">a</span> <span class="mi">0</span>    <span class="mf">0.4019</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>     <span class="mf">0.406</span> <span class="p">(</span><span class="mi">14</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">2</span>      <span class="mf">0.61</span> <span class="p">(</span><span class="mi">36</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">3</span>      <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">4</span>      <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">5</span>      <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
            <span class="n">E</span> <span class="mi">0</span>   <span class="mf">0.90039</span> <span class="p">(</span><span class="mi">54</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">1.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>    <span class="mf">1.8026</span> <span class="p">(</span><span class="mi">82</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">2.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">2</span>      <span class="mf">2.83</span> <span class="p">(</span><span class="mi">19</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">3.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">3</span>      <span class="mf">4.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">4.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">4</span>      <span class="mf">5.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">5.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">5</span>      <span class="mf">6.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">6.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">1</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">2</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="n">E1</span><span class="o">/</span><span class="n">E0</span> <span class="o">=</span> <span class="mf">2.0020</span><span class="p">(</span><span class="mi">86</span><span class="p">)</span>   <span class="n">E2</span><span class="o">/</span><span class="n">E0</span> <span class="o">=</span> <span class="mf">3.14</span><span class="p">(</span><span class="mi">21</span><span class="p">)</span>
<span class="n">a1</span><span class="o">/</span><span class="n">a0</span> <span class="o">=</span> <span class="mf">1.011</span><span class="p">(</span><span class="mi">32</span><span class="p">)</span>   <span class="n">a2</span><span class="o">/</span><span class="n">a0</span> <span class="o">=</span> <span class="mf">1.52</span><span class="p">(</span><span class="mi">89</span><span class="p">)</span>
<span class="o">=================</span> <span class="n">Error</span> <span class="n">Budget</span> <span class="n">Analysis</span>
<span class="n">Values</span><span class="p">:</span>
              <span class="n">E1</span><span class="o">/</span><span class="n">E0</span><span class="p">:</span> <span class="mf">2.0020</span><span class="p">(</span><span class="mi">86</span><span class="p">)</span>          
              <span class="n">E2</span><span class="o">/</span><span class="n">E0</span><span class="p">:</span> <span class="mf">3.14</span><span class="p">(</span><span class="mi">21</span><span class="p">)</span>            
              <span class="n">a1</span><span class="o">/</span><span class="n">a0</span><span class="p">:</span> <span class="mf">1.011</span><span class="p">(</span><span class="mi">32</span><span class="p">)</span>           
              <span class="n">a2</span><span class="o">/</span><span class="n">a0</span><span class="p">:</span> <span class="mf">1.52</span><span class="p">(</span><span class="mi">89</span><span class="p">)</span>            

<span class="n">Partial</span> <span class="o">%</span> <span class="n">Errors</span><span class="p">:</span>
               <span class="n">E1</span><span class="o">/</span><span class="n">E0</span>     <span class="n">E2</span><span class="o">/</span><span class="n">E0</span>     <span class="n">a1</span><span class="o">/</span><span class="n">a0</span>     <span class="n">a2</span><span class="o">/</span><span class="n">a0</span>
<span class="o">--------------------------------------------------</span>
        <span class="n">a</span><span class="p">:</span>      <span class="mf">0.07</span>      <span class="mf">5.47</span>      <span class="mf">0.82</span>     <span class="mf">52.75</span>
        <span class="n">E</span><span class="p">:</span>      <span class="mf">0.12</span>      <span class="mf">3.23</span>      <span class="mf">1.04</span>     <span class="mf">25.36</span>
        <span class="n">y</span><span class="p">:</span>      <span class="mf">0.40</span>      <span class="mf">2.08</span>      <span class="mf">2.78</span>      <span class="mf">5.24</span>
<span class="o">--------------------------------------------------</span>
    <span class="n">total</span><span class="p">:</span>      <span class="mf">0.43</span>      <span class="mf">6.72</span>      <span class="mf">3.15</span>     <span class="mf">58.78</span>

</pre></div>
</div>
<p>There are several things to notice here:</p>
<blockquote>
<div><ul>
<li><p>Clearly two exponentials (<code class="docutils literal notranslate"><span class="pre">nexp=2</span></code>) are not sufficient. The <code class="docutils literal notranslate"><span class="pre">chi**2</span></code>
per degree of freedom (<code class="docutils literal notranslate"><span class="pre">chi2/dof</span></code>) is significantly larger than one.
The
<code class="docutils literal notranslate"><span class="pre">chi**2</span></code> improves substantially for <code class="docutils literal notranslate"><span class="pre">nexp=3</span></code> exponentials,
and there is
essentially no change when further exponentials are added.</p></li>
<li><p>The best-fit values for each parameter are listed for each of the
fits, together with the prior values (in brackets, on the right).
Values for each <code class="docutils literal notranslate"><span class="pre">a[i]</span></code> and <code class="docutils literal notranslate"><span class="pre">E[i]</span></code> are listed in order, starting at
the points indicated by the labels <code class="docutils literal notranslate"><span class="pre">a</span></code> and <code class="docutils literal notranslate"><span class="pre">E</span></code>. Asterisks are
printed at the end of the line if the mean best-fit value differs from
the prior’s mean by more than one standard deviation (see <code class="docutils literal notranslate"><span class="pre">nexp=1</span></code>);
the number
of asterisks, up to a maximum of 5, indicates how many standard
deviations the difference is. Differences of one or two standard
deviations are not uncommon; larger differences could indicate a
problem with the data, prior, or fit function.</p>
<p>Once the fit converges, the best-fit values for the various parameters
agree well — that is to within their errors, approximately — with
the exact values, which we know since we made the data. For
example, <code class="docutils literal notranslate"><span class="pre">a</span></code> and <code class="docutils literal notranslate"><span class="pre">E</span></code> for the first exponential are 0.402(4)
and 0.9004(5), respectively, from the fit, while the exact answers
are 0.4 and 0.9; and we get 0.406(14) and 1.803(8) for
the second exponential where the exact values are 0.4 and 1.8.</p>
</li>
<li><p>Note in the fit with <code class="docutils literal notranslate"><span class="pre">nexp=4</span></code> how the mean and standard deviation for
the parameters governing the fourth (and last) exponential are
identical to the values in the corresponding priors: 0.50(40) from
the fit for <code class="docutils literal notranslate"><span class="pre">a</span></code> and 4.0(4) for <code class="docutils literal notranslate"><span class="pre">E</span></code>. This tells us that our fit
data have no information to add to what we knew <em>a priori</em>
about these parameters — there isn’t enough data and what we have
isn’t accurate enough.</p>
<p>This situation remains true of further terms as they are added in
the <code class="docutils literal notranslate"><span class="pre">nexp=5</span></code> and later fits. This is why the fit results stop
changing once we have <code class="docutils literal notranslate"><span class="pre">nexp=3</span></code> exponentials. There is no point in
including further exponentials, beyond the need to verify that the fit
has indeed converged. Note that the underlying function from
which the data came had 100 exponential terms.</p>
</li>
<li><p>The last fit includes <code class="docutils literal notranslate"><span class="pre">nexp=6</span></code> exponentials and therefore has 12 parameters. This is in a fit to 8 <code class="docutils literal notranslate"><span class="pre">y</span></code>s. Old-fashioned fits,
without
priors, are impossible when the number of parameters exceeds the number
of data points. That is clearly not the case here, where the number of
terms and parameters can be made arbitrarily large, eventually (after
<code class="docutils literal notranslate"><span class="pre">nexp=3</span></code> terms) with no effect at all on the results.</p>
<p>The reason is that the prior that we include for each new parameter
is, in effect, a new piece of data (equal to
the mean and standard deviation of
the <em>a priori</em> expectation for that parameter). Each prior
leads to a new term
in the <code class="docutils literal notranslate"><span class="pre">chi**2</span></code> function; we are fitting both the data and our <em>a
priori</em> expectations for the parameters. So in the <code class="docutils literal notranslate"><span class="pre">nexp=6</span></code> fit,
for example, we actually have 20 pieces of data to fit: the 8 <code class="docutils literal notranslate"><span class="pre">y</span></code>s
plus the 12 prior values for the 12 parameters.</p>
<p>That priors are additional fit data becomes obvious if we rewrite
our fit function as</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">g</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">p</span><span class="p">):</span>         <span class="c1"># function used to fit x, y data</span>
    <span class="n">a</span> <span class="o">=</span> <span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]</span>       <span class="c1"># array of a[i]s</span>
    <span class="n">E</span> <span class="o">=</span> <span class="n">p</span><span class="p">[</span><span class="s1">&#39;E&#39;</span><span class="p">]</span>       <span class="c1"># array of E[i]s</span>
    <span class="k">return</span> <span class="nb">dict</span><span class="p">(</span>
       <span class="n">y</span><span class="o">=</span><span class="nb">sum</span><span class="p">(</span><span class="n">ai</span> <span class="o">*</span> <span class="n">np</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="o">-</span><span class="n">Ei</span> <span class="o">*</span> <span class="n">x</span><span class="p">)</span> <span class="k">for</span> <span class="n">ai</span><span class="p">,</span> <span class="n">Ei</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">E</span><span class="p">)),</span>
       <span class="n">a</span><span class="o">=</span><span class="n">a</span><span class="p">,</span>
       <span class="n">E</span><span class="o">=</span><span class="n">E</span><span class="p">,</span>
       <span class="p">)</span>
</pre></div>
</div>
<p>and add the following right after the loop in the <code class="docutils literal notranslate"><span class="pre">main()</span></code> function:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="nb">print</span><span class="p">(</span><span class="s1">&#39;************************************* nexp =&#39;</span><span class="p">,</span> <span class="n">nexp</span><span class="p">,</span> <span class="s1">&#39;(fit g(x,p))&#39;</span><span class="p">)</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="nb">dict</span><span class="p">(</span><span class="n">y</span><span class="o">=</span><span class="n">y</span><span class="p">,</span> <span class="n">a</span><span class="o">=</span><span class="n">prior</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">],</span> <span class="n">E</span><span class="o">=</span><span class="n">prior</span><span class="p">[</span><span class="s1">&#39;E&#39;</span><span class="p">]))</span>
<span class="n">gfit</span> <span class="o">=</span> <span class="n">lsqfit</span><span class="o">.</span><span class="n">nonlinear_fit</span><span class="p">(</span><span class="n">data</span><span class="o">=</span><span class="n">data</span><span class="p">,</span> <span class="n">fcn</span><span class="o">=</span><span class="n">g</span><span class="p">,</span> <span class="n">prior</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">p0</span><span class="o">=</span><span class="n">p0</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">gfit</span><span class="p">)</span>
<span class="nb">print</span><span class="p">()</span>
</pre></div>
</div>
<p>This gives exactly the same results as the last fit from the loop,
but now with the prior explicitly built into the fit function and data.
This way of implementing priors, although equivalent, is generally less
convenient.</p>
<p>The effective number of degrees of freedom (<code class="docutils literal notranslate"><span class="pre">dof</span></code> in the output
above) is the number of pieces of data minus the number of fit
parameters, or 20-12=8 in this last case. With priors for every
parameter, the number of degrees of freedom is always equal to the
number of <code class="docutils literal notranslate"><span class="pre">y</span></code>s, irrespective of how many fit parameters there are.</p>
</li>
<li><p>The Gaussian Bayes Factor (whose logarithm is
<code class="docutils literal notranslate"><span class="pre">logGBF</span></code> in the output) is a measure of the likelihood that the actual
data being fit could have come from a theory with the prior and
fit function used in the
fit. The larger this number, the more likely it is that the prior/fit-function
and data
could be related. Here it grows dramatically from the first fit
(<code class="docutils literal notranslate"><span class="pre">nexp=1</span></code>) but then stops changing after <code class="docutils literal notranslate"><span class="pre">nexp=3</span></code>. The
implication is that these data are much more likely to have come from a
theory with <code class="docutils literal notranslate"><span class="pre">nexp&gt;=3</span></code> than one with <code class="docutils literal notranslate"><span class="pre">nexp=1</span></code>.</p></li>
<li><p>In the code, results for each fit are captured in a Python object
<code class="docutils literal notranslate"><span class="pre">fit</span></code>, which is of type <a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a>. A summary of the
fit information is obtained by printing <code class="docutils literal notranslate"><span class="pre">fit</span></code>. Also the best-fit
results for each fit parameter can be accessed through <code class="docutils literal notranslate"><span class="pre">fit.p</span></code>, as is
done here to calculate various ratios of parameters.</p>
<p>The errors in these ratios automatically account for any
correlations in the statistical errors for different parameters. This
is evident in the ratio <code class="docutils literal notranslate"><span class="pre">a1/a0</span></code>, which would be 1.010(35) if
there was no statistical correlation between our estimates for <code class="docutils literal notranslate"><span class="pre">a1</span></code>
and <code class="docutils literal notranslate"><span class="pre">a0</span></code>, but in fact is 1.010(31) in this fit. The modest (positive)
correlation is clear from the correlation matrix:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">gv</span><span class="o">.</span><span class="n">evalcorr</span><span class="p">(</span><span class="n">fit</span><span class="o">.</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">][:</span><span class="mi">2</span><span class="p">]))</span>
<span class="go">[[ 1.          0.36353303]</span>
<span class="go"> [ 0.36353303  1.        ]]</span>
</pre></div>
</div>
</li>
<li><p>After the last fit, the code uses function <code class="docutils literal notranslate"><span class="pre">gvar.fmt_errorbudget</span></code>
to create an error budget. This requires dictionaries of
fit inputs and outputs, and uses the dictionary keys to label
columns and rows, respectively, in the error budget table. The
table shows, for example, that the 0.43% uncertainty in <code class="docutils literal notranslate"><span class="pre">E1/E0</span></code>
comes mostly from the fit data (0.40%), with small contributions
from the uncertainties in the priors for <code class="docutils literal notranslate"><span class="pre">a</span></code> and <code class="docutils literal notranslate"><span class="pre">E</span></code> (0.07%
and 0.12%, respectively). The total uncertainty is the sum in
quadrature of these errors. This breakdown suggests that reducing the
errors in <code class="docutils literal notranslate"><span class="pre">y</span></code> by 25% might reduce the error in <code class="docutils literal notranslate"><span class="pre">E1/E0</span></code>
to around 0.3% (and it does). The uncertainty in <code class="docutils literal notranslate"><span class="pre">E2/E0</span></code>, on the other
hand, comes mostly from the priors and is less likely to improve (it
doesn’t).</p></li>
</ul>
</div></blockquote>
<p>Finally we inspect the fit’s quality point by point. The input data are
compared with results from the fit function, evaluated with the best-fit
parameters, in the following table (obtained in the code by printing the
output from <code class="docutils literal notranslate"><span class="pre">fit.format(maxline=True)</span></code>):</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">Fit</span><span class="p">:</span>
     <span class="n">x</span><span class="p">[</span><span class="n">k</span><span class="p">]</span>               <span class="n">y</span><span class="p">[</span><span class="n">k</span><span class="p">]</span>          <span class="n">f</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="n">k</span><span class="p">],</span><span class="n">p</span><span class="p">)</span>
<span class="o">-----------------------------------------------</span>
        <span class="mi">5</span>      <span class="mf">0.004502</span> <span class="p">(</span><span class="mi">46</span><span class="p">)</span>      <span class="mf">0.004506</span> <span class="p">(</span><span class="mi">46</span><span class="p">)</span>
        <span class="mi">6</span>      <span class="mf">0.001817</span> <span class="p">(</span><span class="mi">19</span><span class="p">)</span>      <span class="mf">0.001819</span> <span class="p">(</span><span class="mi">19</span><span class="p">)</span>
        <span class="mi">7</span>     <span class="mf">0.0007362</span> <span class="p">(</span><span class="mi">79</span><span class="p">)</span>     <span class="mf">0.0007373</span> <span class="p">(</span><span class="mi">78</span><span class="p">)</span>
        <span class="mi">8</span>     <span class="mf">0.0002987</span> <span class="p">(</span><span class="mi">33</span><span class="p">)</span>     <span class="mf">0.0002993</span> <span class="p">(</span><span class="mi">32</span><span class="p">)</span>
        <span class="mi">9</span>     <span class="mf">0.0001213</span> <span class="p">(</span><span class="mi">14</span><span class="p">)</span>     <span class="mf">0.0001216</span> <span class="p">(</span><span class="mi">13</span><span class="p">)</span>
       <span class="mi">10</span>    <span class="mf">0.00004926</span> <span class="p">(</span><span class="mi">57</span><span class="p">)</span>    <span class="mf">0.00004941</span> <span class="p">(</span><span class="mi">56</span><span class="p">)</span>
       <span class="mi">12</span>       <span class="mf">8.13</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span><span class="n">e</span><span class="o">-</span><span class="mi">06</span>      <span class="mf">8.160</span><span class="p">(</span><span class="mi">96</span><span class="p">)</span><span class="n">e</span><span class="o">-</span><span class="mi">06</span>
       <span class="mi">14</span>      <span class="mf">1.342</span><span class="p">(</span><span class="mi">19</span><span class="p">)</span><span class="n">e</span><span class="o">-</span><span class="mi">06</span>      <span class="mf">1.348</span><span class="p">(</span><span class="mi">17</span><span class="p">)</span><span class="n">e</span><span class="o">-</span><span class="mi">06</span>
</pre></div>
</div>
<p>The fit is excellent over the entire three orders of magnitude. This
information is presented again in the following plot, which shows the ratio
<code class="docutils literal notranslate"><span class="pre">y/f(x,p)</span></code>, as a function of <code class="docutils literal notranslate"><span class="pre">x</span></code>, using the best-fit parameters <code class="docutils literal notranslate"><span class="pre">p</span></code>.
The correct result for this ratio, of course, is one. The smooth variation
in the data — smooth compared with the size of the statistical-error bars
— is an indication of the statistical correlations between individual
<code class="docutils literal notranslate"><span class="pre">y</span></code>s.</p>
<a class="reference internal image-reference" href="_images/eg1.png"><img alt="_images/eg1.png" src="_images/eg1.png" style="width: 70%;" /></a>
<p>This particular plot was made using the <code class="xref py py-mod docutils literal notranslate"><span class="pre">matplotlib</span></code> module, with the
following code added to the end of <code class="docutils literal notranslate"><span class="pre">main()</span></code> (outside the loop):</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
<span class="n">ratio</span> <span class="o">=</span> <span class="n">y</span> <span class="o">/</span> <span class="n">f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">fit</span><span class="o">.</span><span class="n">pmean</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">xlim</span><span class="p">(</span><span class="mi">4</span><span class="p">,</span> <span class="mi">15</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">ylim</span><span class="p">(</span><span class="mf">0.95</span><span class="p">,</span> <span class="mf">1.05</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">xlabel</span><span class="p">(</span><span class="s1">&#39;x&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">ylabel</span><span class="p">(</span><span class="s1">&#39;y / f(x,p)&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">errorbar</span><span class="p">(</span><span class="n">x</span><span class="o">=</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="o">=</span><span class="n">gv</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">ratio</span><span class="p">),</span> <span class="n">yerr</span><span class="o">=</span><span class="n">gv</span><span class="o">.</span><span class="n">sdev</span><span class="p">(</span><span class="n">ratio</span><span class="p">),</span> <span class="n">fmt</span><span class="o">=</span><span class="s1">&#39;ob&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">([</span><span class="mf">4.0</span><span class="p">,</span> <span class="mf">21.0</span><span class="p">],</span> <span class="p">[</span><span class="mf">1.0</span><span class="p">,</span> <span class="mf">1.0</span><span class="p">],</span> <span class="s1">&#39;b:&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
</pre></div>
</div>
</section>
<section id="chained-fits-large-data-sets">
<h2>Chained Fits; Large Data Sets<a class="headerlink" href="#chained-fits-large-data-sets" title="Permalink to this heading">¶</a></h2>
<p>The priors in a fit represent knowledge that we have about the parameters
before we do the fit. This knowledge might come from theoretical
considerations or experiment. Or it might come from another fit. Here we
look at two examples that exploit the possibility of chaining fits, where
the output of one fit is an input (the prior) to another.</p>
<p>Imagine first that we want to add new information to that extracted from the
fit in the previous section. For example, we might learn from some other
source that the ratio of amplitudes <code class="docutils literal notranslate"><span class="pre">a[1]/a[0]</span></code> equals 1±1e-5. The challenge
is to combine this new information with information extracted from the fit
above without rerunning that fit. (We assume it is not possible to rerun.)</p>
<p>We can combine the new data with the old fit results by creating a new
fit that uses the best-fit parameters, <code class="docutils literal notranslate"><span class="pre">fit.p</span></code>, from the old fit as its
prior. To try this out, we modify
the <code class="docutils literal notranslate"><span class="pre">main()</span></code> function in the previous section, adding the new fit at the
end:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">main</span><span class="p">():</span>
    <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="o">=</span> <span class="n">make_data</span><span class="p">()</span>              <span class="c1"># collect fit data</span>
    <span class="n">p0</span> <span class="o">=</span> <span class="kc">None</span>                       <span class="c1"># make larger fits go faster (opt.)</span>
    <span class="k">for</span> <span class="n">nexp</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">5</span><span class="p">):</span>
        <span class="n">prior</span> <span class="o">=</span> <span class="n">make_prior</span><span class="p">(</span><span class="n">nexp</span><span class="p">)</span>
        <span class="n">fit</span> <span class="o">=</span> <span class="n">lsqfit</span><span class="o">.</span><span class="n">nonlinear_fit</span><span class="p">(</span><span class="n">data</span><span class="o">=</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">),</span> <span class="n">fcn</span><span class="o">=</span><span class="n">fcn</span><span class="p">,</span> <span class="n">prior</span><span class="o">=</span><span class="n">prior</span><span class="p">,</span> <span class="n">p0</span><span class="o">=</span><span class="n">p0</span><span class="p">)</span>
        <span class="k">if</span> <span class="n">fit</span><span class="o">.</span><span class="n">chi2</span> <span class="o">/</span> <span class="n">fit</span><span class="o">.</span><span class="n">dof</span> <span class="o">&lt;</span> <span class="mf">1.</span><span class="p">:</span>
            <span class="n">p0</span> <span class="o">=</span> <span class="n">fit</span><span class="o">.</span><span class="n">pmean</span>          <span class="c1"># starting point for next fit (opt.)</span>

    <span class="c1"># print nexp=4 fit results</span>
    <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;--------------------- original fit&#39;</span><span class="p">)</span>
    <span class="nb">print</span><span class="p">(</span><span class="n">fit</span><span class="p">)</span>
    <span class="n">E</span> <span class="o">=</span> <span class="n">fit</span><span class="o">.</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;E&#39;</span><span class="p">]</span>                  <span class="c1"># best-fit parameters</span>
    <span class="n">a</span> <span class="o">=</span> <span class="n">fit</span><span class="o">.</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]</span>
    <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;E1/E0 =&#39;</span><span class="p">,</span> <span class="n">E</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">/</span> <span class="n">E</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="s1">&#39;  E2/E0 =&#39;</span><span class="p">,</span> <span class="n">E</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">/</span> <span class="n">E</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
    <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;a1/a0 =&#39;</span><span class="p">,</span> <span class="n">a</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">/</span> <span class="n">a</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="s1">&#39;  a2/a0 =&#39;</span><span class="p">,</span> <span class="n">a</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">/</span> <span class="n">a</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>

    <span class="c1"># new fit adds new data about a[1] / a[0]</span>
    <span class="k">def</span> <span class="nf">ratio</span><span class="p">(</span><span class="n">p</span><span class="p">):</span>                   <span class="c1"># new fit function</span>
        <span class="n">a</span> <span class="o">=</span> <span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]</span>
        <span class="k">return</span> <span class="n">a</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">/</span> <span class="n">a</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>

    <span class="n">prior</span> <span class="o">=</span> <span class="n">fit</span><span class="o">.</span><span class="n">p</span>                   <span class="c1"># prior = best-fit parameters from nexp=4 fit</span>
    <span class="n">data</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mf">1e-5</span><span class="p">)</span>         <span class="c1"># new data for the ratio</span>

    <span class="n">newfit</span> <span class="o">=</span> <span class="n">lsqfit</span><span class="o">.</span><span class="n">nonlinear_fit</span><span class="p">(</span><span class="n">data</span><span class="o">=</span><span class="n">data</span><span class="p">,</span> <span class="n">fcn</span><span class="o">=</span><span class="n">ratio</span><span class="p">,</span> <span class="n">prior</span><span class="o">=</span><span class="n">prior</span><span class="p">)</span>
    <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;</span><span class="se">\n</span><span class="s1">--------------------- new fit to extra information&#39;</span><span class="p">)</span>
    <span class="nb">print</span><span class="p">(</span><span class="n">newfit</span><span class="p">)</span>
    <span class="n">E</span> <span class="o">=</span> <span class="n">newfit</span><span class="o">.</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;E&#39;</span><span class="p">]</span>
    <span class="n">a</span> <span class="o">=</span> <span class="n">newfit</span><span class="o">.</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]</span>
    <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;E1/E0 =&#39;</span><span class="p">,</span> <span class="n">E</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">/</span> <span class="n">E</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="s1">&#39;  E2/E0 =&#39;</span><span class="p">,</span> <span class="n">E</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">/</span> <span class="n">E</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
    <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;a1/a0 =&#39;</span><span class="p">,</span> <span class="n">a</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">/</span> <span class="n">a</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="s1">&#39;  a2/a0 =&#39;</span><span class="p">,</span> <span class="n">a</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">/</span> <span class="n">a</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
</pre></div>
</div>
<p>The results of the new fit (to one piece of new data) are at the end of the
output:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="o">---------------------</span> <span class="n">original</span> <span class="n">fit</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.63</span> <span class="p">[</span><span class="mi">8</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.76</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">116.3</span>

<span class="n">Parameters</span><span class="p">:</span>
            <span class="n">a</span> <span class="mi">0</span>    <span class="mf">0.4019</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>     <span class="mf">0.406</span> <span class="p">(</span><span class="mi">14</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">2</span>      <span class="mf">0.61</span> <span class="p">(</span><span class="mi">36</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">3</span>      <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
            <span class="n">E</span> <span class="mi">0</span>   <span class="mf">0.90039</span> <span class="p">(</span><span class="mi">54</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">1.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>    <span class="mf">1.8026</span> <span class="p">(</span><span class="mi">82</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">2.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">2</span>      <span class="mf">2.83</span> <span class="p">(</span><span class="mi">19</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">3.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">3</span>      <span class="mf">4.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">4.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">1</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">4</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="n">E1</span><span class="o">/</span><span class="n">E0</span> <span class="o">=</span> <span class="mf">2.0020</span><span class="p">(</span><span class="mi">86</span><span class="p">)</span>   <span class="n">E2</span><span class="o">/</span><span class="n">E0</span> <span class="o">=</span> <span class="mf">3.14</span><span class="p">(</span><span class="mi">21</span><span class="p">)</span>
<span class="n">a1</span><span class="o">/</span><span class="n">a0</span> <span class="o">=</span> <span class="mf">1.011</span><span class="p">(</span><span class="mi">32</span><span class="p">)</span>   <span class="n">a2</span><span class="o">/</span><span class="n">a0</span> <span class="o">=</span> <span class="mf">1.52</span><span class="p">(</span><span class="mi">89</span><span class="p">)</span>

<span class="o">---------------------</span> <span class="n">new</span> <span class="n">fit</span> <span class="n">to</span> <span class="n">extra</span> <span class="n">information</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.12</span> <span class="p">[</span><span class="mi">1</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.73</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">2.4648</span>

<span class="n">Parameters</span><span class="p">:</span>
            <span class="n">a</span> <span class="mi">0</span>    <span class="mf">0.4018</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.4019</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>    <span class="mf">0.4018</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>      <span class="p">[</span>   <span class="mf">0.406</span> <span class="p">(</span><span class="mi">14</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">2</span>      <span class="mf">0.57</span> <span class="p">(</span><span class="mi">34</span><span class="p">)</span>      <span class="p">[</span>    <span class="mf">0.61</span> <span class="p">(</span><span class="mi">36</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">3</span>      <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>      <span class="p">[</span>    <span class="mf">0.50</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  
            <span class="n">E</span> <span class="mi">0</span>   <span class="mf">0.90033</span> <span class="p">(</span><span class="mi">51</span><span class="p">)</span>      <span class="p">[</span> <span class="mf">0.90039</span> <span class="p">(</span><span class="mi">54</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>    <span class="mf">1.7998</span> <span class="p">(</span><span class="mi">13</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">1.8026</span> <span class="p">(</span><span class="mi">81</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">2</span>      <span class="mf">2.79</span> <span class="p">(</span><span class="mi">14</span><span class="p">)</span>      <span class="p">[</span>    <span class="mf">2.83</span> <span class="p">(</span><span class="mi">19</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">3</span>      <span class="mf">4.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>      <span class="p">[</span>    <span class="mf">4.00</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">14</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="n">E1</span><span class="o">/</span><span class="n">E0</span> <span class="o">=</span> <span class="mf">1.9991</span><span class="p">(</span><span class="mi">12</span><span class="p">)</span>   <span class="n">E2</span><span class="o">/</span><span class="n">E0</span> <span class="o">=</span> <span class="mf">3.10</span><span class="p">(</span><span class="mi">16</span><span class="p">)</span>
<span class="n">a1</span><span class="o">/</span><span class="n">a0</span> <span class="o">=</span> <span class="mf">1.000000</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span>   <span class="n">a2</span><span class="o">/</span><span class="n">a0</span> <span class="o">=</span> <span class="mf">1.43</span><span class="p">(</span><span class="mi">85</span><span class="p">)</span>
</pre></div>
</div>
<p>Parameters <code class="docutils literal notranslate"><span class="pre">a[0]</span></code> and <code class="docutils literal notranslate"><span class="pre">E[0]</span></code> are essentially unchanged by the new
information, but <code class="docutils literal notranslate"><span class="pre">a[1]</span></code> and <code class="docutils literal notranslate"><span class="pre">E[1]</span></code> are much more precise,
as is <code class="docutils literal notranslate"><span class="pre">a[1]/a[0]</span></code>, of course.</p>
<p>It might seem odd that <code class="docutils literal notranslate"><span class="pre">E[1]</span></code>, for example, is changed at
all, since the fit function, <code class="docutils literal notranslate"><span class="pre">ratio(p)</span></code>, makes no mention of it. This
is not surprising, however, since <code class="docutils literal notranslate"><span class="pre">ratio(p)</span></code> does depend upon <code class="docutils literal notranslate"><span class="pre">a[1]</span></code>,
and <code class="docutils literal notranslate"><span class="pre">a[1]</span></code> is strongly correlated with <code class="docutils literal notranslate"><span class="pre">E[1]</span></code> through the prior
(correlation coefficient of 0.94).
It is important to include all parameters from the first fit as
parameters in the new fit, in order to capture the impact of the new
information on parameters correlated with <code class="docutils literal notranslate"><span class="pre">a[1]/a[0]</span></code>.</p>
<p>Obviously, we can use further fits in order to incorporate additional data. The
prior for each new fit is the best-fit output (<code class="docutils literal notranslate"><span class="pre">fit.p</span></code>) from the previous
fit. The output from the chain’s final fit is the cumulative  result of all
of these fits.</p>
<p>Note that this particular problem can be done much more
simply using a weighted average (<a class="reference internal" href="lsqfit.html#lsqfit.wavg" title="lsqfit.wavg"><code class="xref py py-func docutils literal notranslate"><span class="pre">lsqfit.wavg()</span></code></a>).
Adding the following code
onto the end of the <code class="docutils literal notranslate"><span class="pre">main()</span></code> function above</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">fit</span><span class="o">.</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;a1/a0&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">fit</span><span class="o">.</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">][</span><span class="mi">1</span><span class="p">]</span> <span class="o">/</span> <span class="n">fit</span><span class="o">.</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span>
<span class="n">new_data</span> <span class="o">=</span> <span class="p">{</span><span class="s1">&#39;a1/a0&#39;</span> <span class="p">:</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span><span class="mf">1e-5</span><span class="p">)}</span>
<span class="n">new_p</span> <span class="o">=</span> <span class="n">lsqfit</span><span class="o">.</span><span class="n">wavg</span><span class="p">([</span><span class="n">fit</span><span class="o">.</span><span class="n">p</span><span class="p">,</span> <span class="n">new_data</span><span class="p">])</span>

<span class="nb">print</span><span class="p">(</span><span class="s1">&#39;chi2/dof = </span><span class="si">{:.2f}</span><span class="se">\n</span><span class="s1">&#39;</span> <span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">new_p</span><span class="o">.</span><span class="n">chi2</span> <span class="o">/</span> <span class="n">new_p</span><span class="o">.</span><span class="n">dof</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">&#39;E:&#39;</span><span class="p">,</span> <span class="n">new_p</span><span class="p">[</span><span class="s1">&#39;E&#39;</span><span class="p">][:</span><span class="mi">4</span><span class="p">])</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">&#39;a:&#39;</span><span class="p">,</span> <span class="n">new_p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">][:</span><span class="mi">4</span><span class="p">])</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">&#39;a1/a0:&#39;</span><span class="p">,</span> <span class="n">new_p</span><span class="p">[</span><span class="s1">&#39;a1/a0&#39;</span><span class="p">])</span>
</pre></div>
</div>
<p>gives the following output:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="o">=</span> <span class="mf">0.12</span>

<span class="n">E</span><span class="p">:</span> <span class="p">[</span><span class="mf">0.90033</span><span class="p">(</span><span class="mi">51</span><span class="p">)</span> <span class="mf">1.7998</span><span class="p">(</span><span class="mi">13</span><span class="p">)</span> <span class="mf">2.79</span><span class="p">(</span><span class="mi">14</span><span class="p">)</span> <span class="mf">4.00</span><span class="p">(</span><span class="mi">40</span><span class="p">)]</span>
<span class="n">a</span><span class="p">:</span> <span class="p">[</span><span class="mf">0.4018</span><span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="mf">0.4018</span><span class="p">(</span><span class="mi">40</span><span class="p">)</span> <span class="mf">0.57</span><span class="p">(</span><span class="mi">34</span><span class="p">)</span> <span class="mf">0.50</span><span class="p">(</span><span class="mi">40</span><span class="p">)]</span>
<span class="n">a1</span><span class="o">/</span><span class="n">a0</span><span class="p">:</span> <span class="mf">1.000000</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span>
</pre></div>
</div>
<p>Here we do a weighted average of <code class="docutils literal notranslate"><span class="pre">a[1]/a[0]</span></code> from the
original fit (<code class="docutils literal notranslate"><span class="pre">fit.p['a1/a0']</span></code>) with our new piece of data
(<code class="docutils literal notranslate"><span class="pre">new_data['a1/a0']</span></code>). The dictionary <code class="docutils literal notranslate"><span class="pre">new_p</span></code> returned by
<a class="reference internal" href="lsqfit.html#lsqfit.wavg" title="lsqfit.wavg"><code class="xref py py-func docutils literal notranslate"><span class="pre">lsqfit.wavg()</span></code></a> has an entry for
every key in either <code class="docutils literal notranslate"><span class="pre">fit.p</span></code> or <code class="docutils literal notranslate"><span class="pre">new_data</span></code>. The weighted average for
<code class="docutils literal notranslate"><span class="pre">a[1]/a[0]</span></code> is in <code class="docutils literal notranslate"><span class="pre">new_p['a1/a0']</span></code>. New values for the
fit parameters, that take account of the new data, are stored in
<code class="docutils literal notranslate"><span class="pre">new_p['E']</span></code> and <code class="docutils literal notranslate"><span class="pre">new_p['a']</span></code>. The <code class="docutils literal notranslate"><span class="pre">E[i]</span></code> and <code class="docutils literal notranslate"><span class="pre">a[i]</span></code>
estimates differ from their values in <code class="docutils literal notranslate"><span class="pre">fit.p</span></code> since those parameters
are correlated with <code class="docutils literal notranslate"><span class="pre">a[1]/a[0]</span></code>. Consequently when the ratio
is shifted by new data, the  <code class="docutils literal notranslate"><span class="pre">E[i]</span></code> and <code class="docutils literal notranslate"><span class="pre">a[i]</span></code> are shifted as well.
The final results in <code class="docutils literal notranslate"><span class="pre">new_p</span></code>
are identical to what we obtained above.</p>
<p>One place where chained fits can be useful is when there is lots of fit
data. Imagine, as a second example, a situation that involves 10,000 highly
correlated <code class="docutils literal notranslate"><span class="pre">y[i]</span></code>s. A straight fit would take a very long time because
part of the fit process involves diagonalizing the fit data’s (dense)
10,000×10,000 covariance matrix. Instead we break the data up into
batches of 100 and do chained fits of one batch after another:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># read data from disk</span>
<span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="o">=</span> <span class="n">read_data</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">&#39;x = [</span><span class="si">{}</span><span class="s1">  </span><span class="si">{}</span><span class="s1"> ... </span><span class="si">{}</span><span class="s1">]&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">x</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">x</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]))</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">&#39;y = [</span><span class="si">{}</span><span class="s1">  </span><span class="si">{}</span><span class="s1"> ... </span><span class="si">{}</span><span class="s1">]&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">y</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">y</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">y</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]))</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">&#39;corr(y[0],y[9999]) =&#39;</span><span class="p">,</span> <span class="n">gv</span><span class="o">.</span><span class="n">evalcorr</span><span class="p">([</span><span class="n">y</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">y</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]])[</span><span class="mi">1</span><span class="p">,</span><span class="mi">0</span><span class="p">])</span>
<span class="nb">print</span><span class="p">()</span>

<span class="c1"># fit function and prior</span>
<span class="k">def</span> <span class="nf">fcn</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">p</span><span class="p">):</span>
    <span class="k">return</span> <span class="n">p</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">+</span> <span class="n">p</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">*</span> <span class="n">np</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="o">-</span> <span class="n">p</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">*</span> <span class="n">x</span><span class="p">)</span>
<span class="n">prior</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">([</span><span class="s1">&#39;0(1)&#39;</span><span class="p">,</span> <span class="s1">&#39;0(1)&#39;</span><span class="p">,</span> <span class="s1">&#39;0(1)&#39;</span><span class="p">])</span>

<span class="c1"># Nstride fits, each to nfit data points</span>
<span class="n">nfit</span> <span class="o">=</span> <span class="mi">100</span>
<span class="n">Nstride</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">y</span><span class="p">)</span> <span class="o">//</span> <span class="n">nfit</span>
<span class="n">fit_time</span> <span class="o">=</span> <span class="mf">0.0</span>
<span class="k">for</span> <span class="n">n</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="n">Nstride</span><span class="p">):</span>
    <span class="n">fit</span> <span class="o">=</span> <span class="n">lsqfit</span><span class="o">.</span><span class="n">nonlinear_fit</span><span class="p">(</span>
        <span class="n">data</span><span class="o">=</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="n">n</span><span class="p">::</span><span class="n">Nstride</span><span class="p">],</span> <span class="n">y</span><span class="p">[</span><span class="n">n</span><span class="p">::</span><span class="n">Nstride</span><span class="p">]),</span> <span class="n">prior</span><span class="o">=</span><span class="n">prior</span><span class="p">,</span> <span class="n">fcn</span><span class="o">=</span><span class="n">fcn</span>
        <span class="p">)</span>
    <span class="n">prior</span> <span class="o">=</span> <span class="n">fit</span><span class="o">.</span><span class="n">p</span>
    <span class="k">if</span> <span class="n">n</span> <span class="ow">in</span> <span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="mi">9</span><span class="p">]:</span>
        <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;******* Results from &#39;</span><span class="p">,</span> <span class="p">(</span><span class="n">n</span><span class="o">+</span><span class="mi">1</span><span class="p">)</span> <span class="o">*</span> <span class="n">nfit</span><span class="p">,</span> <span class="s1">&#39;data points&#39;</span><span class="p">)</span>
        <span class="nb">print</span><span class="p">(</span><span class="n">fit</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">&#39;******* Results from &#39;</span><span class="p">,</span> <span class="n">Nstride</span> <span class="o">*</span> <span class="n">nfit</span><span class="p">,</span> <span class="s1">&#39;data points (final)&#39;</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">fit</span><span class="p">)</span>
</pre></div>
</div>
<p>In the loop, we fit only 100 data points at a time, but the prior we use is
the best-fit result from the fit to the previous 100 data points,
and its prior comes from fitting the 100 points before those, and so on
for 100 fits in all.
The output from this code is:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">x</span> <span class="o">=</span> <span class="p">[</span><span class="mf">0.2</span>  <span class="mf">0.200080008001</span> <span class="o">...</span> <span class="mf">1.0</span><span class="p">]</span>
<span class="n">y</span> <span class="o">=</span> <span class="p">[</span><span class="mf">0.836</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span>  <span class="mf">0.835</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span> <span class="o">...</span> <span class="mf">0.686</span><span class="p">(</span><span class="mi">10</span><span class="p">)]</span>
<span class="n">corr</span><span class="p">(</span><span class="n">y</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span><span class="n">y</span><span class="p">[</span><span class="mi">9999</span><span class="p">])</span> <span class="o">=</span> <span class="mf">0.990099009901</span>

<span class="o">*******</span> <span class="n">Results</span> <span class="kn">from</span>  <span class="mi">100</span> <span class="n">data</span> <span class="n">points</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">1.1</span> <span class="p">[</span><span class="mi">100</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.23</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">523.92</span>

<span class="n">Parameters</span><span class="p">:</span>
              <span class="mi">0</span>    <span class="mf">0.494</span> <span class="p">(</span><span class="mi">13</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">1.0</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>   <span class="mf">0.3939</span> <span class="p">(</span><span class="mi">75</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">1.0</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">2</span>    <span class="mf">0.715</span> <span class="p">(</span><span class="mi">23</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">1.0</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">11</span><span class="o">/</span><span class="mf">0.1</span><span class="p">)</span>

<span class="o">*******</span> <span class="n">Results</span> <span class="kn">from</span>  <span class="mi">1000</span> <span class="n">data</span> <span class="n">points</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">1.1</span> <span class="p">[</span><span class="mi">100</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.29</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">544.96</span>

<span class="n">Parameters</span><span class="p">:</span>
              <span class="mi">0</span>    <span class="mf">0.491</span> <span class="p">(</span><span class="mi">10</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.492</span> <span class="p">(</span><span class="mi">10</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>   <span class="mf">0.3969</span> <span class="p">(</span><span class="mi">24</span><span class="p">)</span>     <span class="p">[</span> <span class="mf">0.3965</span> <span class="p">(</span><span class="mi">25</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">2</span>   <span class="mf">0.7084</span> <span class="p">(</span><span class="mi">70</span><span class="p">)</span>     <span class="p">[</span> <span class="mf">0.7095</span> <span class="p">(</span><span class="mi">74</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">6</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="o">*******</span> <span class="n">Results</span> <span class="kn">from</span>  <span class="mi">10000</span> <span class="n">data</span> <span class="n">points</span> <span class="p">(</span><span class="n">final</span><span class="p">)</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mi">1</span> <span class="p">[</span><span class="mi">100</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.48</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">548.63</span>

<span class="n">Parameters</span><span class="p">:</span>
              <span class="mi">0</span>     <span class="mf">0.488</span> <span class="p">(</span><span class="mi">10</span><span class="p">)</span>      <span class="p">[</span>   <span class="mf">0.488</span> <span class="p">(</span><span class="mi">10</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>   <span class="mf">0.39988</span> <span class="p">(</span><span class="mi">77</span><span class="p">)</span>      <span class="p">[</span> <span class="mf">0.39982</span> <span class="p">(</span><span class="mi">78</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">2</span>    <span class="mf">0.7002</span> <span class="p">(</span><span class="mi">23</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.7003</span> <span class="p">(</span><span class="mi">23</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">4</span><span class="o">/</span><span class="mf">0.4</span><span class="p">)</span>

</pre></div>
</div>
<p>It shows the errors on <code class="docutils literal notranslate"><span class="pre">p[1]</span></code> and <code class="docutils literal notranslate"><span class="pre">p[2]</span></code> decreasing steadily as more
data points are included. The error on <code class="docutils literal notranslate"><span class="pre">p[0]</span></code>, however, hardly changes
at all. This is a consequence of the strong correlation between different
<code class="docutils literal notranslate"><span class="pre">y[i]</span></code>s (and its lack of x-dependence).
The “correct” answers here are 0.5, 0.4 and 0.7.</p>
<p>Chained fits are slower that straight fits with large amounts of
<em>uncorrelated</em> data, provided <a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a> is informed ahead of
time that the data are uncorrelated (by default it checks for
correlations, which can be expensive for lots of data).
The fitter is informed by using argument
<code class="docutils literal notranslate"><span class="pre">udata</span></code> instead of <code class="docutils literal notranslate"><span class="pre">data</span></code> to specify the fit data:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="o">=</span> <span class="n">read_data</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">&#39;x = [</span><span class="si">{}</span><span class="s1">  </span><span class="si">{}</span><span class="s1"> ... </span><span class="si">{}</span><span class="s1">]&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">x</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">x</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]))</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">&#39;y = [</span><span class="si">{}</span><span class="s1">  </span><span class="si">{}</span><span class="s1"> ... </span><span class="si">{}</span><span class="s1">]&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">y</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">y</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">y</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]))</span>
<span class="nb">print</span><span class="p">()</span>

<span class="c1"># fit function and prior</span>
<span class="k">def</span> <span class="nf">fcn</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">p</span><span class="p">):</span>
    <span class="k">return</span> <span class="n">p</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">+</span> <span class="n">p</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">*</span> <span class="n">np</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="o">-</span> <span class="n">p</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">*</span> <span class="n">x</span><span class="p">)</span>
<span class="n">prior</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">([</span><span class="s1">&#39;0(1)&#39;</span><span class="p">,</span> <span class="s1">&#39;0(1)&#39;</span><span class="p">,</span> <span class="s1">&#39;0(1)&#39;</span><span class="p">])</span>

<span class="n">fit</span> <span class="o">=</span> <span class="n">lsqfit</span><span class="o">.</span><span class="n">nonlinear_fit</span><span class="p">(</span><span class="n">udata</span><span class="o">=</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">),</span> <span class="n">prior</span><span class="o">=</span><span class="n">prior</span><span class="p">,</span> <span class="n">fcn</span><span class="o">=</span><span class="n">fcn</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">fit</span><span class="p">)</span>
</pre></div>
</div>
<p>Using <code class="docutils literal notranslate"><span class="pre">udata</span></code> rather than <code class="docutils literal notranslate"><span class="pre">data</span></code> causes <a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a> to
ignore correlations in the data, whether they exist or not.
Uncorrelated fits are typically
much faster when fitting large amounts of data, so it is then
possible to fit much more data
(e.g., 1,000,000 or more <code class="docutils literal notranslate"><span class="pre">y[i]</span></code>s is straightforward on a laptop).</p>
</section>
<section id="x-has-errors">
<h2><code class="docutils literal notranslate"><span class="pre">x</span></code> has Errors<a class="headerlink" href="#x-has-errors" title="Permalink to this heading">¶</a></h2>
<p>We now consider variations on our basic fit analysis (described in
<a class="reference internal" href="#basic-fits"><span class="std std-ref">Basic Fits</span></a>).
The first variation concerns what to do when the independent variables, the
<code class="docutils literal notranslate"><span class="pre">x</span></code>s, have errors, as well as the <code class="docutils literal notranslate"><span class="pre">y</span></code>s. This is easily handled by
turning the <code class="docutils literal notranslate"><span class="pre">x</span></code>s into fit parameters, and otherwise dispensing
with independent variables.</p>
<p>To illustrate, consider the data assembled by the following <code class="docutils literal notranslate"><span class="pre">make_data</span></code>
function:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">gvar</span> <span class="k">as</span> <span class="nn">gv</span>

<span class="k">def</span> <span class="nf">make_data</span><span class="p">():</span>
    <span class="n">x</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">([</span>
        <span class="s1">&#39;0.73(50)&#39;</span><span class="p">,</span>   <span class="s1">&#39;2.25(50)&#39;</span><span class="p">,</span>  <span class="s1">&#39;3.07(50)&#39;</span><span class="p">,</span>  <span class="s1">&#39;3.62(50)&#39;</span><span class="p">,</span>  <span class="s1">&#39;4.86(50)&#39;</span><span class="p">,</span>
        <span class="s1">&#39;6.41(50)&#39;</span><span class="p">,</span>   <span class="s1">&#39;6.39(50)&#39;</span><span class="p">,</span>  <span class="s1">&#39;7.89(50)&#39;</span><span class="p">,</span>  <span class="s1">&#39;9.32(50)&#39;</span><span class="p">,</span>  <span class="s1">&#39;9.78(50)&#39;</span><span class="p">,</span>
        <span class="s1">&#39;10.83(50)&#39;</span><span class="p">,</span> <span class="s1">&#39;11.98(50)&#39;</span><span class="p">,</span> <span class="s1">&#39;13.37(50)&#39;</span><span class="p">,</span> <span class="s1">&#39;13.84(50)&#39;</span><span class="p">,</span> <span class="s1">&#39;14.89(50)&#39;</span>
        <span class="p">])</span>
    <span class="n">y</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">([</span>
         <span class="s1">&#39;3.85(70)&#39;</span><span class="p">,</span>  <span class="s1">&#39;5.5(1.7)&#39;</span><span class="p">,</span>  <span class="s1">&#39;14.0(2.6)&#39;</span><span class="p">,</span>   <span class="s1">&#39;21.8(3.4)&#39;</span><span class="p">,</span>   <span class="s1">&#39;47.0(5.2)&#39;</span><span class="p">,</span>
        <span class="s1">&#39;79.8(4.6)&#39;</span><span class="p">,</span> <span class="s1">&#39;84.9(4.6)&#39;</span><span class="p">,</span>  <span class="s1">&#39;95.2(2.2)&#39;</span><span class="p">,</span>   <span class="s1">&#39;97.65(79)&#39;</span><span class="p">,</span>   <span class="s1">&#39;98.78(55)&#39;</span><span class="p">,</span>
        <span class="s1">&#39;99.41(25)&#39;</span><span class="p">,</span> <span class="s1">&#39;99.80(12)&#39;</span><span class="p">,</span> <span class="s1">&#39;100.127(77)&#39;</span><span class="p">,</span> <span class="s1">&#39;100.202(73)&#39;</span><span class="p">,</span> <span class="s1">&#39;100.203(71)&#39;</span>
        <span class="p">])</span>
    <span class="k">return</span> <span class="n">x</span><span class="p">,</span><span class="n">y</span>
</pre></div>
</div>
<p>The function call <code class="docutils literal notranslate"><span class="pre">x,y</span> <span class="pre">=</span> <span class="pre">make_data()</span></code> returns values for the <code class="docutils literal notranslate"><span class="pre">x[i]</span></code>s and
the corresponding <code class="docutils literal notranslate"><span class="pre">y[i]</span></code>s, where now both are <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s.</p>
<p>We want to fit the <code class="docutils literal notranslate"><span class="pre">y</span></code> values with a function of the form:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">b0</span> <span class="o">/</span> <span class="p">((</span><span class="mi">1</span> <span class="o">+</span> <span class="n">gv</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="n">b1</span> <span class="o">-</span> <span class="n">b2</span> <span class="o">*</span> <span class="n">x</span><span class="p">))</span> <span class="o">**</span> <span class="p">(</span><span class="mf">1.</span> <span class="o">/</span> <span class="n">b3</span><span class="p">))</span><span class="o">.</span>
</pre></div>
</div>
<p>So we have two sets of parameters for which we need priors: the <code class="docutils literal notranslate"><span class="pre">b[i]</span></code>s and
the <code class="docutils literal notranslate"><span class="pre">x[i]</span></code>s:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">gvar</span> <span class="k">as</span> <span class="nn">gv</span>

<span class="k">def</span> <span class="nf">make_prior</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
    <span class="n">prior</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">BufferDict</span><span class="p">()</span>
    <span class="n">prior</span><span class="p">[</span><span class="s1">&#39;b&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">([</span><span class="s1">&#39;0(500)&#39;</span><span class="p">,</span> <span class="s1">&#39;0(5)&#39;</span><span class="p">,</span> <span class="s1">&#39;0(5)&#39;</span><span class="p">,</span> <span class="s1">&#39;0(5)&#39;</span><span class="p">])</span>
    <span class="n">prior</span><span class="p">[</span><span class="s1">&#39;x&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">x</span>
    <span class="k">return</span> <span class="n">prior</span>
</pre></div>
</div>
<p>The prior values for the <code class="docutils literal notranslate"><span class="pre">x[i]</span></code> are just the values returned by
<code class="docutils literal notranslate"><span class="pre">make_data()</span></code>. The corresponding fit function is:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">gvar</span> <span class="k">as</span> <span class="nn">gv</span>

<span class="k">def</span> <span class="nf">fcn</span><span class="p">(</span><span class="n">p</span><span class="p">):</span>
    <span class="n">b0</span><span class="p">,</span> <span class="n">b1</span><span class="p">,</span> <span class="n">b2</span><span class="p">,</span> <span class="n">b3</span> <span class="o">=</span> <span class="n">p</span><span class="p">[</span><span class="s1">&#39;b&#39;</span><span class="p">]</span>
    <span class="n">x</span> <span class="o">=</span> <span class="n">p</span><span class="p">[</span><span class="s1">&#39;x&#39;</span><span class="p">]</span>
    <span class="k">return</span> <span class="n">b0</span> <span class="o">/</span> <span class="p">((</span><span class="mf">1.</span> <span class="o">+</span> <span class="n">gv</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="n">b1</span> <span class="o">-</span> <span class="n">b2</span> <span class="o">*</span> <span class="n">x</span><span class="p">))</span> <span class="o">**</span> <span class="p">(</span><span class="mf">1.</span> <span class="o">/</span> <span class="n">b3</span><span class="p">))</span>
</pre></div>
</div>
<p>where the dependent variables <code class="docutils literal notranslate"><span class="pre">x[i]</span></code> are no longer arguments
of the function,
but rather are fit parameter in dictionary <code class="docutils literal notranslate"><span class="pre">p</span></code>.</p>
<p>The actual fit is now straightforward:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">lsqfit</span>

<span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="o">=</span> <span class="n">make_data</span><span class="p">()</span>
<span class="n">prior</span> <span class="o">=</span> <span class="n">make_prior</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
<span class="n">fit</span> <span class="o">=</span> <span class="n">lsqfit</span><span class="o">.</span><span class="n">nonlinear_fit</span><span class="p">(</span><span class="n">prior</span><span class="o">=</span><span class="n">prior</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">y</span><span class="p">,</span> <span class="n">fcn</span><span class="o">=</span><span class="n">fcn</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">fit</span><span class="p">)</span>
</pre></div>
</div>
<p>This generates the following output:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.35</span> <span class="p">[</span><span class="mi">15</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.99</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="o">-</span><span class="mf">40.156</span>

<span class="n">Parameters</span><span class="p">:</span>
            <span class="n">b</span> <span class="mi">0</span>   <span class="mf">100.238</span> <span class="p">(</span><span class="mi">60</span><span class="p">)</span>      <span class="p">[</span>    <span class="mi">0</span> <span class="p">(</span><span class="mi">500</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>      <span class="mf">3.5</span> <span class="p">(</span><span class="mf">1.2</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">5.0</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">2</span>     <span class="mf">0.797</span> <span class="p">(</span><span class="mi">87</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">5.0</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">3</span>      <span class="mf">0.77</span> <span class="p">(</span><span class="mi">35</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">5.0</span><span class="p">)</span> <span class="p">]</span>  
            <span class="n">x</span> <span class="mi">0</span>      <span class="mf">1.26</span> <span class="p">(</span><span class="mi">41</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.73</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  <span class="o">*</span>
              <span class="mi">1</span>      <span class="mf">1.87</span> <span class="p">(</span><span class="mi">34</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">2.25</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">2</span>      <span class="mf">2.84</span> <span class="p">(</span><span class="mi">28</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">3.07</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">3</span>      <span class="mf">3.42</span> <span class="p">(</span><span class="mi">29</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">3.62</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">4</span>      <span class="mf">4.72</span> <span class="p">(</span><span class="mi">32</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">4.86</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">5</span>      <span class="mf">6.45</span> <span class="p">(</span><span class="mi">33</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">6.41</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">6</span>      <span class="mf">6.69</span> <span class="p">(</span><span class="mi">35</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">6.39</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">7</span>      <span class="mf">8.15</span> <span class="p">(</span><span class="mi">36</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">7.89</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">8</span>      <span class="mf">9.30</span> <span class="p">(</span><span class="mi">35</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">9.32</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">9</span>      <span class="mf">9.91</span> <span class="p">(</span><span class="mi">37</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">9.78</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
             <span class="mi">10</span>     <span class="mf">10.77</span> <span class="p">(</span><span class="mi">37</span><span class="p">)</span>      <span class="p">[</span> <span class="mf">10.83</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
             <span class="mi">11</span>     <span class="mf">11.70</span> <span class="p">(</span><span class="mi">38</span><span class="p">)</span>      <span class="p">[</span> <span class="mf">11.98</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
             <span class="mi">12</span>     <span class="mf">13.34</span> <span class="p">(</span><span class="mi">46</span><span class="p">)</span>      <span class="p">[</span> <span class="mf">13.37</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
             <span class="mi">13</span>     <span class="mf">13.91</span> <span class="p">(</span><span class="mi">48</span><span class="p">)</span>      <span class="p">[</span> <span class="mf">13.84</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
             <span class="mi">14</span>     <span class="mf">14.88</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span>      <span class="p">[</span> <span class="mf">14.89</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">13</span><span class="o">/</span><span class="mf">0.1</span><span class="p">)</span>

</pre></div>
</div>
<p>The fit gives new results for the <code class="docutils literal notranslate"><span class="pre">b[i]</span></code> parameters that are much
improved from our prior estimates. Results for many of the <code class="docutils literal notranslate"><span class="pre">x[i]</span></code>s
are improved as well, by information from the fit data. The following
plot shows the fit (dashed line) compared with the input data for <code class="docutils literal notranslate"><span class="pre">y</span></code>:</p>
<a class="reference internal image-reference" href="_images/eg2.png"><img alt="_images/eg2.png" src="_images/eg2.png" style="width: 70%;" /></a>
</section>
<section id="correlated-parameters-gaussian-bayes-factor">
<span id="correlated-parameters"></span><h2>Correlated Parameters; Gaussian Bayes Factor<a class="headerlink" href="#correlated-parameters-gaussian-bayes-factor" title="Permalink to this heading">¶</a></h2>
<p><code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code> objects allow for complicated priors, including
priors that correlate different fit parameters.
The following fit analysis code illustrates
how this is done:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">import</span> <span class="nn">gvar</span> <span class="k">as</span> <span class="nn">gv</span>
<span class="kn">import</span> <span class="nn">lsqfit</span>

<span class="k">def</span> <span class="nf">main</span><span class="p">():</span>
    <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="o">=</span> <span class="n">make_data</span><span class="p">()</span>
    <span class="n">prior</span> <span class="o">=</span> <span class="n">make_prior</span><span class="p">()</span>
    <span class="n">fit</span> <span class="o">=</span> <span class="n">lsqfit</span><span class="o">.</span><span class="n">nonlinear_fit</span><span class="p">(</span><span class="n">prior</span><span class="o">=</span><span class="n">prior</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="p">(</span><span class="n">x</span><span class="p">,</span><span class="n">y</span><span class="p">),</span> <span class="n">fcn</span><span class="o">=</span><span class="n">fcn</span><span class="p">)</span>
    <span class="nb">print</span><span class="p">(</span><span class="n">fit</span><span class="p">)</span>
    <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;p1/p0 =&#39;</span><span class="p">,</span> <span class="n">fit</span><span class="o">.</span><span class="n">p</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">/</span> <span class="n">fit</span><span class="o">.</span><span class="n">p</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="s1">&#39;p3/p2 =&#39;</span><span class="p">,</span> <span class="n">fit</span><span class="o">.</span><span class="n">p</span><span class="p">[</span><span class="mi">3</span><span class="p">]</span> <span class="o">/</span> <span class="n">fit</span><span class="o">.</span><span class="n">p</span><span class="p">[</span><span class="mi">2</span><span class="p">])</span>
    <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;corr(p0,p1) =&#39;</span><span class="p">,</span> <span class="n">gv</span><span class="o">.</span><span class="n">evalcorr</span><span class="p">(</span><span class="n">fit</span><span class="o">.</span><span class="n">p</span><span class="p">[:</span><span class="mi">2</span><span class="p">])[</span><span class="mi">1</span><span class="p">,</span><span class="mi">0</span><span class="p">])</span>

<span class="k">def</span> <span class="nf">make_data</span><span class="p">():</span>
    <span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span>
        <span class="mf">4.</span><span class="p">,</span> <span class="mf">2.</span><span class="p">,</span> <span class="mf">1.</span><span class="p">,</span> <span class="mf">0.5</span><span class="p">,</span> <span class="mf">0.25</span><span class="p">,</span> <span class="mf">0.167</span><span class="p">,</span> <span class="mf">0.125</span><span class="p">,</span> <span class="mf">0.1</span><span class="p">,</span> <span class="mf">0.0833</span><span class="p">,</span> <span class="mf">0.0714</span><span class="p">,</span> <span class="mf">0.0625</span>
        <span class="p">])</span>
    <span class="n">y</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">([</span>
        <span class="s1">&#39;0.198(14)&#39;</span><span class="p">,</span> <span class="s1">&#39;0.216(15)&#39;</span><span class="p">,</span> <span class="s1">&#39;0.184(23)&#39;</span><span class="p">,</span> <span class="s1">&#39;0.156(44)&#39;</span><span class="p">,</span> <span class="s1">&#39;0.099(49)&#39;</span><span class="p">,</span>
        <span class="s1">&#39;0.142(40)&#39;</span><span class="p">,</span> <span class="s1">&#39;0.108(32)&#39;</span><span class="p">,</span> <span class="s1">&#39;0.065(26)&#39;</span><span class="p">,</span> <span class="s1">&#39;0.044(22)&#39;</span><span class="p">,</span> <span class="s1">&#39;0.041(19)&#39;</span><span class="p">,</span>
        <span class="s1">&#39;0.044(16)&#39;</span>
        <span class="p">])</span>
    <span class="k">return</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span>

<span class="k">def</span> <span class="nf">make_prior</span><span class="p">():</span>
    <span class="n">p</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">([</span><span class="s1">&#39;0(1)&#39;</span><span class="p">,</span> <span class="s1">&#39;0(1)&#39;</span><span class="p">,</span> <span class="s1">&#39;0(1)&#39;</span><span class="p">,</span> <span class="s1">&#39;0(1)&#39;</span><span class="p">])</span>
    <span class="n">p</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">=</span> <span class="mi">20</span> <span class="o">*</span> <span class="n">p</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">+</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="s1">&#39;0.0(1)&#39;</span><span class="p">)</span>        <span class="c1"># p[1] correlated with p[0]</span>
    <span class="k">return</span> <span class="n">p</span>

<span class="k">def</span> <span class="nf">fcn</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">p</span><span class="p">):</span>
    <span class="k">return</span> <span class="p">(</span><span class="n">p</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">*</span> <span class="p">(</span><span class="n">x</span><span class="o">**</span><span class="mi">2</span> <span class="o">+</span> <span class="n">p</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">*</span> <span class="n">x</span><span class="p">))</span> <span class="o">/</span> <span class="p">(</span><span class="n">x</span><span class="o">**</span><span class="mi">2</span> <span class="o">+</span> <span class="n">x</span> <span class="o">*</span> <span class="n">p</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">+</span> <span class="n">p</span><span class="p">[</span><span class="mi">3</span><span class="p">])</span>

<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">&#39;__main__&#39;</span><span class="p">:</span>
    <span class="n">main</span><span class="p">()</span>
</pre></div>
</div>
<p>Here, again, functions <code class="docutils literal notranslate"><span class="pre">make_data()</span></code> and <code class="docutils literal notranslate"><span class="pre">make_prior()</span></code> assemble the
fit data and prior, and parameters <code class="docutils literal notranslate"><span class="pre">p[i]</span></code> are adjusted by the fitter
to make <code class="docutils literal notranslate"><span class="pre">fcn(x[i],</span> <span class="pre">p)</span></code> agree with the data value <code class="docutils literal notranslate"><span class="pre">y[i]</span></code>. The priors
are fairly broad (0±1) for all of the parameters, except for
<code class="docutils literal notranslate"><span class="pre">p[1]</span></code>. The prior introduces a tight relationship between <code class="docutils literal notranslate"><span class="pre">p[1]</span></code>
and <code class="docutils literal notranslate"><span class="pre">p[0]</span></code>:  it sets <code class="docutils literal notranslate"><span class="pre">p[1]=20*p[0]</span></code> up to corrections of
order 0±0.1. This <em>a priori</em> relationship is built into the prior
and restricts the fit.</p>
<p>Running the code gives the following output:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.61</span> <span class="p">[</span><span class="mi">11</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.82</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">19.129</span>

<span class="n">Parameters</span><span class="p">:</span>
              <span class="mi">0</span>   <span class="mf">0.149</span> <span class="p">(</span><span class="mi">17</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">1.0</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>    <span class="mf">2.97</span> <span class="p">(</span><span class="mi">34</span><span class="p">)</span>     <span class="p">[</span>     <span class="mi">0</span> <span class="p">(</span><span class="mi">20</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">2</span>    <span class="mf">1.23</span> <span class="p">(</span><span class="mi">61</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">1.0</span><span class="p">)</span> <span class="p">]</span>  <span class="o">*</span>
              <span class="mi">3</span>    <span class="mf">0.59</span> <span class="p">(</span><span class="mi">15</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">1.0</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">19</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="n">p1</span><span class="o">/</span><span class="n">p0</span> <span class="o">=</span> <span class="mf">19.97</span><span class="p">(</span><span class="mi">67</span><span class="p">)</span>     <span class="n">p3</span><span class="o">/</span><span class="n">p2</span> <span class="o">=</span> <span class="mf">0.48</span><span class="p">(</span><span class="mi">22</span><span class="p">)</span>
<span class="n">corr</span><span class="p">(</span><span class="n">p0</span><span class="p">,</span><span class="n">p1</span><span class="p">)</span> <span class="o">=</span> <span class="mf">0.9570678206095675</span>
</pre></div>
</div>
<p>Note how the ratio <code class="docutils literal notranslate"><span class="pre">p1/p0</span></code> is much more accurate than either quantity
separately. The prior introduces a strong, positive correlation between
the two parameters that
survives the fit: the correlation coefficient is 0.96. Comparing the
fit function with the best-fit parameters (dashed line)
with the data shows a good fit:</p>
<a class="reference internal image-reference" href="_images/eg3.png"><img alt="_images/eg3.png" src="_images/eg3.png" style="width: 70%;" /></a>
<p>If we omit the constraint in the prior,</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">make_prior</span><span class="p">():</span>
    <span class="n">p</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">([</span><span class="s1">&#39;0(1)&#39;</span><span class="p">,</span> <span class="s1">&#39;0(20)&#39;</span><span class="p">,</span> <span class="s1">&#39;0(1)&#39;</span><span class="p">,</span> <span class="s1">&#39;0(1)&#39;</span><span class="p">])</span>
    <span class="k">return</span> <span class="n">p</span>
</pre></div>
</div>
<p>we obtain quite different fit results:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.35</span> <span class="p">[</span><span class="mi">11</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.97</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">11.036</span>

<span class="n">Parameters</span><span class="p">:</span>
              <span class="mi">0</span>   <span class="mf">0.211</span> <span class="p">(</span><span class="mi">18</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">1.0</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>   <span class="o">-</span><span class="mf">0.02</span> <span class="p">(</span><span class="mi">14</span><span class="p">)</span>     <span class="p">[</span>     <span class="mi">0</span> <span class="p">(</span><span class="mi">20</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">2</span>    <span class="mf">0.07</span> <span class="p">(</span><span class="mi">10</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">1.0</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">3</span>   <span class="mf">0.008</span> <span class="p">(</span><span class="mi">43</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">1.0</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">28</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="n">p1</span><span class="o">/</span><span class="n">p0</span> <span class="o">=</span> <span class="o">-</span><span class="mf">0.08</span><span class="p">(</span><span class="mi">64</span><span class="p">)</span>     <span class="n">p3</span><span class="o">/</span><span class="n">p2</span> <span class="o">=</span> <span class="mf">0.10</span><span class="p">(</span><span class="mi">62</span><span class="p">)</span>
<span class="n">corr</span><span class="p">(</span><span class="n">p0</span><span class="p">,</span><span class="n">p1</span><span class="p">)</span> <span class="o">=</span> <span class="o">-</span><span class="mf">0.5928698851782287</span>
</pre></div>
</div>
<p>Note that the Gaussian Bayes Factor (see <code class="docutils literal notranslate"><span class="pre">logGBF</span></code> in the output) is
larger with the correlated prior (<code class="docutils literal notranslate"><span class="pre">logGBF</span> <span class="pre">=</span> <span class="pre">19.1</span></code>) than it
was for the uncorrelated prior (<code class="docutils literal notranslate"><span class="pre">logGBF</span> <span class="pre">=</span> <span class="pre">11.0</span></code>). Had we been
uncertain as to which prior was more appropriate, this difference says that
the data prefers the correlated prior. (More precisely, it says that we
would be <code class="docutils literal notranslate"><span class="pre">exp(19.1-11.0)</span> <span class="pre">=</span> <span class="pre">3300</span></code> times more likely to get
our <code class="docutils literal notranslate"><span class="pre">x,y</span></code> data
from a theory with the
correlated prior than from one with the uncorrelated prior.) This
difference is significant despite the fact that the <code class="docutils literal notranslate"><span class="pre">chi**2</span></code> is
lower for the uncorrelated case. <code class="docutils literal notranslate"><span class="pre">chi**2</span></code> tests goodness of fit,
but there are
usually more ways than one to get a good fit. Some are more plausible
than others, and the Bayes Factor helps sort out which.</p>
<p>The Gaussian Bayes Factor is an approximation to
the Bayes Factor which is valid in the limit where
all distributions can be approximated by Gaussians. The Bayes Factor
is the probability
(density) that the fit data would be generated randomly from the
fit function and priors (the <em>model</em>) used in the fit.
Ratios of Bayes Factors from fits with different models tell us about the
relative likelihood of the different models given the data. (Actually the
ratio gives the ratio of probabilities for obtaining the data
from the models, as opposed to the probabilities for the models given
the data. See the discussion below.)</p>
</section>
<section id="y-has-no-error-marginalization">
<h2><code class="docutils literal notranslate"><span class="pre">y</span></code> has No Error; Marginalization<a class="headerlink" href="#y-has-no-error-marginalization" title="Permalink to this heading">¶</a></h2>
<p>Occasionally there are fit problems where values for the dependent
variable <code class="docutils literal notranslate"><span class="pre">y</span></code> are known exactly (to machine precision). This poses a
problem for least-squares fitting since the <code class="docutils literal notranslate"><span class="pre">chi**2</span></code> function is
infinite when standard deviations are zero. How does one assign errors
to exact <code class="docutils literal notranslate"><span class="pre">y</span></code>s in order to define a <code class="docutils literal notranslate"><span class="pre">chi**2</span></code> function that can be
usefully minimized?</p>
<p>It is almost always the case in physical applications of this sort that the
fit function has in principle an infinite number of parameters. It is, of
course, impossible to extract information about infinitely many parameters
from a finite number of <code class="docutils literal notranslate"><span class="pre">y</span></code>s. In practice, however, we generally care about
only a few of the parameters in the fit function.
The goal for a least-squares fit is to figure out what a finite
number of exact <code class="docutils literal notranslate"><span class="pre">y</span></code>s can tell us about the parameters we want to know.</p>
<p>The key idea here is to use priors to model the part of the fit function
that we don’t care about, and to remove that part of the function from
the analysis by subtracting it out from the input data. This is called
<em>marginalization</em>.</p>
<p>To illustrate how it is done,
we consider data that is generated from
an infinite sum of decaying exponentials, like that in <a class="reference internal" href="#basic-fits"><span class="std std-ref">Basic Fits</span></a>:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>

  <span class="k">def</span> <span class="nf">make_data</span><span class="p">():</span>
      <span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span> <span class="mf">1.</span><span class="p">,</span> <span class="mf">1.2</span><span class="p">,</span> <span class="mf">1.4</span><span class="p">,</span> <span class="mf">1.6</span><span class="p">,</span> <span class="mf">1.8</span><span class="p">,</span> <span class="mf">2.</span><span class="p">,</span> <span class="mf">2.2</span><span class="p">,</span> <span class="mf">2.4</span><span class="p">,</span> <span class="mf">2.6</span><span class="p">])</span>
      <span class="n">y</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span>
          <span class="mf">0.2740471001620033</span><span class="p">,</span>  <span class="mf">0.2056894154005132</span><span class="p">,</span>  <span class="mf">0.158389402324004</span><span class="p">,</span>
          <span class="mf">0.1241967645280511</span><span class="p">,</span>  <span class="mf">0.0986901274726867</span><span class="p">,</span>  <span class="mf">0.0792134506060024</span><span class="p">,</span>
          <span class="mf">0.0640743982173861</span><span class="p">,</span>  <span class="mf">0.052143504367789</span> <span class="p">,</span>  <span class="mf">0.0426383022456816</span><span class="p">,</span>
          <span class="p">])</span>
      <span class="k">return</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span>
</pre></div>
</div>
<p>Now <code class="docutils literal notranslate"><span class="pre">x,y</span> <span class="pre">=</span> <span class="pre">make_data()</span></code> returns nine <code class="docutils literal notranslate"><span class="pre">x[i]</span></code>s together with the
corresponding <code class="docutils literal notranslate"><span class="pre">y[i]</span></code>s, but where the <code class="docutils literal notranslate"><span class="pre">y[i]</span></code>s are exact and so
no longer represented by <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s.</p>
<p>We want to fit these data
with a sum of exponentials, as before:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>

<span class="k">def</span> <span class="nf">fcn</span><span class="p">(</span><span class="n">x</span><span class="p">,</span><span class="n">p</span><span class="p">):</span>
    <span class="n">a</span> <span class="o">=</span> <span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]</span>       <span class="c1"># array of a[i]s</span>
    <span class="n">E</span> <span class="o">=</span> <span class="n">p</span><span class="p">[</span><span class="s1">&#39;E&#39;</span><span class="p">]</span>       <span class="c1"># array of E[i]s</span>
    <span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">ai</span> <span class="o">*</span> <span class="n">np</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="o">-</span><span class="n">Ei</span><span class="o">*</span><span class="n">x</span><span class="p">)</span> <span class="k">for</span> <span class="n">ai</span><span class="p">,</span> <span class="n">Ei</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">E</span><span class="p">))</span>
</pre></div>
</div>
<p>We know that the amplitudes <code class="docutils literal notranslate"><span class="pre">a[i]</span></code> are of order 0.5±0.5, and
that the leading exponent <code class="docutils literal notranslate"><span class="pre">E[0]</span></code> is 1±0.1, as are the differences
between subsequent exponents <code class="docutils literal notranslate"><span class="pre">dE[i]</span> <span class="pre">=</span> <span class="pre">E[i]</span> <span class="pre">-</span> <span class="pre">E[i-1]</span></code>. This <em>a priori</em>
knowledge is encoded in the priors:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">import</span> <span class="nn">gvar</span> <span class="k">as</span> <span class="nn">gv</span>

<span class="k">def</span> <span class="nf">make_prior</span><span class="p">(</span><span class="n">nexp</span><span class="p">):</span>
    <span class="n">prior</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">BufferDict</span><span class="p">()</span>
    <span class="n">prior</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="n">nexp</span> <span class="o">*</span> <span class="p">[</span><span class="s1">&#39;0.5(5)&#39;</span><span class="p">])</span>
    <span class="n">dE</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="n">nexp</span> <span class="o">*</span> <span class="p">[</span><span class="s1">&#39;1.0(1)&#39;</span><span class="p">])</span>
    <span class="n">prior</span><span class="p">[</span><span class="s1">&#39;E&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">cumsum</span><span class="p">(</span><span class="n">dE</span><span class="p">)</span>
    <span class="k">return</span> <span class="n">prior</span>
</pre></div>
</div>
<p>We use a large number of exponential terms since
our <code class="docutils literal notranslate"><span class="pre">y[i]</span></code>s are exact: we keep 100 terms in all,
but our results are unchanged
with any number greater than about 10. Only a small number <code class="docutils literal notranslate"><span class="pre">nexp</span></code>
of these are included in the fit function. The <code class="docutils literal notranslate"><span class="pre">100-nexp</span></code> terms left out
are subtracted from the <code class="docutils literal notranslate"><span class="pre">y[i]</span></code> before the fit, using
the prior values for the omitted parameters to evaluate these terms.
This gives
new fit data <code class="docutils literal notranslate"><span class="pre">ymod[i]</span></code>:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">prior</span> <span class="o">=</span> <span class="n">make_prior</span><span class="p">(</span><span class="mi">100</span><span class="p">)</span>

<span class="c1"># the first nexp terms are fit; the remainder go into ymod</span>
<span class="n">fit_prior</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">BufferDict</span><span class="p">()</span>
<span class="n">ymod_prior</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">BufferDict</span><span class="p">()</span>
<span class="k">for</span> <span class="n">k</span> <span class="ow">in</span> <span class="n">prior</span><span class="p">:</span>
    <span class="n">fit_prior</span><span class="p">[</span><span class="n">k</span><span class="p">]</span> <span class="o">=</span> <span class="n">prior</span><span class="p">[:</span><span class="n">nexp</span><span class="p">]</span>
    <span class="n">ymod_prior</span><span class="p">[</span><span class="n">k</span><span class="p">]</span> <span class="o">=</span> <span class="n">prior</span><span class="p">[</span><span class="n">nexp</span><span class="p">:]</span>

<span class="n">ymod</span> <span class="o">=</span> <span class="n">y</span> <span class="o">-</span> <span class="n">fcn</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">ymod_prior</span><span class="p">)</span>
<span class="n">fit</span> <span class="o">=</span> <span class="n">lsqfit</span><span class="o">.</span><span class="n">nonlinear_fit</span><span class="p">(</span><span class="n">data</span><span class="o">=</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">ymod</span><span class="p">),</span> <span class="n">prior</span><span class="o">=</span><span class="n">fit_prior</span><span class="p">,</span> <span class="n">fcn</span><span class="o">=</span><span class="n">fcn</span><span class="p">)</span>
</pre></div>
</div>
<p>By subtracting <code class="docutils literal notranslate"><span class="pre">fcn(x,</span> <span class="pre">ymod_prior)</span></code> from <code class="docutils literal notranslate"><span class="pre">y</span></code>, we remove the parameters
that are in <code class="docutils literal notranslate"><span class="pre">ymod_prior</span></code> from the data, and consequently those parameters
need not be included in fit function. The fitter uses only
the parameters left in <code class="docutils literal notranslate"><span class="pre">fit_prior</span></code>.</p>
<p>Our complete code, therefore, is:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">import</span> <span class="nn">gvar</span> <span class="k">as</span> <span class="nn">gv</span>
<span class="kn">import</span> <span class="nn">lsqfit</span>

<span class="k">def</span> <span class="nf">main</span><span class="p">():</span>
    <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="o">=</span> <span class="n">make_data</span><span class="p">()</span>
    <span class="n">prior</span> <span class="o">=</span> <span class="n">make_prior</span><span class="p">(</span><span class="mi">100</span><span class="p">)</span>             <span class="c1"># 100 exponential terms in all</span>
    <span class="n">p0</span> <span class="o">=</span> <span class="kc">None</span>
    <span class="k">for</span> <span class="n">nexp</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">6</span><span class="p">):</span>
        <span class="c1"># marginalize the last 100 - nexp terms (in ymod_prior)</span>
        <span class="n">fit_prior</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">BufferDict</span><span class="p">()</span>     <span class="c1"># part of prior used in fit</span>
        <span class="n">ymod_prior</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">BufferDict</span><span class="p">()</span>    <span class="c1"># part of prior absorbed in ymod</span>
        <span class="k">for</span> <span class="n">k</span> <span class="ow">in</span> <span class="n">prior</span><span class="p">:</span>
            <span class="n">fit_prior</span><span class="p">[</span><span class="n">k</span><span class="p">]</span> <span class="o">=</span> <span class="n">prior</span><span class="p">[</span><span class="n">k</span><span class="p">][:</span><span class="n">nexp</span><span class="p">]</span>
            <span class="n">ymod_prior</span><span class="p">[</span><span class="n">k</span><span class="p">]</span> <span class="o">=</span> <span class="n">prior</span><span class="p">[</span><span class="n">k</span><span class="p">][</span><span class="n">nexp</span><span class="p">:]</span>
        <span class="n">ymod</span> <span class="o">=</span> <span class="n">y</span> <span class="o">-</span> <span class="n">fcn</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">ymod_prior</span><span class="p">)</span>   <span class="c1"># remove temrs in ymod_prior</span>

        <span class="c1"># fit modified data with just nexp terms (in fit_prior)</span>
        <span class="n">fit</span> <span class="o">=</span> <span class="n">lsqfit</span><span class="o">.</span><span class="n">nonlinear_fit</span><span class="p">(</span>
            <span class="n">data</span><span class="o">=</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">ymod</span><span class="p">),</span> <span class="n">prior</span><span class="o">=</span><span class="n">fit_prior</span><span class="p">,</span> <span class="n">fcn</span><span class="o">=</span><span class="n">fcn</span><span class="p">,</span> <span class="n">p0</span><span class="o">=</span><span class="n">p0</span><span class="p">,</span> <span class="n">tol</span><span class="o">=</span><span class="mf">1e-10</span><span class="p">,</span>
            <span class="p">)</span>

        <span class="c1"># print fit information</span>
        <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;************************************* nexp =&#39;</span><span class="p">,</span><span class="n">nexp</span><span class="p">)</span>
        <span class="nb">print</span><span class="p">(</span><span class="n">fit</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="kc">True</span><span class="p">))</span>
        <span class="n">p0</span> <span class="o">=</span> <span class="n">fit</span><span class="o">.</span><span class="n">pmean</span>

    <span class="c1"># print summary information and error budget</span>
    <span class="n">E</span> <span class="o">=</span> <span class="n">fit</span><span class="o">.</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;E&#39;</span><span class="p">]</span>                      <span class="c1"># best-fit parameters</span>
    <span class="n">a</span> <span class="o">=</span> <span class="n">fit</span><span class="o">.</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]</span>
    <span class="n">outputs</span> <span class="o">=</span> <span class="p">{</span>
        <span class="s1">&#39;E1/E0&#39;</span><span class="p">:</span><span class="n">E</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">/</span> <span class="n">E</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="s1">&#39;E2/E0&#39;</span><span class="p">:</span><span class="n">E</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">/</span> <span class="n">E</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span>
        <span class="s1">&#39;a1/a0&#39;</span><span class="p">:</span><span class="n">a</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">/</span> <span class="n">a</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="s1">&#39;a2/a0&#39;</span><span class="p">:</span><span class="n">a</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">/</span> <span class="n">a</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
        <span class="p">}</span>
    <span class="n">inputs</span> <span class="o">=</span> <span class="p">{</span>
        <span class="s1">&#39;E prior&#39;</span><span class="p">:</span><span class="n">prior</span><span class="p">[</span><span class="s1">&#39;E&#39;</span><span class="p">],</span> <span class="s1">&#39;a prior&#39;</span><span class="p">:</span><span class="n">prior</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">],</span>
        <span class="s1">&#39;svd cut&#39;</span><span class="p">:</span><span class="n">fit</span><span class="o">.</span><span class="n">correction</span><span class="p">,</span>
        <span class="p">}</span>
    <span class="nb">print</span><span class="p">(</span><span class="n">gv</span><span class="o">.</span><span class="n">fmt_values</span><span class="p">(</span><span class="n">outputs</span><span class="p">))</span>
    <span class="nb">print</span><span class="p">(</span><span class="n">gv</span><span class="o">.</span><span class="n">fmt_errorbudget</span><span class="p">(</span><span class="n">outputs</span><span class="p">,</span> <span class="n">inputs</span><span class="p">))</span>

<span class="k">def</span> <span class="nf">fcn</span><span class="p">(</span><span class="n">x</span><span class="p">,</span><span class="n">p</span><span class="p">):</span>
    <span class="n">a</span> <span class="o">=</span> <span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]</span>       <span class="c1"># array of a[i]s</span>
    <span class="n">E</span> <span class="o">=</span> <span class="n">p</span><span class="p">[</span><span class="s1">&#39;E&#39;</span><span class="p">]</span>       <span class="c1"># array of E[i]s</span>
    <span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">ai</span> <span class="o">*</span> <span class="n">np</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="o">-</span><span class="n">Ei</span><span class="o">*</span><span class="n">x</span><span class="p">)</span> <span class="k">for</span> <span class="n">ai</span><span class="p">,</span> <span class="n">Ei</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">E</span><span class="p">))</span>

<span class="k">def</span> <span class="nf">make_prior</span><span class="p">(</span><span class="n">nexp</span><span class="p">):</span>
    <span class="n">prior</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">BufferDict</span><span class="p">()</span>
    <span class="n">prior</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="n">nexp</span> <span class="o">*</span> <span class="p">[</span><span class="s1">&#39;0.5(5)&#39;</span><span class="p">])</span>
    <span class="n">dE</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="n">nexp</span> <span class="o">*</span> <span class="p">[</span><span class="s1">&#39;1.0(1)&#39;</span><span class="p">])</span>
    <span class="n">prior</span><span class="p">[</span><span class="s1">&#39;E&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">cumsum</span><span class="p">(</span><span class="n">dE</span><span class="p">)</span>
    <span class="k">return</span> <span class="n">prior</span>

<span class="k">def</span> <span class="nf">make_data</span><span class="p">():</span>
    <span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span> <span class="mf">1.</span><span class="p">,</span> <span class="mf">1.2</span><span class="p">,</span> <span class="mf">1.4</span><span class="p">,</span> <span class="mf">1.6</span><span class="p">,</span> <span class="mf">1.8</span><span class="p">,</span> <span class="mf">2.</span><span class="p">,</span> <span class="mf">2.2</span><span class="p">,</span> <span class="mf">2.4</span><span class="p">,</span> <span class="mf">2.6</span><span class="p">])</span>
    <span class="n">y</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span>
        <span class="mf">0.2740471001620033</span><span class="p">,</span>  <span class="mf">0.2056894154005132</span><span class="p">,</span>  <span class="mf">0.158389402324004</span> <span class="p">,</span>
        <span class="mf">0.1241967645280511</span><span class="p">,</span>  <span class="mf">0.0986901274726867</span><span class="p">,</span>  <span class="mf">0.0792134506060024</span><span class="p">,</span>
        <span class="mf">0.0640743982173861</span><span class="p">,</span>  <span class="mf">0.052143504367789</span> <span class="p">,</span>  <span class="mf">0.0426383022456816</span><span class="p">,</span>
        <span class="p">])</span>
    <span class="k">return</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span>

<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">&#39;__main__&#39;</span><span class="p">:</span>
    <span class="n">main</span><span class="p">()</span>
</pre></div>
</div>
<p>We loop over <code class="docutils literal notranslate"><span class="pre">nexp</span></code>, moving parameters from <code class="docutils literal notranslate"><span class="pre">ymod</span></code> back into the fit
as <code class="docutils literal notranslate"><span class="pre">nexp</span></code> increases. The output from this script is:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="o">*************************************</span> <span class="n">nexp</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.19</span> <span class="p">[</span><span class="mi">9</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.99</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">79.803</span>

<span class="n">Parameters</span><span class="p">:</span>
            <span class="n">a</span> <span class="mi">0</span>   <span class="mf">0.4067</span> <span class="p">(</span><span class="mi">32</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
            <span class="n">E</span> <span class="mi">0</span>   <span class="mf">0.9030</span> <span class="p">(</span><span class="mi">16</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">1.00</span> <span class="p">(</span><span class="mi">10</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Fit</span><span class="p">:</span>
     <span class="n">x</span><span class="p">[</span><span class="n">k</span><span class="p">]</span>           <span class="n">y</span><span class="p">[</span><span class="n">k</span><span class="p">]</span>       <span class="n">f</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="n">k</span><span class="p">],</span><span class="n">p</span><span class="p">)</span>
<span class="o">----------------------------------------</span>
        <span class="mi">1</span>     <span class="mf">0.167</span> <span class="p">(</span><span class="mi">74</span><span class="p">)</span>     <span class="mf">0.1648</span> <span class="p">(</span><span class="mi">10</span><span class="p">)</span>  
      <span class="mf">1.2</span>     <span class="mf">0.141</span> <span class="p">(</span><span class="mi">49</span><span class="p">)</span>    <span class="mf">0.13760</span> <span class="p">(</span><span class="mi">82</span><span class="p">)</span>  
      <span class="mf">1.4</span>     <span class="mf">0.118</span> <span class="p">(</span><span class="mi">32</span><span class="p">)</span>    <span class="mf">0.11487</span> <span class="p">(</span><span class="mi">65</span><span class="p">)</span>  
      <span class="mf">1.6</span>     <span class="mf">0.099</span> <span class="p">(</span><span class="mi">22</span><span class="p">)</span>    <span class="mf">0.09589</span> <span class="p">(</span><span class="mi">51</span><span class="p">)</span>  
      <span class="mf">1.8</span>     <span class="mf">0.082</span> <span class="p">(</span><span class="mi">14</span><span class="p">)</span>    <span class="mf">0.08004</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>  
        <span class="mi">2</span>    <span class="mf">0.0686</span> <span class="p">(</span><span class="mi">97</span><span class="p">)</span>    <span class="mf">0.06682</span> <span class="p">(</span><span class="mi">31</span><span class="p">)</span>  
      <span class="mf">2.2</span>    <span class="mf">0.0572</span> <span class="p">(</span><span class="mi">65</span><span class="p">)</span>    <span class="mf">0.05578</span> <span class="p">(</span><span class="mi">24</span><span class="p">)</span>  
      <span class="mf">2.4</span>    <span class="mf">0.0476</span> <span class="p">(</span><span class="mi">44</span><span class="p">)</span>    <span class="mf">0.04656</span> <span class="p">(</span><span class="mi">19</span><span class="p">)</span>  
      <span class="mf">2.6</span>    <span class="mf">0.0397</span> <span class="p">(</span><span class="mi">30</span><span class="p">)</span>    <span class="mf">0.03887</span> <span class="p">(</span><span class="mi">15</span><span class="p">)</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">2</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-10</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">11</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="o">*************************************</span> <span class="n">nexp</span> <span class="o">=</span> <span class="mi">2</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.19</span> <span class="p">[</span><span class="mi">9</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mi">1</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">81.799</span>

<span class="n">Parameters</span><span class="p">:</span>
            <span class="n">a</span> <span class="mi">0</span>   <span class="mf">0.4015</span> <span class="p">(</span><span class="mi">23</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>    <span class="mf">0.435</span> <span class="p">(</span><span class="mi">24</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
            <span class="n">E</span> <span class="mi">0</span>   <span class="mf">0.9007</span> <span class="p">(</span><span class="mi">11</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">1.00</span> <span class="p">(</span><span class="mi">10</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>    <span class="mf">1.830</span> <span class="p">(</span><span class="mi">28</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">2.00</span> <span class="p">(</span><span class="mi">14</span><span class="p">)</span> <span class="p">]</span>  <span class="o">*</span>

<span class="n">Fit</span><span class="p">:</span>
     <span class="n">x</span><span class="p">[</span><span class="n">k</span><span class="p">]</span>            <span class="n">y</span><span class="p">[</span><span class="n">k</span><span class="p">]</span>        <span class="n">f</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="n">k</span><span class="p">],</span><span class="n">p</span><span class="p">)</span>
<span class="o">------------------------------------------</span>
        <span class="mi">1</span>      <span class="mf">0.235</span> <span class="p">(</span><span class="mi">28</span><span class="p">)</span>      <span class="mf">0.2330</span> <span class="p">(</span><span class="mi">27</span><span class="p">)</span>  
      <span class="mf">1.2</span>      <span class="mf">0.186</span> <span class="p">(</span><span class="mi">15</span><span class="p">)</span>      <span class="mf">0.1847</span> <span class="p">(</span><span class="mi">17</span><span class="p">)</span>  
      <span class="mf">1.4</span>     <span class="mf">0.1484</span> <span class="p">(</span><span class="mi">81</span><span class="p">)</span>      <span class="mf">0.1474</span> <span class="p">(</span><span class="mi">10</span><span class="p">)</span>  
      <span class="mf">1.6</span>     <span class="mf">0.1190</span> <span class="p">(</span><span class="mi">44</span><span class="p">)</span>     <span class="mf">0.11833</span> <span class="p">(</span><span class="mi">63</span><span class="p">)</span>  
      <span class="mf">1.8</span>     <span class="mf">0.0960</span> <span class="p">(</span><span class="mi">24</span><span class="p">)</span>     <span class="mf">0.09552</span> <span class="p">(</span><span class="mi">38</span><span class="p">)</span>  
        <span class="mi">2</span>     <span class="mf">0.0778</span> <span class="p">(</span><span class="mi">13</span><span class="p">)</span>     <span class="mf">0.07749</span> <span class="p">(</span><span class="mi">23</span><span class="p">)</span>  
      <span class="mf">2.2</span>    <span class="mf">0.06331</span> <span class="p">(</span><span class="mi">74</span><span class="p">)</span>     <span class="mf">0.06313</span> <span class="p">(</span><span class="mi">14</span><span class="p">)</span>  
      <span class="mf">2.4</span>    <span class="mf">0.05173</span> <span class="p">(</span><span class="mi">41</span><span class="p">)</span>    <span class="mf">0.051624</span> <span class="p">(</span><span class="mi">84</span><span class="p">)</span>  
      <span class="mf">2.6</span>    <span class="mf">0.04242</span> <span class="p">(</span><span class="mi">23</span><span class="p">)</span>    <span class="mf">0.042351</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">2</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-10</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">27</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="o">*************************************</span> <span class="n">nexp</span> <span class="o">=</span> <span class="mi">3</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.2</span> <span class="p">[</span><span class="mi">9</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.99</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">83.077</span>

<span class="n">Parameters</span><span class="p">:</span>
            <span class="n">a</span> <span class="mi">0</span>    <span class="mf">0.4011</span> <span class="p">(</span><span class="mi">18</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>     <span class="mf">0.426</span> <span class="p">(</span><span class="mi">28</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">2</span>     <span class="mf">0.468</span> <span class="p">(</span><span class="mi">56</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
            <span class="n">E</span> <span class="mi">0</span>   <span class="mf">0.90045</span> <span class="p">(</span><span class="mi">77</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">1.00</span> <span class="p">(</span><span class="mi">10</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>     <span class="mf">1.822</span> <span class="p">(</span><span class="mi">27</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">2.00</span> <span class="p">(</span><span class="mi">14</span><span class="p">)</span> <span class="p">]</span>  <span class="o">*</span>
              <span class="mi">2</span>      <span class="mf">2.84</span> <span class="p">(</span><span class="mi">12</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">3.00</span> <span class="p">(</span><span class="mi">17</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Fit</span><span class="p">:</span>
     <span class="n">x</span><span class="p">[</span><span class="n">k</span><span class="p">]</span>             <span class="n">y</span><span class="p">[</span><span class="n">k</span><span class="p">]</span>         <span class="n">f</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="n">k</span><span class="p">],</span><span class="n">p</span><span class="p">)</span>
<span class="o">--------------------------------------------</span>
        <span class="mi">1</span>       <span class="mf">0.260</span> <span class="p">(</span><span class="mi">10</span><span class="p">)</span>       <span class="mf">0.2593</span> <span class="p">(</span><span class="mi">22</span><span class="p">)</span>  
      <span class="mf">1.2</span>      <span class="mf">0.1998</span> <span class="p">(</span><span class="mi">45</span><span class="p">)</span>       <span class="mf">0.1995</span> <span class="p">(</span><span class="mi">11</span><span class="p">)</span>  
      <span class="mf">1.4</span>      <span class="mf">0.1559</span> <span class="p">(</span><span class="mi">20</span><span class="p">)</span>      <span class="mf">0.15576</span> <span class="p">(</span><span class="mi">54</span><span class="p">)</span>  
      <span class="mf">1.6</span>     <span class="mf">0.12316</span> <span class="p">(</span><span class="mi">91</span><span class="p">)</span>      <span class="mf">0.12305</span> <span class="p">(</span><span class="mi">27</span><span class="p">)</span>  
      <span class="mf">1.8</span>     <span class="mf">0.09824</span> <span class="p">(</span><span class="mi">41</span><span class="p">)</span>      <span class="mf">0.09818</span> <span class="p">(</span><span class="mi">13</span><span class="p">)</span>  
        <span class="mi">2</span>     <span class="mf">0.07902</span> <span class="p">(</span><span class="mi">19</span><span class="p">)</span>     <span class="mf">0.078988</span> <span class="p">(</span><span class="mi">62</span><span class="p">)</span>  
      <span class="mf">2.2</span>    <span class="mf">0.063990</span> <span class="p">(</span><span class="mi">85</span><span class="p">)</span>     <span class="mf">0.063973</span> <span class="p">(</span><span class="mi">30</span><span class="p">)</span>  
      <span class="mf">2.4</span>    <span class="mf">0.052106</span> <span class="p">(</span><span class="mi">38</span><span class="p">)</span>     <span class="mf">0.052098</span> <span class="p">(</span><span class="mi">14</span><span class="p">)</span>  
      <span class="mf">2.6</span>    <span class="mf">0.042622</span> <span class="p">(</span><span class="mi">17</span><span class="p">)</span>    <span class="mf">0.0426176</span> <span class="p">(</span><span class="mi">68</span><span class="p">)</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">3</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-10</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">64</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="o">*************************************</span> <span class="n">nexp</span> <span class="o">=</span> <span class="mi">4</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.21</span> <span class="p">[</span><span class="mi">9</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.99</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">83.212</span>

<span class="n">Parameters</span><span class="p">:</span>
            <span class="n">a</span> <span class="mi">0</span>    <span class="mf">0.4009</span> <span class="p">(</span><span class="mi">10</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>     <span class="mf">0.424</span> <span class="p">(</span><span class="mi">22</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">2</span>     <span class="mf">0.469</span> <span class="p">(</span><span class="mi">61</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">3</span>     <span class="mf">0.426</span> <span class="p">(</span><span class="mi">94</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
            <span class="n">E</span> <span class="mi">0</span>   <span class="mf">0.90036</span> <span class="p">(</span><span class="mi">44</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">1.00</span> <span class="p">(</span><span class="mi">10</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>     <span class="mf">1.819</span> <span class="p">(</span><span class="mi">19</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">2.00</span> <span class="p">(</span><span class="mi">14</span><span class="p">)</span> <span class="p">]</span>  <span class="o">*</span>
              <span class="mi">2</span>      <span class="mf">2.83</span> <span class="p">(</span><span class="mi">11</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">3.00</span> <span class="p">(</span><span class="mi">17</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">3</span>      <span class="mf">3.83</span> <span class="p">(</span><span class="mi">15</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">4.00</span> <span class="p">(</span><span class="mi">20</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Fit</span><span class="p">:</span>
     <span class="n">x</span><span class="p">[</span><span class="n">k</span><span class="p">]</span>              <span class="n">y</span><span class="p">[</span><span class="n">k</span><span class="p">]</span>          <span class="n">f</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="n">k</span><span class="p">],</span><span class="n">p</span><span class="p">)</span>
<span class="o">----------------------------------------------</span>
        <span class="mi">1</span>       <span class="mf">0.2687</span> <span class="p">(</span><span class="mi">38</span><span class="p">)</span>       <span class="mf">0.26843</span> <span class="p">(</span><span class="mi">95</span><span class="p">)</span>  
      <span class="mf">1.2</span>       <span class="mf">0.2039</span> <span class="p">(</span><span class="mi">14</span><span class="p">)</span>       <span class="mf">0.20376</span> <span class="p">(</span><span class="mi">39</span><span class="p">)</span>  
      <span class="mf">1.4</span>      <span class="mf">0.15778</span> <span class="p">(</span><span class="mi">51</span><span class="p">)</span>       <span class="mf">0.15771</span> <span class="p">(</span><span class="mi">16</span><span class="p">)</span>  
      <span class="mf">1.6</span>      <span class="mf">0.12399</span> <span class="p">(</span><span class="mi">19</span><span class="p">)</span>      <span class="mf">0.123955</span> <span class="p">(</span><span class="mi">63</span><span class="p">)</span>  
      <span class="mf">1.8</span>     <span class="mf">0.098616</span> <span class="p">(</span><span class="mi">69</span><span class="p">)</span>      <span class="mf">0.098603</span> <span class="p">(</span><span class="mi">25</span><span class="p">)</span>  
        <span class="mi">2</span>     <span class="mf">0.079187</span> <span class="p">(</span><span class="mi">26</span><span class="p">)</span>      <span class="mf">0.079182</span> <span class="p">(</span><span class="mi">10</span><span class="p">)</span>  
      <span class="mf">2.2</span>    <span class="mf">0.0640650</span> <span class="p">(</span><span class="mi">96</span><span class="p">)</span>     <span class="mf">0.0640627</span> <span class="p">(</span><span class="mi">39</span><span class="p">)</span>  
      <span class="mf">2.4</span>    <span class="mf">0.0521401</span> <span class="p">(</span><span class="mi">36</span><span class="p">)</span>     <span class="mf">0.0521392</span> <span class="p">(</span><span class="mi">15</span><span class="p">)</span>  
      <span class="mf">2.6</span>    <span class="mf">0.0426371</span> <span class="p">(</span><span class="mi">13</span><span class="p">)</span>    <span class="mf">0.04263670</span> <span class="p">(</span><span class="mi">61</span><span class="p">)</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">3</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-10</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">140</span><span class="o">/</span><span class="mf">0.1</span><span class="p">)</span>

<span class="o">*************************************</span> <span class="n">nexp</span> <span class="o">=</span> <span class="mi">5</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.21</span> <span class="p">[</span><span class="mi">9</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.99</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">83.28</span>

<span class="n">Parameters</span><span class="p">:</span>
            <span class="n">a</span> <span class="mi">0</span>    <span class="mf">0.4009</span> <span class="p">(</span><span class="mi">10</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>     <span class="mf">0.424</span> <span class="p">(</span><span class="mi">22</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">2</span>     <span class="mf">0.468</span> <span class="p">(</span><span class="mi">62</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">3</span>      <span class="mf">0.42</span> <span class="p">(</span><span class="mi">11</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">4</span>      <span class="mf">0.45</span> <span class="p">(</span><span class="mi">18</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
            <span class="n">E</span> <span class="mi">0</span>   <span class="mf">0.90036</span> <span class="p">(</span><span class="mi">43</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">1.00</span> <span class="p">(</span><span class="mi">10</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>     <span class="mf">1.819</span> <span class="p">(</span><span class="mi">19</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">2.00</span> <span class="p">(</span><span class="mi">14</span><span class="p">)</span> <span class="p">]</span>  <span class="o">*</span>
              <span class="mi">2</span>      <span class="mf">2.83</span> <span class="p">(</span><span class="mi">11</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">3.00</span> <span class="p">(</span><span class="mi">17</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">3</span>      <span class="mf">3.83</span> <span class="p">(</span><span class="mi">15</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">4.00</span> <span class="p">(</span><span class="mi">20</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">4</span>      <span class="mf">4.83</span> <span class="p">(</span><span class="mi">18</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">5.00</span> <span class="p">(</span><span class="mi">22</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Fit</span><span class="p">:</span>
     <span class="n">x</span><span class="p">[</span><span class="n">k</span><span class="p">]</span>               <span class="n">y</span><span class="p">[</span><span class="n">k</span><span class="p">]</span>          <span class="n">f</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="n">k</span><span class="p">],</span><span class="n">p</span><span class="p">)</span>
<span class="o">-----------------------------------------------</span>
        <span class="mi">1</span>        <span class="mf">0.2721</span> <span class="p">(</span><span class="mi">14</span><span class="p">)</span>       <span class="mf">0.27196</span> <span class="p">(</span><span class="mi">64</span><span class="p">)</span>  
      <span class="mf">1.2</span>       <span class="mf">0.20516</span> <span class="p">(</span><span class="mi">42</span><span class="p">)</span>       <span class="mf">0.20510</span> <span class="p">(</span><span class="mi">21</span><span class="p">)</span>  
      <span class="mf">1.4</span>       <span class="mf">0.15824</span> <span class="p">(</span><span class="mi">13</span><span class="p">)</span>      <span class="mf">0.158219</span> <span class="p">(</span><span class="mi">68</span><span class="p">)</span>  
      <span class="mf">1.6</span>      <span class="mf">0.124154</span> <span class="p">(</span><span class="mi">38</span><span class="p">)</span>      <span class="mf">0.124147</span> <span class="p">(</span><span class="mi">22</span><span class="p">)</span>  
      <span class="mf">1.8</span>      <span class="mf">0.098678</span> <span class="p">(</span><span class="mi">12</span><span class="p">)</span>     <span class="mf">0.0986752</span> <span class="p">(</span><span class="mi">70</span><span class="p">)</span>  
        <span class="mi">2</span>     <span class="mf">0.0792099</span> <span class="p">(</span><span class="mi">36</span><span class="p">)</span>     <span class="mf">0.0792090</span> <span class="p">(</span><span class="mi">23</span><span class="p">)</span>  
      <span class="mf">2.2</span>     <span class="mf">0.0640734</span> <span class="p">(</span><span class="mi">11</span><span class="p">)</span>    <span class="mf">0.06407305</span> <span class="p">(</span><span class="mi">85</span><span class="p">)</span>  
      <span class="mf">2.4</span>    <span class="mf">0.05214320</span> <span class="p">(</span><span class="mi">33</span><span class="p">)</span>    <span class="mf">0.05214310</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>  
      <span class="mf">2.6</span>    <span class="mf">0.04263821</span> <span class="p">(</span><span class="mi">10</span><span class="p">)</span>    <span class="mf">0.04263818</span> <span class="p">(</span><span class="mi">27</span><span class="p">)</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">3</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-10</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">248</span><span class="o">/</span><span class="mf">0.1</span><span class="p">)</span>

<span class="n">Values</span><span class="p">:</span>
              <span class="n">E2</span><span class="o">/</span><span class="n">E0</span><span class="p">:</span> <span class="mf">3.15</span><span class="p">(</span><span class="mi">12</span><span class="p">)</span>            
              <span class="n">E1</span><span class="o">/</span><span class="n">E0</span><span class="p">:</span> <span class="mf">2.021</span><span class="p">(</span><span class="mi">20</span><span class="p">)</span>           
              <span class="n">a2</span><span class="o">/</span><span class="n">a0</span><span class="p">:</span> <span class="mf">1.17</span><span class="p">(</span><span class="mi">15</span><span class="p">)</span>            
              <span class="n">a1</span><span class="o">/</span><span class="n">a0</span><span class="p">:</span> <span class="mf">1.057</span><span class="p">(</span><span class="mi">52</span><span class="p">)</span>           

<span class="n">Partial</span> <span class="o">%</span> <span class="n">Errors</span><span class="p">:</span>
               <span class="n">E2</span><span class="o">/</span><span class="n">E0</span>     <span class="n">E1</span><span class="o">/</span><span class="n">E0</span>     <span class="n">a2</span><span class="o">/</span><span class="n">a0</span>     <span class="n">a1</span><span class="o">/</span><span class="n">a0</span>
<span class="o">--------------------------------------------------</span>
  <span class="n">E</span> <span class="n">prior</span><span class="p">:</span>      <span class="mf">3.87</span>      <span class="mf">0.86</span>     <span class="mf">12.10</span>      <span class="mf">4.51</span>
  <span class="n">svd</span> <span class="n">cut</span><span class="p">:</span>      <span class="mf">0.04</span>      <span class="mf">0.04</span>      <span class="mf">0.33</span>      <span class="mf">0.16</span>
  <span class="n">a</span> <span class="n">prior</span><span class="p">:</span>      <span class="mf">0.84</span>      <span class="mf">0.47</span>      <span class="mf">5.33</span>      <span class="mf">1.92</span>
<span class="o">--------------------------------------------------</span>
    <span class="n">total</span><span class="p">:</span>      <span class="mf">3.96</span>      <span class="mf">0.98</span>     <span class="mf">13.23</span>      <span class="mf">4.90</span>

</pre></div>
</div>
<p>Here we use <code class="docutils literal notranslate"><span class="pre">fit.format(True)</span></code> to print out a table of <code class="docutils literal notranslate"><span class="pre">x</span></code> and
<code class="docutils literal notranslate"><span class="pre">y</span></code> (actually <code class="docutils literal notranslate"><span class="pre">ymod</span></code>) values, together with the value of the
fit function using the best-fit parameters. There are several things
to notice:</p>
<blockquote>
<div><ul>
<li><p>Even the <code class="docutils literal notranslate"><span class="pre">nexp=1</span></code> fit, where we fit the data with just a single
exponential, gives results for the two parameters
that are accurate to 1% or better. The results don’t change much
as further terms are shifted from <code class="docutils literal notranslate"><span class="pre">ymod</span></code> to the fit function,
and stop changing completely by <code class="docutils literal notranslate"><span class="pre">nexp=4</span></code>.</p>
<p>In fact it is straightforward to prove that best-fit parameter means
and standard deviations, as well as <code class="docutils literal notranslate"><span class="pre">chi**2</span></code>, should be exactly the
same in such situations provided the fit function is linear in all fit
parameters. Here the fit function is approximately linear, given our
small standard deviations, and so results are only approximately
independent of <code class="docutils literal notranslate"><span class="pre">nexp</span></code>.</p>
</li>
<li><p><code class="docutils literal notranslate"><span class="pre">ymod</span></code> has large
uncertainties when <code class="docutils literal notranslate"><span class="pre">nexp</span></code> is small, because of the uncertainties
in the priors used to evaluate <code class="docutils literal notranslate"><span class="pre">fcn(x,</span> <span class="pre">ymod_prior)</span></code>. This
is clear from the following plots:</p>
<a class="reference internal image-reference" href="_images/eg5.png"><img alt="_images/eg5.png" src="_images/eg5.png" style="width: 80%;" /></a>
<p>The solid lines in these plot show the exact results, from <code class="docutils literal notranslate"><span class="pre">y</span></code> in the
code. The dashed lines show the fit function with the best-fit parameters
for the <code class="docutils literal notranslate"><span class="pre">nexp</span></code> terms used in each fit, and the data points show <code class="docutils literal notranslate"><span class="pre">ymod</span></code>
— these last two agree well, as expected from the excellent
<code class="docutils literal notranslate"><span class="pre">chi**2</span></code> values.
The uncertainties in different <code class="docutils literal notranslate"><span class="pre">ymod[i]</span></code>s are highly correlated
with each other
because they come from the same priors (in <code class="docutils literal notranslate"><span class="pre">ymod_prior</span></code>). These
correlations are evident in the plots and are essential to this
procedure.</p>
</li>
<li><p>Although we motivated this example by the need to deal with <code class="docutils literal notranslate"><span class="pre">y</span></code>s
having no errors, it is straightforward to apply the same ideas to
a situation where the <code class="docutils literal notranslate"><span class="pre">y</span></code>s have errors. Often in a fit we are
interested in only one or two of many fit parameters. Getting rid
of the uninteresting parameters (by absorbing them into <code class="docutils literal notranslate"><span class="pre">ymod</span></code>)
can greatly reduce the number of parameters varied by the fit,
thereby speeding up the fit. Here we are in effect doing a
100-exponential fit to our data, but actually fitting with only
a handful of parameters (only 2 for <code class="docutils literal notranslate"><span class="pre">nexp=1</span></code>). The parameters
removed in this way are said to be <em>marginalized</em>.</p></li>
</ul>
</div></blockquote>
</section>
<section id="svd-cuts-and-roundoff-error">
<h2>SVD Cuts and Roundoff Error<a class="headerlink" href="#svd-cuts-and-roundoff-error" title="Permalink to this heading">¶</a></h2>
<p>All of the fits discussed above have (default) SVD cuts of 1e-12. This
has little impact in most of the problems, but makes a big difference
in the problem discussed in the previous section. Had we run that fit,
for example, with an SVD cut of 1e-19, instead of 1e-12, we would have
obtained the following output:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="o">*************************************</span> <span class="n">nexp</span> <span class="o">=</span> <span class="mi">5</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.21</span> <span class="p">[</span><span class="mi">9</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.99</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">85.403</span>

<span class="n">Parameters</span><span class="p">:</span>
            <span class="n">a</span> <span class="mi">0</span>    <span class="mf">0.4009</span> <span class="p">(</span><span class="mi">10</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>     <span class="mf">0.424</span> <span class="p">(</span><span class="mi">22</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">2</span>     <span class="mf">0.469</span> <span class="p">(</span><span class="mi">62</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">3</span>      <span class="mf">0.42</span> <span class="p">(</span><span class="mi">11</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">4</span>      <span class="mf">0.46</span> <span class="p">(</span><span class="mi">18</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.50</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
            <span class="n">E</span> <span class="mi">0</span>   <span class="mf">0.90036</span> <span class="p">(</span><span class="mi">43</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">1.00</span> <span class="p">(</span><span class="mi">10</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>     <span class="mf">1.819</span> <span class="p">(</span><span class="mi">19</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">2.00</span> <span class="p">(</span><span class="mi">14</span><span class="p">)</span> <span class="p">]</span>  <span class="o">*</span>
              <span class="mi">2</span>      <span class="mf">2.83</span> <span class="p">(</span><span class="mi">11</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">3.00</span> <span class="p">(</span><span class="mi">17</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">3</span>      <span class="mf">3.83</span> <span class="p">(</span><span class="mi">15</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">4.00</span> <span class="p">(</span><span class="mi">20</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">4</span>      <span class="mf">4.83</span> <span class="p">(</span><span class="mi">18</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">5.00</span> <span class="p">(</span><span class="mi">22</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Fit</span><span class="p">:</span>
     <span class="n">x</span><span class="p">[</span><span class="n">k</span><span class="p">]</span>               <span class="n">y</span><span class="p">[</span><span class="n">k</span><span class="p">]</span>      <span class="n">f</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="n">k</span><span class="p">],</span><span class="n">p</span><span class="p">)</span>
<span class="o">-------------------------------------------</span>
        <span class="mi">1</span>        <span class="mf">0.2721</span> <span class="p">(</span><span class="mi">14</span><span class="p">)</span>     <span class="mf">0.272</span> <span class="p">(</span><span class="mi">69</span><span class="p">)</span>  
      <span class="mf">1.2</span>       <span class="mf">0.20516</span> <span class="p">(</span><span class="mi">42</span><span class="p">)</span>     <span class="mf">0.205</span> <span class="p">(</span><span class="mi">57</span><span class="p">)</span>  
      <span class="mf">1.4</span>       <span class="mf">0.15824</span> <span class="p">(</span><span class="mi">13</span><span class="p">)</span>     <span class="mf">0.158</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>  
      <span class="mf">1.6</span>      <span class="mf">0.124154</span> <span class="p">(</span><span class="mi">38</span><span class="p">)</span>     <span class="mf">0.124</span> <span class="p">(</span><span class="mi">27</span><span class="p">)</span>  
      <span class="mf">1.8</span>      <span class="mf">0.098678</span> <span class="p">(</span><span class="mi">12</span><span class="p">)</span>     <span class="mf">0.099</span> <span class="p">(</span><span class="mi">18</span><span class="p">)</span>  
        <span class="mi">2</span>     <span class="mf">0.0792099</span> <span class="p">(</span><span class="mi">36</span><span class="p">)</span>     <span class="mf">0.079</span> <span class="p">(</span><span class="mi">11</span><span class="p">)</span>  
      <span class="mf">2.2</span>     <span class="mf">0.0640734</span> <span class="p">(</span><span class="mi">11</span><span class="p">)</span>    <span class="mf">0.0641</span> <span class="p">(</span><span class="mi">70</span><span class="p">)</span>  
      <span class="mf">2.4</span>    <span class="mf">0.05214320</span> <span class="p">(</span><span class="mi">33</span><span class="p">)</span>    <span class="mf">0.0521</span> <span class="p">(</span><span class="mi">43</span><span class="p">)</span>  
      <span class="mf">2.6</span>    <span class="mf">0.04263821</span> <span class="p">(</span><span class="mi">10</span><span class="p">)</span>    <span class="mf">0.0426</span> <span class="p">(</span><span class="mi">25</span><span class="p">)</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-19</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-10</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">309</span><span class="o">/</span><span class="mf">0.1</span><span class="p">)</span>

<span class="n">Values</span><span class="p">:</span>
              <span class="n">E2</span><span class="o">/</span><span class="n">E0</span><span class="p">:</span> <span class="mf">3.1</span><span class="p">(</span><span class="mf">2.7</span><span class="p">)</span>            
              <span class="n">E1</span><span class="o">/</span><span class="n">E0</span><span class="p">:</span> <span class="mf">2.02</span><span class="p">(</span><span class="mi">45</span><span class="p">)</span>            
              <span class="n">a2</span><span class="o">/</span><span class="n">a0</span><span class="p">:</span> <span class="mf">1.2</span><span class="p">(</span><span class="mf">5.3</span><span class="p">)</span>            
              <span class="n">a1</span><span class="o">/</span><span class="n">a0</span><span class="p">:</span> <span class="mf">1.1</span><span class="p">(</span><span class="mf">1.4</span><span class="p">)</span>            

<span class="n">Partial</span> <span class="o">%</span> <span class="n">Errors</span><span class="p">:</span>
               <span class="n">E2</span><span class="o">/</span><span class="n">E0</span>     <span class="n">E1</span><span class="o">/</span><span class="n">E0</span>     <span class="n">a2</span><span class="o">/</span><span class="n">a0</span>     <span class="n">a1</span><span class="o">/</span><span class="n">a0</span>
<span class="o">--------------------------------------------------</span>
  <span class="n">E</span> <span class="n">prior</span><span class="p">:</span>     <span class="mf">54.30</span>      <span class="mf">8.06</span>    <span class="mf">256.81</span>     <span class="mf">56.39</span>
  <span class="n">svd</span> <span class="n">cut</span><span class="p">:</span>      <span class="mf">0.00</span>      <span class="mf">0.00</span>      <span class="mf">0.00</span>      <span class="mf">0.00</span>
  <span class="n">a</span> <span class="n">prior</span><span class="p">:</span>     <span class="mf">67.92</span>     <span class="mf">20.58</span>    <span class="mf">376.32</span>    <span class="mf">124.24</span>
<span class="o">--------------------------------------------------</span>
    <span class="n">total</span><span class="p">:</span>     <span class="mf">86.96</span>     <span class="mf">22.10</span>    <span class="mf">455.59</span>    <span class="mf">136.44</span>

</pre></div>
</div>
<p>The standard deviations quoted for <code class="docutils literal notranslate"><span class="pre">E1/E0</span></code>, <em>etc.</em> are much too large
compared with the standard deviations than what we obtained
in the previous section.
This is due to roundoff error. The strong correlations between the
different data points (<code class="docutils literal notranslate"><span class="pre">ymod[i]</span></code> — see the previous section) in this
analysis result
in a data covariance matrix that is too ill-conditioned without an SVD cut.</p>
<p>The inverse of the data’s covariance matrix is used in the <code class="docutils literal notranslate"><span class="pre">chi**2</span></code>
function that is minimized by <a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a>. Given the
finite precision of computer hardware, it is impossible to compute this
inverse accurately if the matrix is almost singular, and in
such situations the reliability of the fit results is in question. The
eigenvalues of the covariance matrix in this example (for <code class="docutils literal notranslate"><span class="pre">nexp=5</span></code>)
cover a range of about 18 orders of magnitude — too large
to be handled in normal double precision computation.
The smallest eigenvalues and their
eigenvectors are likely to be quite inaccurate.</p>
<p>A standard solution to this common problem in least-squares fitting is
to introduce an SVD cut, here called <code class="docutils literal notranslate"><span class="pre">svdcut</span></code>:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">fit</span> <span class="o">=</span> <span class="n">nonlinear_fit</span><span class="p">(</span><span class="n">data</span><span class="o">=</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">ymod</span><span class="p">),</span> <span class="n">fcn</span><span class="o">=</span><span class="n">f</span><span class="p">,</span> <span class="n">prior</span><span class="o">=</span><span class="n">prior</span><span class="p">,</span> <span class="n">p0</span><span class="o">=</span><span class="n">p0</span><span class="p">,</span> <span class="n">svdcut</span><span class="o">=</span><span class="mf">1e-12</span><span class="p">)</span>
</pre></div>
</div>
<p>This regulates the singularity of the covariance matrix by
replacing its smallest eigenvalues with a larger, minimum
eigenvalue. The cost is less precision in the final results
since we are decreasing the
precision of the input <code class="docutils literal notranslate"><span class="pre">y</span></code> data. This is a conservative move, but numerical
stability is worth the trade off. The listing shows that 3 eigenvalues are
modified when <code class="docutils literal notranslate"><span class="pre">svdcut=1e-12</span></code> (see entry for <code class="docutils literal notranslate"><span class="pre">svdcut/n</span></code>); no
eigenvalues are changed when <code class="docutils literal notranslate"><span class="pre">svdcut=1e-19</span></code>.</p>
<p>The SVD cut is actually applied to the correlation matrix, which is the
covariance matrix rescaled by standard deviations so that  all diagonal
elements equal 1. Working with the correlation matrix rather than the
covariance matrix helps mitigate problems caused by large scale differences
between different variables. Eigenvalues of the correlation matrix that are
smaller than a minimum eigenvalue, equal to <code class="docutils literal notranslate"><span class="pre">svdcut</span></code> times the largest
eigenvalue,  are replaced by the minimum eigenvalue, while leaving their
eigenvectors unchanged. This defines a new, less singular correlation matrix
from which a new, less singular covariance matrix is constructed. Larger
values of <code class="docutils literal notranslate"><span class="pre">svdcut</span></code> affect larger numbers of eigenmodes and increase errors
in the final results.</p>
<p>The results shown in the previous section include an error budget, and it
has an entry for the error introduced by the (default) SVD cut (obtained
from <code class="docutils literal notranslate"><span class="pre">fit.correction</span></code>).
The contribution is negligible. It is zero when <code class="docutils literal notranslate"><span class="pre">svdcut=1e-19</span></code>, of course,
but the instability caused by the ill-conditioned covariance matrix in
that case makes it unacceptable.</p>
<p>The SVD cut is applied separately to each block diagonal sub-matrix of the
correlation matrix. This means, among other things, that errors for
uncorrelated data are unaffected by the SVD cut. Applying an SVD
cut of 1e-4, for example, to the following singular covariance matrix,</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="p">[[</span>  <span class="mf">1.0</span>   <span class="mf">1.0</span>   <span class="mf">0.0</span>  <span class="p">]</span>
 <span class="p">[</span>  <span class="mf">1.0</span>   <span class="mf">1.0</span>   <span class="mf">0.0</span>  <span class="p">]</span>
 <span class="p">[</span>  <span class="mf">0.0</span>   <span class="mf">0.0</span>   <span class="mf">1e-20</span><span class="p">]],</span>
</pre></div>
</div>
<p>gives a new, non-singular matrix</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="p">[[</span>  <span class="mf">1.0001</span>   <span class="mf">0.9999</span>   <span class="mf">0.0</span>  <span class="p">]</span>
 <span class="p">[</span>  <span class="mf">0.9999</span>   <span class="mf">1.0001</span>   <span class="mf">0.0</span>  <span class="p">]</span>
 <span class="p">[</span>  <span class="mf">0.0</span>      <span class="mf">0.0</span>      <span class="mf">1e-20</span><span class="p">]]</span>
</pre></div>
</div>
<p>where only the upper left sub-matrix is different.</p>
<p><a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a> uses a default value for <code class="docutils literal notranslate"><span class="pre">svdcut</span></code> of 1e-12.
This default can be overridden, as shown above, but for many
problems it is a good choice. Roundoff errors become more accute, however,
when there are strong correlations between different parts of the fit
data or prior.  Then much larger <code class="docutils literal notranslate"><span class="pre">svdcut</span></code>s may be needed.</p>
<p>The SVD cut is applied to both the data and the prior. It is possible to
apply SVD cuts to either of these separately using <code class="xref py py-func docutils literal notranslate"><span class="pre">gvar.regulate()</span></code> before
the fit: for example,</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">y</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">regulate</span><span class="p">(</span><span class="n">ymod</span><span class="p">,</span> <span class="n">svdcut</span><span class="o">=</span><span class="mf">1e-10</span><span class="p">)</span>
<span class="n">prior</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">regulate</span><span class="p">(</span><span class="n">prior</span><span class="p">,</span> <span class="n">svdcut</span><span class="o">=</span><span class="mf">1e-12</span><span class="p">)</span>
<span class="n">fit</span> <span class="o">=</span> <span class="n">nonlinear_fit</span><span class="p">(</span><span class="n">data</span><span class="o">=</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">),</span> <span class="n">fcn</span><span class="o">=</span><span class="n">f</span><span class="p">,</span> <span class="n">prior</span><span class="o">=</span><span class="n">prior</span><span class="p">,</span> <span class="n">svdcut</span><span class="o">=</span><span class="kc">None</span><span class="p">)</span>
</pre></div>
</div>
<p>applies different SVD cuts to the prior and data.</p>
<p>Note that taking <code class="docutils literal notranslate"><span class="pre">svdcut=-1e-12</span></code>, with a
minus sign, causes the problematic modes to be dropped. This is a more
conventional implementation of SVD cuts, but here it results in much less
precision than using <code class="docutils literal notranslate"><span class="pre">svdcut=1e-12</span></code> (giving, for example, 2.094(94)
for <code class="docutils literal notranslate"><span class="pre">E1/E0</span></code>, which is almost five times less precise). Dropping modes is
equivalent to setting the corresponding variances to infinity, which is
(obviously) much more conservative and less realistic than setting them equal
to the SVD-cutoff variance.</p>
<p>The method <a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit.check_roundoff" title="lsqfit.nonlinear_fit.check_roundoff"><code class="xref py py-func docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit.check_roundoff()</span></code></a> can be used to check
for roundoff errors by adding the line <code class="docutils literal notranslate"><span class="pre">fit.check_roundoff()</span></code> after the
fit. It generates a warning if roundoff looks to be a problem. This check
is done automatically if <code class="docutils literal notranslate"><span class="pre">debug=True</span></code> is added to the argument list of
<a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a>.</p>
</section>
<section id="svd-cuts-and-inadequate-statistics">
<span id="svd-cuts-statistics"></span><h2>SVD Cuts and Inadequate Statistics<a class="headerlink" href="#svd-cuts-and-inadequate-statistics" title="Permalink to this heading">¶</a></h2>
<p>Roundoff error is one reason to use SVD cuts. Another is inadequate
statistics. Consider the following example, which seeks to fit
data obtained by averaging <code class="docutils literal notranslate"><span class="pre">N=5</span></code> random samples (a very small
number of samples):</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">import</span> <span class="nn">gvar</span> <span class="k">as</span> <span class="nn">gv</span>
<span class="kn">import</span> <span class="nn">lsqfit</span>

<span class="k">def</span> <span class="nf">main</span><span class="p">():</span>
    <span class="n">ysamples</span> <span class="o">=</span> <span class="p">[</span>
        <span class="p">[</span><span class="mf">0.0092441016</span><span class="p">,</span> <span class="mf">0.0068974057</span><span class="p">,</span> <span class="mf">0.0051480509</span><span class="p">,</span> <span class="mf">0.0038431422</span><span class="p">,</span> <span class="mf">0.0028690492</span><span class="p">],</span>
        <span class="p">[</span><span class="mf">0.0092477405</span><span class="p">,</span> <span class="mf">0.0069030565</span><span class="p">,</span> <span class="mf">0.0051531383</span><span class="p">,</span> <span class="mf">0.0038455855</span><span class="p">,</span> <span class="mf">0.0028700587</span><span class="p">],</span>
        <span class="p">[</span><span class="mf">0.0092558569</span><span class="p">,</span> <span class="mf">0.0069102437</span><span class="p">,</span> <span class="mf">0.0051596569</span><span class="p">,</span> <span class="mf">0.0038514537</span><span class="p">,</span> <span class="mf">0.0028749153</span><span class="p">],</span>
        <span class="p">[</span><span class="mf">0.0092294581</span><span class="p">,</span> <span class="mf">0.0068865156</span><span class="p">,</span> <span class="mf">0.0051395262</span><span class="p">,</span> <span class="mf">0.003835656</span><span class="p">,</span> <span class="mf">0.0028630454</span><span class="p">],</span>
        <span class="p">[</span><span class="mf">0.009240534</span><span class="p">,</span> <span class="mf">0.0068961523</span><span class="p">,</span> <span class="mf">0.0051480046</span><span class="p">,</span> <span class="mf">0.0038424661</span><span class="p">,</span> <span class="mf">0.0028675632</span><span class="p">],</span>
        <span class="p">]</span>
    <span class="n">y</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">dataset</span><span class="o">.</span><span class="n">avg_data</span><span class="p">(</span><span class="n">ysamples</span><span class="p">)</span>
    <span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">15.</span><span class="p">,</span> <span class="mf">16.</span><span class="p">,</span> <span class="mf">17.</span><span class="p">,</span> <span class="mf">18.</span><span class="p">,</span> <span class="mf">19.</span><span class="p">])</span>
    <span class="k">def</span> <span class="nf">fcn</span><span class="p">(</span><span class="n">p</span><span class="p">):</span>
        <span class="k">return</span> <span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]</span> <span class="o">*</span> <span class="n">gv</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="o">-</span> <span class="n">p</span><span class="p">[</span><span class="s1">&#39;b&#39;</span><span class="p">]</span> <span class="o">*</span> <span class="n">x</span><span class="p">)</span>
    <span class="n">prior</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">a</span><span class="o">=</span><span class="s1">&#39;0.75(5)&#39;</span><span class="p">,</span> <span class="n">b</span><span class="o">=</span><span class="s1">&#39;0.30(3)&#39;</span><span class="p">)</span>
    <span class="n">fit</span> <span class="o">=</span> <span class="n">lsqfit</span><span class="o">.</span><span class="n">nonlinear_fit</span><span class="p">(</span><span class="n">data</span><span class="o">=</span><span class="n">y</span><span class="p">,</span> <span class="n">prior</span><span class="o">=</span><span class="n">prior</span><span class="p">,</span> <span class="n">fcn</span><span class="o">=</span><span class="n">fcn</span><span class="p">,</span> <span class="n">svdcut</span><span class="o">=</span><span class="mf">0.0</span><span class="p">)</span>
    <span class="nb">print</span><span class="p">(</span><span class="n">fit</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="kc">True</span><span class="p">))</span>

<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">&#39;__main__&#39;</span><span class="p">:</span>
    <span class="n">main</span><span class="p">()</span>
</pre></div>
</div>
<p>This gives a terrible fit:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">4.4e+03</span> <span class="p">[</span><span class="mi">5</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mi">0</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="o">-</span><span class="mi">10844</span>

<span class="n">Parameters</span><span class="p">:</span>
              <span class="n">a</span>    <span class="mf">0.77794</span> <span class="p">(</span><span class="mi">26</span><span class="p">)</span>      <span class="p">[</span> <span class="mf">0.750</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="n">b</span>   <span class="mf">0.296452</span> <span class="p">(</span><span class="mi">32</span><span class="p">)</span>      <span class="p">[</span> <span class="mf">0.300</span> <span class="p">(</span><span class="mi">30</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mi">0</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">4</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

</pre></div>
</div>
<p>The problem is that the small eigenvalues of the fit data’s correlation
matrix are badly underestimated when we have only a small number of samples
(compared with the number of data points being fit). Indeed the smallest
eigenvalues vanish (exactly) when the number of samples is smaller
than the number of data points.</p>
<p>An SVD cut is needed. We use <code class="xref py py-func docutils literal notranslate"><span class="pre">gvar.dataset.svd_analysis()</span></code> to estimate
the appropriate size of the cut (by means of a bootstrap simulation that
identifies
the eigenmodes of the correlation matrix  that are poorly estimated from
our data sample). To use it we replace the line</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">y</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">dataset</span><span class="o">.</span><span class="n">avg_data</span><span class="p">(</span><span class="n">ysamples</span><span class="p">)</span>
</pre></div>
</div>
<p>in the code by</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">s</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">dataset</span><span class="o">.</span><span class="n">svd_diagnosis</span><span class="p">(</span><span class="n">ysamples</span><span class="p">)</span>
<span class="n">y</span> <span class="o">=</span> <span class="n">s</span><span class="o">.</span><span class="n">avgdata</span>
</pre></div>
</div>
<p>and set <code class="docutils literal notranslate"><span class="pre">svdcut=s.svdcut</span></code> in the call to the fitter. The result is
the following (excellent) fit:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.38</span> <span class="p">[</span><span class="mi">5</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.86</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">53.598</span>

<span class="n">Parameters</span><span class="p">:</span>
              <span class="n">a</span>    <span class="mf">0.74338</span> <span class="p">(</span><span class="mi">29</span><span class="p">)</span>      <span class="p">[</span> <span class="mf">0.750</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span> <span class="p">]</span>  
              <span class="n">b</span>   <span class="mf">0.292480</span> <span class="p">(</span><span class="mi">42</span><span class="p">)</span>      <span class="p">[</span> <span class="mf">0.300</span> <span class="p">(</span><span class="mi">30</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">0.0028</span><span class="o">/</span><span class="mi">3</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">5</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

</pre></div>
</div>
<p>The SVD cut is set to <code class="docutils literal notranslate"><span class="pre">0.0028</span></code> here and modifies 3 of the 5 eigenmodes in the
correlation matrix. Generally one needs an SVD cut unless there
are many more samples than data points — 10 or 100 times as many.</p>
<p>See the discussions in <a class="reference internal" href="testing.html#goodness-of-fit"><span class="std std-ref">Goodness of Fit</span></a> and <a class="reference internal" href="testing.html#fit-residuals"><span class="std std-ref">Fit Residuals and Q-Q Plots</span></a> for further
analysis of this example.</p>
<p><a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a> will apply an SVD cut if keyword parameter
<code class="docutils literal notranslate"><span class="pre">svdcut</span></code> is set. Another way to implement SVD cuts is using
<code class="xref py py-meth docutils literal notranslate"><span class="pre">gvar.regulate()</span></code> to modify the fit data before it is fit.</p>
</section>
<section id="y-has-unknown-errors">
<h2><code class="docutils literal notranslate"><span class="pre">y</span></code> has Unknown Errors<a class="headerlink" href="#y-has-unknown-errors" title="Permalink to this heading">¶</a></h2>
<p>There are situations where the input data <code class="docutils literal notranslate"><span class="pre">y</span></code> is known to have
uncertainties, but where we do not know how big those uncertainties are.
A common approach is to infer these uncertainties from the fluctuations
of the data around the best-fit result.</p>
<p>As an example, consider the following data:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">1.</span><span class="p">,</span> <span class="mf">2.</span><span class="p">,</span> <span class="mf">3.</span><span class="p">,</span> <span class="mf">4.</span><span class="p">])</span>
<span class="n">y</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">3.4422</span><span class="p">,</span> <span class="mf">1.2929</span><span class="p">,</span> <span class="mf">0.4798</span><span class="p">,</span> <span class="mf">0.1725</span><span class="p">])</span>
</pre></div>
</div>
<p>We want to fit these data with a simple exponential:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">p</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">*</span> <span class="n">gv</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span> <span class="o">-</span> <span class="n">p</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">*</span> <span class="n">x</span><span class="p">)</span>
</pre></div>
</div>
<p>where from we know <em>a priori</em> that <code class="docutils literal notranslate"><span class="pre">p[0]</span></code> is 10±1 and
<code class="docutils literal notranslate"><span class="pre">p[1]</span></code> is 1±0.1. We assume that the relative uncertainty in
<code class="docutils literal notranslate"><span class="pre">y</span></code> is <code class="docutils literal notranslate"><span class="pre">x</span></code>-independent and uncorrelated.</p>
<p>Our strategy is to introduce a relative error for the data and to vary
its size to maximize the <code class="docutils literal notranslate"><span class="pre">logGBF</span></code> that results from a fit to our
exponential. The choice that maximizes the
Bayes Factor is the one that is favored by the data. This procedure
is called the <em>Empirical Bayes</em> method.</p>
<p>This method is implemented in a driver program</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">fit</span><span class="p">,</span> <span class="n">z</span> <span class="o">=</span> <span class="n">lsqfit</span><span class="o">.</span><span class="n">empbayes_fit</span><span class="p">(</span><span class="n">z0</span><span class="p">,</span> <span class="n">fitargs</span><span class="p">)</span>
</pre></div>
</div>
<p>which varies parameter <code class="docutils literal notranslate"><span class="pre">z</span></code>, starting at <code class="docutils literal notranslate"><span class="pre">z0</span></code>, to maximize
<code class="docutils literal notranslate"><span class="pre">fit.logGBF</span></code> where</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">fit</span> <span class="o">=</span> <span class="n">lsqfit</span><span class="o">.</span><span class="n">nonlinear_fit</span><span class="p">(</span><span class="o">**</span><span class="n">fitargs</span><span class="p">(</span><span class="n">z</span><span class="p">))</span><span class="o">.</span>
</pre></div>
</div>
<p>Function <code class="docutils literal notranslate"><span class="pre">fitargs(z)</span></code> returns a dictionary containing the arguments for
<a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a>. These arguments are
varied as functions of <code class="docutils literal notranslate"><span class="pre">z</span></code>. The optimal fit (that is, the one for which
<code class="docutils literal notranslate"><span class="pre">fit.logGBF</span></code> is maximum) and <code class="docutils literal notranslate"><span class="pre">z</span></code> are returned.</p>
<p>Here we want to vary the relative error assigned to the data values,
so we use the following code, where the uncertainty in <code class="docutils literal notranslate"><span class="pre">y[i]</span></code> is set
equal to <code class="docutils literal notranslate"><span class="pre">dy[i]</span> <span class="pre">=</span> <span class="pre">y[i]</span> <span class="pre">*</span> <span class="pre">z</span></code>:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">import</span> <span class="nn">gvar</span> <span class="k">as</span> <span class="nn">gv</span>
<span class="kn">import</span> <span class="nn">lsqfit</span>

<span class="c1"># fit data and prior</span>
<span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">1.</span><span class="p">,</span> <span class="mf">2.</span><span class="p">,</span> <span class="mf">3.</span><span class="p">,</span> <span class="mf">4.</span><span class="p">])</span>
<span class="n">y</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">3.4422</span><span class="p">,</span> <span class="mf">1.2929</span><span class="p">,</span> <span class="mf">0.4798</span><span class="p">,</span> <span class="mf">0.1725</span><span class="p">])</span>
<span class="n">prior</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">([</span><span class="s1">&#39;10(1)&#39;</span><span class="p">,</span> <span class="s1">&#39;1.0(1)&#39;</span><span class="p">])</span>

<span class="c1"># fit function</span>
<span class="k">def</span> <span class="nf">fcn</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">p</span><span class="p">):</span>
    <span class="k">return</span> <span class="n">p</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">*</span> <span class="n">gv</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="o">-</span><span class="n">p</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">*</span> <span class="n">x</span><span class="p">)</span>

<span class="c1"># find optimal dy</span>
<span class="k">def</span> <span class="nf">fitargs</span><span class="p">(</span><span class="n">z</span><span class="p">):</span>
    <span class="n">dy</span> <span class="o">=</span> <span class="n">y</span> <span class="o">*</span> <span class="n">z</span>
    <span class="n">newy</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">dy</span><span class="p">)</span>
    <span class="k">return</span> <span class="nb">dict</span><span class="p">(</span><span class="n">data</span><span class="o">=</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">newy</span><span class="p">),</span> <span class="n">fcn</span><span class="o">=</span><span class="n">fcn</span><span class="p">,</span> <span class="n">prior</span><span class="o">=</span><span class="n">prior</span><span class="p">)</span>

<span class="n">fit</span><span class="p">,</span> <span class="n">z</span> <span class="o">=</span> <span class="n">lsqfit</span><span class="o">.</span><span class="n">empbayes_fit</span><span class="p">(</span><span class="mf">0.001</span><span class="p">,</span> <span class="n">fitargs</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">fit</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="kc">True</span><span class="p">))</span>
</pre></div>
</div>
<p>This code produces the following output:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.59</span> <span class="p">[</span><span class="mi">4</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.67</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">7.4834</span>

<span class="n">Parameters</span><span class="p">:</span>
              <span class="mi">0</span>     <span class="mf">9.44</span> <span class="p">(</span><span class="mi">18</span><span class="p">)</span>     <span class="p">[</span> <span class="mf">10.0</span> <span class="p">(</span><span class="mf">1.0</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>   <span class="mf">0.9978</span> <span class="p">(</span><span class="mi">68</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">1.00</span> <span class="p">(</span><span class="mi">10</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Fit</span><span class="p">:</span>
     <span class="n">x</span><span class="p">[</span><span class="n">k</span><span class="p">]</span>           <span class="n">y</span><span class="p">[</span><span class="n">k</span><span class="p">]</span>      <span class="n">f</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="n">k</span><span class="p">],</span><span class="n">p</span><span class="p">)</span>
<span class="o">---------------------------------------</span>
        <span class="mi">1</span>     <span class="mf">3.442</span> <span class="p">(</span><span class="mi">54</span><span class="p">)</span>     <span class="mf">3.481</span> <span class="p">(</span><span class="mi">45</span><span class="p">)</span>  
        <span class="mi">2</span>     <span class="mf">1.293</span> <span class="p">(</span><span class="mi">20</span><span class="p">)</span>     <span class="mf">1.283</span> <span class="p">(</span><span class="mi">11</span><span class="p">)</span>  
        <span class="mi">3</span>    <span class="mf">0.4798</span> <span class="p">(</span><span class="mi">75</span><span class="p">)</span>    <span class="mf">0.4731</span> <span class="p">(</span><span class="mi">40</span><span class="p">)</span>  
        <span class="mi">4</span>    <span class="mf">0.1725</span> <span class="p">(</span><span class="mi">27</span><span class="p">)</span>    <span class="mf">0.1744</span> <span class="p">(</span><span class="mi">23</span><span class="p">)</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">3</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

</pre></div>
</div>
<p>The variation in the data suggests a relative error of about 1.6%
for the input data. The overall fit is excellent.</p>
<p>It is important to appreciate that the outcome of a such a fit depends
in detail on the assumptions you make about <code class="docutils literal notranslate"><span class="pre">y</span></code>’s uncertainties <code class="docutils literal notranslate"><span class="pre">dy</span></code>.
We assume <code class="docutils literal notranslate"><span class="pre">dy/y</span></code> is <code class="docutils literal notranslate"><span class="pre">x</span></code>-independent above, but we get a somewhat different
answer if instead we assume that <code class="docutils literal notranslate"><span class="pre">dy</span></code> is constant. Then <code class="docutils literal notranslate"><span class="pre">fitrargs</span></code>
becomes</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">fitargs</span><span class="p">(</span><span class="n">z</span><span class="p">):</span>
    <span class="n">dy</span> <span class="o">=</span>  <span class="n">np</span><span class="o">.</span><span class="n">ones_like</span><span class="p">(</span><span class="n">y</span><span class="p">)</span> <span class="o">*</span> <span class="n">z</span>
    <span class="n">newy</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">dy</span><span class="p">)</span>
    <span class="k">return</span> <span class="nb">dict</span><span class="p">(</span><span class="n">data</span><span class="o">=</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">newy</span><span class="p">),</span> <span class="n">fcn</span><span class="o">=</span><span class="n">fcn</span><span class="p">,</span> <span class="n">prior</span><span class="o">=</span><span class="n">prior</span><span class="p">)</span>
</pre></div>
</div>
<p>and the output is:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.67</span> <span class="p">[</span><span class="mi">4</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.61</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">7.7643</span>

<span class="n">Parameters</span><span class="p">:</span>
              <span class="mi">0</span>    <span class="mf">9.207</span> <span class="p">(</span><span class="mi">47</span><span class="p">)</span>     <span class="p">[</span> <span class="mf">10.0</span> <span class="p">(</span><span class="mf">1.0</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>   <span class="mf">0.9834</span> <span class="p">(</span><span class="mi">42</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">1.00</span> <span class="p">(</span><span class="mi">10</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Fit</span><span class="p">:</span>
     <span class="n">x</span><span class="p">[</span><span class="n">k</span><span class="p">]</span>           <span class="n">y</span><span class="p">[</span><span class="n">k</span><span class="p">]</span>      <span class="n">f</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="n">k</span><span class="p">],</span><span class="n">p</span><span class="p">)</span>
<span class="o">---------------------------------------</span>
        <span class="mi">1</span>    <span class="mf">3.4422</span> <span class="p">(</span><span class="mi">66</span><span class="p">)</span>    <span class="mf">3.4435</span> <span class="p">(</span><span class="mi">66</span><span class="p">)</span>  
        <span class="mi">2</span>    <span class="mf">1.2929</span> <span class="p">(</span><span class="mi">66</span><span class="p">)</span>    <span class="mf">1.2879</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span>  
        <span class="mi">3</span>    <span class="mf">0.4798</span> <span class="p">(</span><span class="mi">66</span><span class="p">)</span>    <span class="mf">0.4817</span> <span class="p">(</span><span class="mi">38</span><span class="p">)</span>  
        <span class="mi">4</span>    <span class="mf">0.1725</span> <span class="p">(</span><span class="mi">66</span><span class="p">)</span>    <span class="mf">0.1802</span> <span class="p">(</span><span class="mi">22</span><span class="p">)</span>  <span class="o">*</span>

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">3</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

</pre></div>
</div>
<p>The data suggest an uncertainty of 0.0066 in each <code class="docutils literal notranslate"><span class="pre">y[i]</span></code>.
Results for the fit parameters <code class="docutils literal notranslate"><span class="pre">fit.p[i]</span></code> are similar in the two cases,
but the error on <code class="docutils literal notranslate"><span class="pre">p[0]</span></code> is almost four times smaller with
constant <code class="docutils literal notranslate"><span class="pre">dy</span></code>.</p>
<p>There is no way to tell from the data which of these error scenarios for <code class="docutils literal notranslate"><span class="pre">y</span></code> is correct. <code class="docutils literal notranslate"><span class="pre">logGBF</span></code> is slightly larger for the second fit,
despite its larger <code class="docutils literal notranslate"><span class="pre">chi2/dof</span></code>, but the difference is not significant.
There isn’t enough data and it doesn’t cover a large enough range to
distinguish between these two options. Additional information about
the data or data taking is needed to decide.</p>
<p>The Empirical Bayes method for setting <code class="docutils literal notranslate"><span class="pre">dy</span></code> becomes trivial when there
are no priors and when <code class="docutils literal notranslate"><span class="pre">dy</span></code> is assumed to be <code class="docutils literal notranslate"><span class="pre">x</span></code>-independent. Then it is
possible to minimize the <em>chi**2</em> function without knowing <code class="docutils literal notranslate"><span class="pre">dy</span></code>, since
<code class="docutils literal notranslate"><span class="pre">dy</span></code> factors out. The
optimal <code class="docutils literal notranslate"><span class="pre">dy</span></code> is just the standard deviation of the fit residuals
<code class="docutils literal notranslate"><span class="pre">y[i]</span> <span class="pre">-</span> <span class="pre">fcn(x[i],p)</span></code> with the best-fit parameters <code class="docutils literal notranslate"><span class="pre">p</span></code>. This
assumption is implicit in many fit routines that fit
data without errors (and without priors).</p>
</section>
<section id="tuning-priors-with-the-empirical-bayes-criterion">
<span id="empirical-bayes"></span><h2>Tuning Priors with the Empirical Bayes Criterion<a class="headerlink" href="#tuning-priors-with-the-empirical-bayes-criterion" title="Permalink to this heading">¶</a></h2>
<p>Given two choices of prior for a parameter, the one that results in a larger
Gaussian Bayes Factor after fitting (see <code class="docutils literal notranslate"><span class="pre">logGBF</span></code> in fit output or
<code class="docutils literal notranslate"><span class="pre">fit.logGBF</span></code>) is the one preferred by the data. We can use this fact to tune
a prior or set of priors in situations where we are uncertain about the
correct <em>a priori</em> value: we vary the widths and/or central values of the
priors of interest to maximize <code class="docutils literal notranslate"><span class="pre">logGBF</span></code>. In effect
we are using the data to get a feel for what is a reasonable prior. This
procedure for setting priors is again, as in the previous section,
an example of the Empirical Bayes method and can be implemented using
function <a class="reference internal" href="lsqfit.html#lsqfit.empbayes_fit" title="lsqfit.empbayes_fit"><code class="xref py py-func docutils literal notranslate"><span class="pre">lsqfit.empbayes_fit()</span></code></a>.</p>
<p>The following code illustrates how this is done:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">import</span> <span class="nn">gvar</span> <span class="k">as</span> <span class="nn">gv</span>
<span class="kn">import</span> <span class="nn">lsqfit</span>

<span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">0.1</span><span class="p">,</span> <span class="mf">0.2</span><span class="p">,</span> <span class="mf">0.3</span><span class="p">,</span> <span class="mf">0.4</span><span class="p">,</span> <span class="mf">0.5</span><span class="p">,</span> <span class="mf">0.6</span><span class="p">,</span> <span class="mf">0.7</span><span class="p">])</span>
<span class="n">y</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span>
    <span class="s1">&#39;0.133426(95)&#39;</span><span class="p">,</span> <span class="s1">&#39;0.20525(15)&#39;</span><span class="p">,</span> <span class="s1">&#39;0.27491(20)&#39;</span><span class="p">,</span> <span class="s1">&#39;0.32521(25)&#39;</span><span class="p">,</span>
    <span class="s1">&#39;0.34223(28)&#39;</span><span class="p">,</span> <span class="s1">&#39;0.32394(28)&#39;</span><span class="p">,</span> <span class="s1">&#39;0.27857(27)&#39;</span>
    <span class="p">])</span>

<span class="k">def</span> <span class="nf">fcn</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">p</span><span class="p">):</span>
    <span class="k">return</span> <span class="n">gv</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="o">-</span><span class="n">p</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">-</span> <span class="n">p</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">*</span> <span class="n">x</span> <span class="o">-</span> <span class="n">p</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">*</span> <span class="n">x</span><span class="o">**</span><span class="mi">2</span> <span class="o">-</span> <span class="n">p</span><span class="p">[</span><span class="mi">3</span><span class="p">]</span> <span class="o">*</span> <span class="n">x</span><span class="o">**</span><span class="mi">3</span><span class="p">)</span>

<span class="k">def</span> <span class="nf">fitargs</span><span class="p">(</span><span class="n">z</span><span class="p">):</span>
    <span class="n">dp</span> <span class="o">=</span> <span class="n">z</span>
    <span class="n">prior</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">([</span><span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="n">dp</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">4</span><span class="p">)])</span>
    <span class="k">return</span> <span class="nb">dict</span><span class="p">(</span><span class="n">prior</span><span class="o">=</span><span class="n">prior</span><span class="p">,</span> <span class="n">fcn</span><span class="o">=</span><span class="n">fcn</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="p">(</span><span class="n">x</span><span class="p">,</span><span class="n">y</span><span class="p">))</span>

<span class="n">fit</span><span class="p">,</span><span class="n">z</span> <span class="o">=</span> <span class="n">lsqfit</span><span class="o">.</span><span class="n">empbayes_fit</span><span class="p">(</span><span class="mf">1.0</span><span class="p">,</span> <span class="n">fitargs</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">fit</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="kc">True</span><span class="p">))</span>
</pre></div>
</div>
<p>Here the fitter varies parameters <code class="docutils literal notranslate"><span class="pre">p</span></code> until <code class="docutils literal notranslate"><span class="pre">fcn(x,p)</span></code> equals the
input data <code class="docutils literal notranslate"><span class="pre">y</span></code>. We don’t know <em>a priori</em> how large the coefficients
<code class="docutils literal notranslate"><span class="pre">p[i]</span></code> are. In <code class="docutils literal notranslate"><span class="pre">fitargs</span></code> we assume they are all of order
<code class="docutils literal notranslate"><span class="pre">dp</span> <span class="pre">=</span> <span class="pre">z</span></code>. Function <code class="docutils literal notranslate"><span class="pre">empbayes_fit</span></code> varies <code class="docutils literal notranslate"><span class="pre">z</span></code> to maximize <code class="docutils literal notranslate"><span class="pre">fit.logGBF</span></code>. The output is as follows:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.81</span> <span class="p">[</span><span class="mi">7</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.58</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">21.274</span>

<span class="n">Parameters</span><span class="p">:</span>
              <span class="mi">0</span>   <span class="mf">2.5904</span> <span class="p">(</span><span class="mi">22</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">5.3</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>   <span class="o">-</span><span class="mf">6.530</span> <span class="p">(</span><span class="mi">22</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">5.3</span><span class="p">)</span> <span class="p">]</span>  <span class="o">*</span>
              <span class="mi">2</span>    <span class="mf">7.832</span> <span class="p">(</span><span class="mi">65</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">5.3</span><span class="p">)</span> <span class="p">]</span>  <span class="o">*</span>
              <span class="mi">3</span>   <span class="o">-</span><span class="mf">1.688</span> <span class="p">(</span><span class="mi">55</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">5.3</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Fit</span><span class="p">:</span>
     <span class="n">x</span><span class="p">[</span><span class="n">k</span><span class="p">]</span>             <span class="n">y</span><span class="p">[</span><span class="n">k</span><span class="p">]</span>        <span class="n">f</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="n">k</span><span class="p">],</span><span class="n">p</span><span class="p">)</span>
<span class="o">-------------------------------------------</span>
      <span class="mf">0.1</span>    <span class="mf">0.133426</span> <span class="p">(</span><span class="mi">95</span><span class="p">)</span>    <span class="mf">0.133451</span> <span class="p">(</span><span class="mi">92</span><span class="p">)</span>  
      <span class="mf">0.2</span>     <span class="mf">0.20525</span> <span class="p">(</span><span class="mi">15</span><span class="p">)</span>     <span class="mf">0.20512</span> <span class="p">(</span><span class="mi">10</span><span class="p">)</span>  
      <span class="mf">0.3</span>     <span class="mf">0.27491</span> <span class="p">(</span><span class="mi">20</span><span class="p">)</span>     <span class="mf">0.27509</span> <span class="p">(</span><span class="mi">14</span><span class="p">)</span>  
      <span class="mf">0.4</span>     <span class="mf">0.32521</span> <span class="p">(</span><span class="mi">25</span><span class="p">)</span>     <span class="mf">0.32516</span> <span class="p">(</span><span class="mi">15</span><span class="p">)</span>  
      <span class="mf">0.5</span>     <span class="mf">0.34223</span> <span class="p">(</span><span class="mi">28</span><span class="p">)</span>     <span class="mf">0.34220</span> <span class="p">(</span><span class="mi">19</span><span class="p">)</span>  
      <span class="mf">0.6</span>     <span class="mf">0.32394</span> <span class="p">(</span><span class="mi">28</span><span class="p">)</span>     <span class="mf">0.32392</span> <span class="p">(</span><span class="mi">18</span><span class="p">)</span>  
      <span class="mf">0.7</span>     <span class="mf">0.27857</span> <span class="p">(</span><span class="mi">27</span><span class="p">)</span>     <span class="mf">0.27859</span> <span class="p">(</span><span class="mi">26</span><span class="p">)</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">1</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

</pre></div>
</div>
<p>The data suggest that the coefficients are of order 0±5.3.
The actual values of the parameters are, of course,
consistent with the Empirical Bayes estimate.</p>
<p>The Bayes factor, <code class="docutils literal notranslate"><span class="pre">exp(fit.logGBF)</span></code>, is useful for deciding about
fit functions as well as priors. If we repeat the analysis above
but with the following data</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">0.1</span><span class="p">,</span> <span class="mf">0.2</span><span class="p">,</span> <span class="mf">0.3</span><span class="p">,</span> <span class="mf">0.4</span><span class="p">,</span> <span class="mf">0.5</span><span class="p">,</span> <span class="mf">0.6</span><span class="p">,</span> <span class="mf">0.7</span><span class="p">])</span>
<span class="n">y</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span>
    <span class="s1">&#39;0.133213(95)&#39;</span><span class="p">,</span> <span class="s1">&#39;0.20245(15)&#39;</span><span class="p">,</span> <span class="s1">&#39;0.26282(19)&#39;</span><span class="p">,</span> <span class="s1">&#39;0.29099(22)&#39;</span><span class="p">,</span>
    <span class="s1">&#39;0.27589(22)&#39;</span><span class="p">,</span> <span class="s1">&#39;0.22328(19)&#39;</span><span class="p">,</span> <span class="s1">&#39;0.15436(14)&#39;</span>
    <span class="p">])</span>
</pre></div>
</div>
<p>we find that fits with 3 or 4 <code class="docutils literal notranslate"><span class="pre">p[i]</span></code>s give the following results:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="o">==========</span> <span class="n">fcn</span><span class="p">(</span><span class="n">x</span><span class="p">,</span><span class="n">p</span><span class="p">)</span> <span class="o">=</span> <span class="n">exp</span><span class="p">(</span><span class="o">-</span><span class="n">p</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">-</span> <span class="n">p</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">*</span> <span class="n">x</span> <span class="o">-</span> <span class="n">p</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">*</span> <span class="n">x</span><span class="o">**</span><span class="mi">2</span><span class="p">)</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.86</span> <span class="p">[</span><span class="mi">7</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.53</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">27.07</span>

<span class="n">Parameters</span><span class="p">:</span>
              <span class="mi">0</span>    <span class="mf">2.5911</span> <span class="p">(</span><span class="mi">12</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">5.3</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>   <span class="o">-</span><span class="mf">6.5420</span> <span class="p">(</span><span class="mi">68</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">5.3</span><span class="p">)</span> <span class="p">]</span>  <span class="o">*</span>
              <span class="mi">2</span>    <span class="mf">7.8711</span> <span class="p">(</span><span class="mi">86</span><span class="p">)</span>      <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">5.3</span><span class="p">)</span> <span class="p">]</span>  <span class="o">*</span>

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">8</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>


<span class="o">==========</span> <span class="n">fcn</span><span class="p">(</span><span class="n">x</span><span class="p">,</span><span class="n">p</span><span class="p">)</span> <span class="o">=</span> <span class="n">exp</span><span class="p">(</span><span class="o">-</span><span class="n">p</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">-</span> <span class="n">p</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">*</span> <span class="n">x</span> <span class="o">-</span> <span class="n">p</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">*</span> <span class="n">x</span><span class="o">**</span><span class="mi">2</span> <span class="o">-</span> <span class="n">p</span><span class="p">[</span><span class="mi">3</span><span class="p">]</span> <span class="o">*</span> <span class="n">x</span><span class="o">**</span><span class="mi">3</span><span class="p">)</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.82</span> <span class="p">[</span><span class="mi">7</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.57</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">22.617</span>

<span class="n">Parameters</span><span class="p">:</span>
              <span class="mi">0</span>   <span class="mf">2.5920</span> <span class="p">(</span><span class="mi">21</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">5.3</span><span class="p">)</span> <span class="p">]</span>  
              <span class="mi">1</span>   <span class="o">-</span><span class="mf">6.553</span> <span class="p">(</span><span class="mi">22</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">5.3</span><span class="p">)</span> <span class="p">]</span>  <span class="o">*</span>
              <span class="mi">2</span>    <span class="mf">7.905</span> <span class="p">(</span><span class="mi">64</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">5.3</span><span class="p">)</span> <span class="p">]</span>  <span class="o">*</span>
              <span class="mi">3</span>   <span class="o">-</span><span class="mf">0.029</span> <span class="p">(</span><span class="mi">54</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">5.3</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">11</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

</pre></div>
</div>
<p>The two fits are almost equally good, giving almost the same <code class="docutils literal notranslate"><span class="pre">chi**2</span></code>
values. The first fit, with only 3 <code class="docutils literal notranslate"><span class="pre">p[i]</span></code>s, however, has a
significantly larger <code class="docutils literal notranslate"><span class="pre">logGBF</span></code>. This indicates that this data is
<code class="docutils literal notranslate"><span class="pre">exp(27.1-22.6)</span> <span class="pre">=</span> <span class="pre">90</span></code> times more likely to come from the theory with
only 3 <code class="docutils literal notranslate"><span class="pre">p[i]</span></code>s than from the one with 4. The data much prefer
the 3-parameter theory, and they do, as it turns out,
come from such a theory. Note that the value for <code class="docutils literal notranslate"><span class="pre">p[3]</span></code> in
the second case is consistent with zero, but the errors on the other
parameters are much larger if it is included in the fit.</p>
<p>The Empirical Bayes procedure can be abused, because it is possible to make
<code class="docutils literal notranslate"><span class="pre">logGBF</span></code> arbitrarily large. For example, setting</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">prior</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">([</span>
    <span class="s1">&#39;2.5904 +- 2.6e-16&#39;</span><span class="p">,</span> <span class="s1">&#39;-6.53012 +- 6.5e-16&#39;</span><span class="p">,</span>
    <span class="s1">&#39;7.83211 +- 7.8e-16&#39;</span><span class="p">,</span> <span class="s1">&#39;-1.68813 +- 1.7e-16&#39;</span><span class="p">,</span>
    <span class="p">])</span>
</pre></div>
</div>
<p>in the problem above and then fitting gives <code class="docutils literal notranslate"><span class="pre">logGBF=52.2</span></code>,  which is much
larger than the  alternatives above. This “prior” is ridiculous, however: it
has means equal to the best-fit results with standard deviations that are  16 orders of magnitude smaller. This is the kind of prior you  get from
Empirical Bayes if you vary the means and standard deviations of all
parameters independently.</p>
<p>Bayes Theorem explains what is wrong with such priors. The Bayes Factor is
proportional to the probability <code class="docutils literal notranslate"><span class="pre">P(y|model)</span></code>  that the fit data would arise
given the model (priors plus fit function). When selecting models, we really
want to maximize <code class="docutils literal notranslate"><span class="pre">P(model|y)</span></code>, the probability of the model given the data.
These two probabilities are different, but are related by Bayes Theorem:
<code class="docutils literal notranslate"><span class="pre">P(model|y)</span></code> is proportional to <code class="docutils literal notranslate"><span class="pre">P(y|model)</span></code> times <code class="docutils literal notranslate"><span class="pre">P(model)</span></code>, where
<code class="docutils literal notranslate"><span class="pre">P(model)</span></code> is the <em>a priori</em> probability of the model being correct. When we
choose a model by maximizing <code class="docutils literal notranslate"><span class="pre">logGBF</span></code> (that is, by
maximizing <code class="docutils literal notranslate"><span class="pre">P(y|model)</span></code>),
we are implicitly assuming that the
various models we are considering are all equally likely candidates — that
is, we are assuming that <code class="docutils literal notranslate"><span class="pre">P(model)</span></code> is approximately constant across the
model space we are exploring. The <em>a priori</em> probability for the ridiculous
prior just
above is vanishingly small,
and so comparing its <code class="docutils literal notranslate"><span class="pre">logGBF</span></code> to
the others is nonsensical.</p>
<p>Note that <code class="xref py py-func docutils literal notranslate"><span class="pre">empbayes_fit()</span></code> allows <code class="docutils literal notranslate"><span class="pre">fitargs(z)</span></code> to return
a dictionary of arguments for the fitter together with a <code class="docutils literal notranslate"><span class="pre">plausibility</span></code>
for <code class="docutils literal notranslate"><span class="pre">z</span></code>, which corresponds to <code class="docutils literal notranslate"><span class="pre">log(P(model))</span></code> in the discussion
above. This allows you steer the search away from completely
implausible solutions.</p>
<p>Empirical Bayes tends to be most useful when varying the width of the prior for
a single parameter,  or varying the widths of a group of parameters together.
It is also useful for validating (rather than setting) the choice of a
prior or set of priors for a fit, by comparing the optimal choice (according
to the data) with choice actually used.</p>
</section>
<section id="positive-parameters-non-gaussian-priors">
<span id="positive-parameters"></span><h2>Positive Parameters; Non-Gaussian Priors<a class="headerlink" href="#positive-parameters-non-gaussian-priors" title="Permalink to this heading">¶</a></h2>
<p>The priors for <a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a> are all Gaussian. There are situations,
however, where other distributions would be desirable. One such case is where
a parameter is known to be positive, but is close to zero in value (“close”
being defined relative to the <em>a priori</em> uncertainty). For such cases we would
like to use  non-Gaussian priors that force positivity — for example, priors
that  impose log-normal or exponential distributions on the parameter.
Ideally the decision to use such a distribution is made on a parameter-
by-parameter basis, when creating the priors, and has no impact on the
definition of the fit function itself.</p>
<p><a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a> supports log-normal distributions among others. This
functionality is implemented through the <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.BufferDict</span></code> dictionary
class, which is the standard dictionary used by <a class="reference internal" href="lsqfit.html#module-lsqfit" title="lsqfit: Nonlinear least squares fitting."><code class="xref py py-mod docutils literal notranslate"><span class="pre">lsqfit</span></code></a> internally and
for results. See the <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.BufferDict</span></code> documentation for more
information.</p>
<p>The prior for a parameter <code class="docutils literal notranslate"><span class="pre">'a'</span></code>, for example, is switched from
a Gaussian distribution to a log-normal distribution by replacing
<code class="docutils literal notranslate"><span class="pre">prior['a']</span></code> in the fit prior with a prior for its logarithm,
<code class="docutils literal notranslate"><span class="pre">prior['log(a)']</span></code>. This causes <a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a> to use the logarithm as the
fit parameter (with its Gaussian prior). Fit parameters are stored in a
<code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.BufferDict</span></code> <code class="docutils literal notranslate"><span class="pre">p</span></code>, which returns values for <code class="docutils literal notranslate"><span class="pre">p['log(a)']</span></code>, as expected, but
also for <code class="docutils literal notranslate"><span class="pre">p['a']</span></code>, where the latter is automatically set equal to the
exponential of the former. Consequently only the prior need be changed to
switch distributions. In particular the fit function can be expressed directly
in terms of fit parameter <code class="docutils literal notranslate"><span class="pre">p['a']</span></code>, so that it is independent of the
distribution chosen for the <code class="docutils literal notranslate"><span class="pre">'a'</span></code> prior.</p>
<p>To illustrate consider a simple problem where an experimental quantity <code class="docutils literal notranslate"><span class="pre">y</span></code> is
known to be positive, but experimental errors mean that measured values can
often be negative:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">gvar</span> <span class="k">as</span> <span class="nn">gv</span>
<span class="kn">import</span> <span class="nn">lsqfit</span>

<span class="n">y</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">([</span>
    <span class="s1">&#39;-0.17(20)&#39;</span><span class="p">,</span> <span class="s1">&#39;-0.03(20)&#39;</span><span class="p">,</span> <span class="s1">&#39;-0.39(20)&#39;</span><span class="p">,</span> <span class="s1">&#39;0.10(20)&#39;</span><span class="p">,</span> <span class="s1">&#39;-0.03(20)&#39;</span><span class="p">,</span>
    <span class="s1">&#39;0.06(20)&#39;</span><span class="p">,</span> <span class="s1">&#39;-0.23(20)&#39;</span><span class="p">,</span> <span class="s1">&#39;-0.23(20)&#39;</span><span class="p">,</span> <span class="s1">&#39;-0.15(20)&#39;</span><span class="p">,</span> <span class="s1">&#39;-0.01(20)&#39;</span><span class="p">,</span>
    <span class="s1">&#39;-0.12(20)&#39;</span><span class="p">,</span> <span class="s1">&#39;0.05(20)&#39;</span><span class="p">,</span> <span class="s1">&#39;-0.09(20)&#39;</span><span class="p">,</span> <span class="s1">&#39;-0.36(20)&#39;</span><span class="p">,</span> <span class="s1">&#39;0.09(20)&#39;</span><span class="p">,</span>
    <span class="s1">&#39;-0.07(20)&#39;</span><span class="p">,</span> <span class="s1">&#39;-0.31(20)&#39;</span><span class="p">,</span> <span class="s1">&#39;0.12(20)&#39;</span><span class="p">,</span> <span class="s1">&#39;0.11(20)&#39;</span><span class="p">,</span> <span class="s1">&#39;0.13(20)&#39;</span>
    <span class="p">])</span>
</pre></div>
</div>
<p>We want to know the average value <code class="docutils literal notranslate"><span class="pre">a</span></code> of the <code class="docutils literal notranslate"><span class="pre">y</span></code>s and so could
use the following fitting code:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">prior</span> <span class="o">=</span> <span class="p">{</span><span class="s1">&#39;a&#39;</span><span class="p">:</span><span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="s1">&#39;0.02(2)&#39;</span><span class="p">)}</span>               <span class="c1"># a = average of y&#39;s</span>

<span class="k">def</span> <span class="nf">fcn</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="n">N</span><span class="o">=</span><span class="nb">len</span><span class="p">(</span><span class="n">y</span><span class="p">)):</span>
   <span class="k">return</span> <span class="n">N</span> <span class="o">*</span> <span class="p">[</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]]</span>

<span class="n">fit</span> <span class="o">=</span> <span class="n">lsqfit</span><span class="o">.</span><span class="n">nonlinear_fit</span><span class="p">(</span><span class="n">prior</span><span class="o">=</span><span class="n">prior</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">y</span><span class="p">,</span> <span class="n">fcn</span><span class="o">=</span><span class="n">fcn</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">fit</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">&#39;a =&#39;</span><span class="p">,</span> <span class="n">fit</span><span class="o">.</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">])</span>
</pre></div>
</div>
<p>where we are assuming <em>a priori</em> information that suggests
the average is around 0.02. The output from this code is:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.84</span> <span class="p">[</span><span class="mi">20</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.67</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">5.3431</span>

<span class="n">Parameters</span><span class="p">:</span>
              <span class="n">a</span>   <span class="mf">0.004</span> <span class="p">(</span><span class="mi">18</span><span class="p">)</span>     <span class="p">[</span> <span class="mf">0.020</span> <span class="p">(</span><span class="mi">20</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">4</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="n">a</span> <span class="o">=</span> <span class="mf">0.004</span><span class="p">(</span><span class="mi">18</span><span class="p">)</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.85</span> <span class="p">[</span><span class="mi">20</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.65</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">5.252</span>

<span class="n">Parameters</span><span class="p">:</span>
         <span class="n">log</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>   <span class="o">-</span><span class="mf">4.44</span> <span class="p">(</span><span class="mi">97</span><span class="p">)</span>     <span class="p">[</span> <span class="o">-</span><span class="mf">3.9</span> <span class="p">(</span><span class="mf">1.0</span><span class="p">)</span> <span class="p">]</span>  
<span class="o">-----------------------------------------------</span>
              <span class="n">a</span>   <span class="mf">0.012</span> <span class="p">(</span><span class="mi">11</span><span class="p">)</span>     <span class="p">[</span> <span class="mf">0.020</span> <span class="p">(</span><span class="mi">20</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">25</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="n">a</span> <span class="o">=</span> <span class="mf">0.012</span><span class="p">(</span><span class="mi">11</span><span class="p">)</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.85</span> <span class="p">[</span><span class="mi">20</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.66</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">5.2858</span>

<span class="n">Parameters</span><span class="p">:</span>
        <span class="n">sqrt</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>   <span class="mf">0.099</span> <span class="p">(</span><span class="mi">67</span><span class="p">)</span>     <span class="p">[</span> <span class="mf">0.141</span> <span class="p">(</span><span class="mi">71</span><span class="p">)</span> <span class="p">]</span>  
<span class="o">-----------------------------------------------</span>
              <span class="n">a</span>   <span class="mf">0.010</span> <span class="p">(</span><span class="mi">13</span><span class="p">)</span>     <span class="p">[</span> <span class="mf">0.020</span> <span class="p">(</span><span class="mi">20</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">19</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="n">a</span> <span class="o">=</span> <span class="mf">0.010</span><span class="p">(</span><span class="mi">13</span><span class="p">)</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.85</span> <span class="p">[</span><span class="mi">20</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.65</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">5.2385</span>

<span class="n">Parameters</span><span class="p">:</span>
          <span class="n">ga</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>   <span class="o">-</span><span class="mf">0.59</span> <span class="p">(</span><span class="mi">96</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">1.0</span><span class="p">)</span> <span class="p">]</span>  
<span class="o">-----------------------------------------------</span>
              <span class="n">a</span>   <span class="mf">0.011</span> <span class="p">(</span><span class="mi">13</span><span class="p">)</span>     <span class="p">[</span> <span class="mf">0.020</span> <span class="p">(</span><span class="mi">16</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">15</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="n">a</span> <span class="o">=</span> <span class="mf">0.011</span><span class="p">(</span><span class="mi">13</span><span class="p">)</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.85</span> <span class="p">[</span><span class="mi">20</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.65</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">5.2385</span>

<span class="n">Parameters</span><span class="p">:</span>
          <span class="n">ga</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>   <span class="o">-</span><span class="mf">0.59</span> <span class="p">(</span><span class="mi">96</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">1.0</span><span class="p">)</span> <span class="p">]</span>  
<span class="o">-----------------------------------------------</span>
              <span class="n">a</span>   <span class="mf">0.011</span> <span class="p">(</span><span class="mi">13</span><span class="p">)</span>     <span class="p">[</span> <span class="mf">0.020</span> <span class="p">(</span><span class="mi">16</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">15</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="n">a</span> <span class="o">=</span> <span class="mf">0.011</span><span class="p">(</span><span class="mi">13</span><span class="p">)</span>

<span class="n">Histogram</span> <span class="n">Analysis</span><span class="p">:</span>
   <span class="n">mean</span> <span class="o">=</span> <span class="mf">0.014054</span><span class="p">(</span><span class="mi">22</span><span class="p">)</span>   <span class="n">sdev</span> <span class="o">=</span> <span class="mf">0.010659</span><span class="p">(</span><span class="mi">21</span><span class="p">)</span>   <span class="n">skew</span> <span class="o">=</span> <span class="mf">0.6181</span><span class="p">(</span><span class="mi">26</span><span class="p">)</span>   <span class="n">ex_kurt</span> <span class="o">=</span> <span class="o">-</span><span class="mf">0.5593</span><span class="p">(</span><span class="mi">50</span><span class="p">)</span>
   <span class="n">median</span> <span class="o">=</span> <span class="mf">0.011867</span><span class="p">(</span><span class="mi">19</span><span class="p">)</span>   <span class="n">plus</span> <span class="o">=</span> <span class="mf">0.014385</span><span class="p">(</span><span class="mi">31</span><span class="p">)</span>   <span class="n">minus</span> <span class="o">=</span> <span class="mf">0.008761</span><span class="p">(</span><span class="mi">18</span><span class="p">)</span>
</pre></div>
</div>
<p>This is not such a useful result since much of the one-sigma range for <code class="docutils literal notranslate"><span class="pre">a</span></code>
is negative, and yet we know that <code class="docutils literal notranslate"><span class="pre">a</span></code> must be postive.</p>
<p>A better analysis uses a log-normal distribution for <code class="docutils literal notranslate"><span class="pre">a</span></code>:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">prior</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">prior</span><span class="p">[</span><span class="s1">&#39;log(a)&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="s1">&#39;0.02(2)&#39;</span><span class="p">))</span>   <span class="c1"># log(a) not a</span>

<span class="k">def</span> <span class="nf">fcn</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="n">N</span><span class="o">=</span><span class="nb">len</span><span class="p">(</span><span class="n">y</span><span class="p">)):</span>
   <span class="k">return</span> <span class="n">N</span> <span class="o">*</span> <span class="p">[</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]]</span>

<span class="n">fit</span> <span class="o">=</span> <span class="n">lsqfit</span><span class="o">.</span><span class="n">nonlinear_fit</span><span class="p">(</span><span class="n">prior</span><span class="o">=</span><span class="n">prior</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">y</span><span class="p">,</span> <span class="n">fcn</span><span class="o">=</span><span class="n">fcn</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">fit</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">&#39;a =&#39;</span><span class="p">,</span> <span class="n">fit</span><span class="o">.</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">])</span>                       <span class="c1"># exp(log(a))</span>
</pre></div>
</div>
<p>The fit parameter is now <code class="docutils literal notranslate"><span class="pre">log(a)</span></code> rather than <code class="docutils literal notranslate"><span class="pre">a</span></code> itself, but the code
is unchanged except for the definition of the prior. In particular the
fit function is identical to what we used in the first case since
parameter dictionary <code class="docutils literal notranslate"><span class="pre">p</span></code> returns values for both <code class="docutils literal notranslate"><span class="pre">'a'</span></code> and <code class="docutils literal notranslate"><span class="pre">'log(a)'</span></code>.</p>
<p>The result from this fit is</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.85</span> <span class="p">[</span><span class="mi">20</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.65</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">5.252</span>

<span class="n">Parameters</span><span class="p">:</span>
         <span class="n">log</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>   <span class="o">-</span><span class="mf">4.44</span> <span class="p">(</span><span class="mi">97</span><span class="p">)</span>     <span class="p">[</span> <span class="o">-</span><span class="mf">3.9</span> <span class="p">(</span><span class="mf">1.0</span><span class="p">)</span> <span class="p">]</span>  
<span class="o">-----------------------------------------------</span>
              <span class="n">a</span>   <span class="mf">0.012</span> <span class="p">(</span><span class="mi">11</span><span class="p">)</span>     <span class="p">[</span> <span class="mf">0.020</span> <span class="p">(</span><span class="mi">20</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">25</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="n">a</span> <span class="o">=</span> <span class="mf">0.012</span><span class="p">(</span><span class="mi">11</span><span class="p">)</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.85</span> <span class="p">[</span><span class="mi">20</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.66</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">5.2858</span>

<span class="n">Parameters</span><span class="p">:</span>
        <span class="n">sqrt</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>   <span class="mf">0.099</span> <span class="p">(</span><span class="mi">67</span><span class="p">)</span>     <span class="p">[</span> <span class="mf">0.141</span> <span class="p">(</span><span class="mi">71</span><span class="p">)</span> <span class="p">]</span>  
<span class="o">-----------------------------------------------</span>
              <span class="n">a</span>   <span class="mf">0.010</span> <span class="p">(</span><span class="mi">13</span><span class="p">)</span>     <span class="p">[</span> <span class="mf">0.020</span> <span class="p">(</span><span class="mi">20</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">19</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="n">a</span> <span class="o">=</span> <span class="mf">0.010</span><span class="p">(</span><span class="mi">13</span><span class="p">)</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.85</span> <span class="p">[</span><span class="mi">20</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.65</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">5.2385</span>

<span class="n">Parameters</span><span class="p">:</span>
          <span class="n">ga</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>   <span class="o">-</span><span class="mf">0.59</span> <span class="p">(</span><span class="mi">96</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">1.0</span><span class="p">)</span> <span class="p">]</span>  
<span class="o">-----------------------------------------------</span>
              <span class="n">a</span>   <span class="mf">0.011</span> <span class="p">(</span><span class="mi">13</span><span class="p">)</span>     <span class="p">[</span> <span class="mf">0.020</span> <span class="p">(</span><span class="mi">16</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">15</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="n">a</span> <span class="o">=</span> <span class="mf">0.011</span><span class="p">(</span><span class="mi">13</span><span class="p">)</span>
<span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.85</span> <span class="p">[</span><span class="mi">20</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.65</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">5.2385</span>

<span class="n">Parameters</span><span class="p">:</span>
          <span class="n">ga</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>   <span class="o">-</span><span class="mf">0.59</span> <span class="p">(</span><span class="mi">96</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">1.0</span><span class="p">)</span> <span class="p">]</span>  
<span class="o">-----------------------------------------------</span>
              <span class="n">a</span>   <span class="mf">0.011</span> <span class="p">(</span><span class="mi">13</span><span class="p">)</span>     <span class="p">[</span> <span class="mf">0.020</span> <span class="p">(</span><span class="mi">16</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">15</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="n">a</span> <span class="o">=</span> <span class="mf">0.011</span><span class="p">(</span><span class="mi">13</span><span class="p">)</span>

<span class="n">Histogram</span> <span class="n">Analysis</span><span class="p">:</span>
   <span class="n">mean</span> <span class="o">=</span> <span class="mf">0.014054</span><span class="p">(</span><span class="mi">22</span><span class="p">)</span>   <span class="n">sdev</span> <span class="o">=</span> <span class="mf">0.010659</span><span class="p">(</span><span class="mi">21</span><span class="p">)</span>   <span class="n">skew</span> <span class="o">=</span> <span class="mf">0.6181</span><span class="p">(</span><span class="mi">26</span><span class="p">)</span>   <span class="n">ex_kurt</span> <span class="o">=</span> <span class="o">-</span><span class="mf">0.5593</span><span class="p">(</span><span class="mi">50</span><span class="p">)</span>
   <span class="n">median</span> <span class="o">=</span> <span class="mf">0.011867</span><span class="p">(</span><span class="mi">19</span><span class="p">)</span>   <span class="n">plus</span> <span class="o">=</span> <span class="mf">0.014385</span><span class="p">(</span><span class="mi">31</span><span class="p">)</span>   <span class="n">minus</span> <span class="o">=</span> <span class="mf">0.008761</span><span class="p">(</span><span class="mi">18</span><span class="p">)</span>
</pre></div>
</div>
<p>which is more compelling. Parameters listed  above the dashed line in the
parameter table are the actual  parameters used in the fit; those listed below
the dashed line are derived from those above the line. The “correct” value for
<code class="docutils literal notranslate"><span class="pre">a</span></code> here is 0.015 (given the method used to generate the <code class="docutils literal notranslate"><span class="pre">y</span></code>s).</p>
<p>Other distributions are available. For example, the code</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">prior</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">prior</span><span class="p">[</span><span class="s1">&#39;f(a)&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">BufferDict</span><span class="o">.</span><span class="n">uniform</span><span class="p">(</span><span class="s1">&#39;f&#39;</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mf">0.04</span><span class="p">)</span>

<span class="k">def</span> <span class="nf">fcn</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="n">N</span><span class="o">=</span><span class="nb">len</span><span class="p">(</span><span class="n">y</span><span class="p">)):</span>
   <span class="k">return</span> <span class="n">N</span> <span class="o">*</span> <span class="p">[</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]]</span>

<span class="n">fit</span> <span class="o">=</span> <span class="n">lsqfit</span><span class="o">.</span><span class="n">nonlinear_fit</span><span class="p">(</span><span class="n">prior</span><span class="o">=</span><span class="n">prior</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">y</span><span class="p">,</span> <span class="n">fcn</span><span class="o">=</span><span class="n">fcn</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">fit</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">&#39;a =&#39;</span><span class="p">,</span> <span class="n">fit</span><span class="o">.</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">])</span>                       <span class="c1"># exp(log(a))</span>
</pre></div>
</div>
<p>creates a function <code class="docutils literal notranslate"><span class="pre">f(a)</span></code> such that the prior for
parameter <code class="docutils literal notranslate"><span class="pre">p['a']</span></code> is uniformly distributed between 0
and 0.04, and zero otherwise. The function name <code class="docutils literal notranslate"><span class="pre">f</span></code>
is arbitrary; <code class="docutils literal notranslate"><span class="pre">f(a)</span></code> has a Gaussian prior 0 ± 1.
This code gives the following output:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">Least</span> <span class="n">Square</span> <span class="n">Fit</span><span class="p">:</span>
  <span class="n">chi2</span><span class="o">/</span><span class="n">dof</span> <span class="p">[</span><span class="n">dof</span><span class="p">]</span> <span class="o">=</span> <span class="mf">0.85</span> <span class="p">[</span><span class="mi">20</span><span class="p">]</span>    <span class="n">Q</span> <span class="o">=</span> <span class="mf">0.65</span>    <span class="n">logGBF</span> <span class="o">=</span> <span class="mf">5.2385</span>

<span class="n">Parameters</span><span class="p">:</span>
           <span class="n">f</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>   <span class="o">-</span><span class="mf">0.59</span> <span class="p">(</span><span class="mi">96</span><span class="p">)</span>     <span class="p">[</span>  <span class="mf">0.0</span> <span class="p">(</span><span class="mf">1.0</span><span class="p">)</span> <span class="p">]</span>  
<span class="o">-----------------------------------------------</span>
              <span class="n">a</span>   <span class="mf">0.011</span> <span class="p">(</span><span class="mi">13</span><span class="p">)</span>     <span class="p">[</span> <span class="mf">0.020</span> <span class="p">(</span><span class="mi">16</span><span class="p">)</span> <span class="p">]</span>  

<span class="n">Settings</span><span class="p">:</span>
  <span class="n">svdcut</span><span class="o">/</span><span class="n">n</span> <span class="o">=</span> <span class="mf">1e-12</span><span class="o">/</span><span class="mi">0</span>    <span class="n">tol</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1e-08</span><span class="o">*</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">,</span><span class="mf">1e-10</span><span class="p">)</span>    <span class="p">(</span><span class="n">itns</span><span class="o">/</span><span class="n">time</span> <span class="o">=</span> <span class="mi">15</span><span class="o">/</span><span class="mf">0.0</span><span class="p">)</span>

<span class="n">a</span> <span class="o">=</span> <span class="mf">0.011</span><span class="p">(</span><span class="mi">13</span><span class="p">)</span>
</pre></div>
</div>
<p>This fit implies that <code class="docutils literal notranslate"><span class="pre">a=0.011(13)</span></code>
which is almost identical to the result obtained from the log-normal
distribution.</p>
<p>New distributions can be defined
using <code class="xref py py-meth docutils literal notranslate"><span class="pre">gvar.BufferDict.add_distribution()</span></code>.
For example,</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">lsqfit</span>
<span class="kn">import</span> <span class="nn">gvar</span> <span class="k">as</span> <span class="nn">gv</span>

<span class="k">def</span> <span class="nf">invf</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
    <span class="k">return</span> <span class="mf">0.02</span> <span class="o">+</span> <span class="mf">0.02</span> <span class="o">*</span> <span class="n">gv</span><span class="o">.</span><span class="n">tanh</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>

<span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>                                   <span class="c1"># not used</span>
    <span class="k">return</span> <span class="n">gv</span><span class="o">.</span><span class="n">arctanh</span><span class="p">((</span><span class="n">x</span> <span class="o">-</span> <span class="mf">0.02</span><span class="p">)</span> <span class="o">/</span> <span class="mf">0.02</span><span class="p">)</span>

<span class="n">gv</span><span class="o">.</span><span class="n">add_parameter_distribution</span><span class="p">(</span><span class="s1">&#39;f&#39;</span><span class="p">,</span> <span class="n">invf</span><span class="p">)</span>

<span class="n">prior</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">prior</span><span class="p">[</span><span class="s1">&#39;f(a)&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="s1">&#39;0.00(75)&#39;</span><span class="p">)</span>

<span class="k">def</span> <span class="nf">fcn</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="n">N</span><span class="o">=</span><span class="nb">len</span><span class="p">(</span><span class="n">y</span><span class="p">)):</span>
    <span class="k">return</span> <span class="n">N</span> <span class="o">*</span> <span class="p">[</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]]</span>

<span class="n">fit</span> <span class="o">=</span> <span class="n">lsqfit</span><span class="o">.</span><span class="n">nonlinear_fit</span><span class="p">(</span><span class="n">prior</span><span class="o">=</span><span class="n">prior</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">y</span><span class="p">,</span> <span class="n">fcn</span><span class="o">=</span><span class="n">fcn</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">fit</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">&#39;a =&#39;</span><span class="p">,</span> <span class="n">fit</span><span class="o">.</span><span class="n">p</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">])</span>
</pre></div>
</div>
<p>does a fit with Gaussian parameter <code class="docutils literal notranslate"><span class="pre">f(a)</span></code>, which forces <code class="docutils literal notranslate"><span class="pre">a</span></code>
to lie between 0 and 0.04. This fit gives <code class="docutils literal notranslate"><span class="pre">a=0.012(12)</span></code>, which
again agrees well with log-normal fit. The prior 0±0.75 for <code class="docutils literal notranslate"><span class="pre">f(a)</span></code>
is chosen to make the prior probability
distribution for parameter <code class="docutils literal notranslate"><span class="pre">a</span></code> almost flat
across most (80%) of the interval 0.02±0.02.</p>
</section>
<section id="faster-fitters">
<span id="id2"></span><h2>Faster Fitters<a class="headerlink" href="#faster-fitters" title="Permalink to this heading">¶</a></h2>
<p><a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a> uses fitters from the Gnu Scientific Library (GSL) and/or
from the <code class="xref py py-mod docutils literal notranslate"><span class="pre">scipy</span></code> Python module to do the actual fitting, depending
upon which of these is installed. It is worth trying a different fitter
or fit algorithm
if a fit is causing trouble, since different fitters are optimized for
different problems. The fitter is selected using the <code class="docutils literal notranslate"><span class="pre">fitter</span></code> argument
in <a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a>. There are currently three fitters available:</p>
<blockquote>
<div><dl class="simple">
<dt><code class="docutils literal notranslate"><span class="pre">fitter='gsl_multifit'</span></code></dt><dd><p>The standard GSL least-squares fitter which
is wrapped in Python class <a class="reference internal" href="gsl.html#lsqfit.gsl_multifit" title="lsqfit.gsl_multifit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.gsl_multifit</span></code></a>. This is the
default fitter provided GSL is installed. It offers a wide range
of options, including several different algorithms that are selected
by setting <a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a> parameter <code class="docutils literal notranslate"><span class="pre">alg</span></code> equal to <code class="docutils literal notranslate"><span class="pre">'lm'</span></code>, <code class="docutils literal notranslate"><span class="pre">'lmaccel'</span></code>,
<code class="docutils literal notranslate"><span class="pre">'subspace2D'</span></code>, <code class="docutils literal notranslate"><span class="pre">'dogleg'</span></code>, and so on. See the documentation.</p>
</dd>
<dt><code class="docutils literal notranslate"><span class="pre">fitter='gsl_v1_multifit'</span></code></dt><dd><p>The GSL fitter from version 1 of the
GSL library. This is wrapped in Python class
<a class="reference internal" href="gsl.html#lsqfit.gsl_v1_multifit" title="lsqfit.gsl_v1_multifit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.gsl_v1_multifit</span></code></a>. It was the fitter used in <a class="reference internal" href="lsqfit.html#module-lsqfit" title="lsqfit: Nonlinear least squares fitting."><code class="xref py py-mod docutils literal notranslate"><span class="pre">lsqfit</span></code></a>
versions earlier than version 9.0. It supports a few
different algorithms (parameter <code class="docutils literal notranslate"><span class="pre">alg</span></code>) including
<code class="docutils literal notranslate"><span class="pre">'lmsder'</span></code> and <code class="docutils literal notranslate"><span class="pre">'lmder'</span></code>.</p>
</dd>
<dt><code class="docutils literal notranslate"><span class="pre">fitter='scipy_least_squares'</span></code></dt><dd><p>The standard <code class="xref py py-mod docutils literal notranslate"><span class="pre">scipy</span></code> least-squares
fitter, here provided with an <a class="reference internal" href="lsqfit.html#module-lsqfit" title="lsqfit: Nonlinear least squares fitting."><code class="xref py py-mod docutils literal notranslate"><span class="pre">lsqfit</span></code></a> interface by
class <a class="reference internal" href="scipy.html#lsqfit.scipy_least_squares" title="lsqfit.scipy_least_squares"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.scipy_least_squares</span></code></a>. This is the default fitter
when GSL is not available. It also provides a variety of algorithms
(set parameter <code class="docutils literal notranslate"><span class="pre">method</span></code>), and other options, such as loss functions
for handling outliers. See the <code class="xref py py-mod docutils literal notranslate"><span class="pre">scipy</span></code> documentation.</p>
</dd>
</dl>
</div></blockquote>
<p>The default configurations for these fitters are chosen to emphasize
robustness rather than speed, and therefore some of the non-default options
can be much faster. Adding</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">fitter</span><span class="o">=</span><span class="s1">&#39;gsl_multifit&#39;</span><span class="p">,</span> <span class="n">alg</span><span class="o">=</span><span class="s1">&#39;subspace2D&#39;</span><span class="p">,</span> <span class="n">scaler</span><span class="o">=</span><span class="s1">&#39;more&#39;</span><span class="p">,</span> <span class="n">solver</span><span class="o">=</span><span class="s1">&#39;cholesky&#39;</span>
</pre></div>
</div>
<p>to <a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a>’s argument list, for example, can double or triple
the fitter’s speed for  large problems.
The more robust choices are important for challenging fits, but
straightforward fits can be greatly accelerated by using different options.
The <code class="docutils literal notranslate"><span class="pre">scipy_least_squares</span></code> fitter can also be much faster than the default.
It is worth experimenting when fits become costly.</p>
<p>Method <a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit.set" title="lsqfit.nonlinear_fit.set"><code class="xref py py-meth docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit.set()</span></code></a>
modifies the defaults used by <a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a>.
For example, we can make the fast
option mentioned above the default choice for any subsequent fit
by calling:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">lsqfit</span><span class="o">.</span><span class="n">nonlinear_fit</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
    <span class="n">fitter</span><span class="o">=</span><span class="s1">&#39;gsl_multifit&#39;</span><span class="p">,</span>
    <span class="n">alg</span><span class="o">=</span><span class="s1">&#39;subspace2D&#39;</span><span class="p">,</span>
    <span class="n">scaler</span><span class="o">=</span><span class="s1">&#39;more&#39;</span><span class="p">,</span>
    <span class="n">solver</span><span class="o">=</span><span class="s1">&#39;cholesky&#39;</span><span class="p">,</span>
    <span class="p">)</span>
</pre></div>
</div>
<p>Default values for parameters <code class="docutils literal notranslate"><span class="pre">svdcut</span></code>,
<code class="docutils literal notranslate"><span class="pre">debug</span></code>, <code class="docutils literal notranslate"><span class="pre">maxit</span></code>, <code class="docutils literal notranslate"><span class="pre">fitter</span></code>, and <code class="docutils literal notranslate"><span class="pre">tol</span></code> can be reset,
as can any parameters that are
sent to the underlying fitter
(e.g., <code class="docutils literal notranslate"><span class="pre">alg</span></code>, <code class="docutils literal notranslate"><span class="pre">scaler</span></code>, and <code class="docutils literal notranslate"><span class="pre">solver</span></code> here).
Calling the function with no arguments returns a
dictionary containing the current defaults. <code class="docutils literal notranslate"><span class="pre">nonlinear_fit.set(clear=True)</span></code>
restores the original defaults.</p>
<p><a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a> is easier to use than the underlying fitters because it can
handle correlated data, and it automatically generates
Jacobian functions for the fitter,
using automatic differentiation. It also is integrated with the <code class="xref py py-mod docutils literal notranslate"><span class="pre">gvar</span></code>
module, which provides powerful tools for error propagation, generating error
budgets, and creating potentially complicated priors for Bayesian fitting. The
underlying fitters are available from <a class="reference internal" href="lsqfit.html#module-lsqfit" title="lsqfit: Nonlinear least squares fitting."><code class="xref py py-mod docutils literal notranslate"><span class="pre">lsqfit</span></code></a> for use in other
more specialized applications.</p>
</section>
<section id="debugging-and-troubleshooting">
<h2>Debugging and Troubleshooting<a class="headerlink" href="#debugging-and-troubleshooting" title="Permalink to this heading">¶</a></h2>
<p>When a fit refuses to work, the first thing to check is that the data, prior, and fit
function are properly constructed. Setting parameter <code class="docutils literal notranslate"><span class="pre">debug=True</span></code> in
<a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a> causes the code to look for common mistakes and report on
them with more intelligible error messages. The code also then checks for
significant roundoff errors in the matrix inversion of the covariance matrix.
It is a good idea to set this parameter in the early stages of any project.</p>
<p>Sometimes the optimization algorithm has trouble finding the true
minimum of the <code class="docutils literal notranslate"><span class="pre">chi**2</span></code> function, leading to unexpectedly poor fits.
Trying a different algorithm
(see <a class="reference internal" href="#faster-fitters"><span class="std std-ref">Faster Fitters</span></a>) will sometimes help. Another option is
to replace <code class="docutils literal notranslate"><span class="pre">svdcut</span></code> with <code class="docutils literal notranslate"><span class="pre">eps</span></code> in <a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a>, which changes
the underlying fit functions from which <code class="docutils literal notranslate"><span class="pre">chi**2</span></code> is constructed.
A third option is to try different starting points <code class="docutils literal notranslate"><span class="pre">p0</span></code>. The
following code, for example, tries 5 starting points drawn at
random from the prior:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">gvar</span> <span class="k">as</span> <span class="nn">gv</span>
<span class="kn">import</span> <span class="nn">lsqfit</span>
<span class="o">...</span>
<span class="n">gv</span><span class="o">.</span><span class="n">ranseed</span><span class="p">(</span><span class="mi">12345</span><span class="p">)</span>     <span class="c1"># gives same results if run script again</span>
<span class="k">for</span> <span class="n">p0</span> <span class="ow">in</span> <span class="n">gv</span><span class="o">.</span><span class="n">raniter</span><span class="p">(</span><span class="n">prior</span><span class="p">,</span> <span class="n">n</span><span class="o">=</span><span class="mi">5</span><span class="p">):</span>
    <span class="n">fit</span> <span class="o">=</span> <span class="n">lsqfit</span><span class="o">.</span><span class="n">nonlinear_fit</span><span class="p">(</span><span class="n">data</span><span class="o">=</span><span class="n">data</span><span class="p">,</span> <span class="n">prior</span><span class="o">=</span><span class="n">prior</span><span class="p">,</span> <span class="n">fcn</span><span class="o">=</span><span class="n">fcn</span><span class="p">,</span> <span class="n">p0</span><span class="o">=</span><span class="n">p0</span><span class="p">)</span>
    <span class="nb">print</span><span class="p">(</span><span class="n">fit</span><span class="p">)</span>
</pre></div>
</div>
<p>Alternatively, simply set parameter <code class="docutils literal notranslate"><span class="pre">p0=True</span></code> to generate a random starting
point for each fit.</p>
<p>A common mistake is a mismatch between the format of the data and the
format of what comes back from the fit function. Another mistake is when  a
fit function <code class="docutils literal notranslate"><span class="pre">fcn(p)</span></code> returns results containing <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>s  when the
parameters <code class="docutils literal notranslate"><span class="pre">p</span></code> are all just numbers (or arrays of numbers). The only way a
<code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code> should get into a fit  function is through the parameters; if a fit
function requires an extra <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>, that <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code> should be turned into a
parameter by adding it to the prior.</p>
<p>Error messages that come from inside the GSL routines used by
<a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a> are usually due to errors
in one of the inputs to the fit  (that is, the fit data, the prior, or the fit
function). Again setting <code class="docutils literal notranslate"><span class="pre">debug=True</span></code> may catch the errors before they
land in GSL.</p>
<p>Occasionally <a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a> appears to go crazy, with gigantic
<code class="docutils literal notranslate"><span class="pre">chi**2</span></code>s (<em>e.g.</em>, <code class="docutils literal notranslate"><span class="pre">1e78</span></code>). This could be because there is a genuine
zero-eigenvalue mode in the covariance matrix of the data or prior. Such a
zero mode makes it impossible to invert the covariance matrix when evaluating
<code class="docutils literal notranslate"><span class="pre">chi**2</span></code>. One fix is to regulate the covariance matrix used in the fit by setting,
for example, <code class="docutils literal notranslate"><span class="pre">svdcut=1e-8</span></code> (or <code class="docutils literal notranslate"><span class="pre">eps=1e-8</span></code>) in the call
to <a class="reference internal" href="lsqfit.html#lsqfit.nonlinear_fit" title="lsqfit.nonlinear_fit"><code class="xref py py-class docutils literal notranslate"><span class="pre">lsqfit.nonlinear_fit</span></code></a>. Such cuts regulate exact or nearly
exact zero modes, while leaving important modes mostly unaffected.</p>
<p>Even if regulation works in such a case, the question remains as to why one
of the covariance matrices has a zero mode. A common cause is if the same
<code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code> was used for more than one prior. For example, one might
think that</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="kn">import</span> <span class="nn">gvar</span> <span class="k">as</span> <span class="nn">gv</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">z</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">prior</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">BufferDict</span><span class="p">(</span><span class="n">a</span><span class="o">=</span><span class="n">z</span><span class="p">,</span> <span class="n">b</span><span class="o">=</span><span class="n">z</span><span class="p">)</span>
</pre></div>
</div>
<p>creates a prior 1±1 for each of parameter <code class="docutils literal notranslate"><span class="pre">a</span></code> and parameter <code class="docutils literal notranslate"><span class="pre">b</span></code>.
Indeed each parameter separately is of order 1±1, but in a fit the two
parameters would be forced equal to each other because their priors are both
set equal to the same <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>, <code class="docutils literal notranslate"><span class="pre">z</span></code>:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">prior</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">],</span> <span class="n">prior</span><span class="p">[</span><span class="s1">&#39;b&#39;</span><span class="p">])</span>
<span class="go">1.0(1.0) 1.0(1.0)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">prior</span><span class="p">[</span><span class="s1">&#39;a&#39;</span><span class="p">]</span><span class="o">-</span><span class="n">prior</span><span class="p">[</span><span class="s1">&#39;b&#39;</span><span class="p">])</span>
<span class="go">0(0)</span>
</pre></div>
</div>
<p>That is, while parameters <code class="docutils literal notranslate"><span class="pre">a</span></code> and <code class="docutils literal notranslate"><span class="pre">b</span></code> fluctuate over a range of
1±1, they fluctuate together, in exact lock-step. The covariance matrix
for <code class="docutils literal notranslate"><span class="pre">a</span></code> and <code class="docutils literal notranslate"><span class="pre">b</span></code> must therefore be singular, with a zero mode corresponding
to the combination <code class="docutils literal notranslate"><span class="pre">a-b</span></code>; it is all 1s in this case:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">cov</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">evalcov</span><span class="p">(</span><span class="n">prior</span><span class="o">.</span><span class="n">flat</span><span class="p">)</span>    <span class="c1"># prior&#39;s covariance matrix</span>
<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">linalg</span><span class="o">.</span><span class="n">det</span><span class="p">(</span><span class="n">cov</span><span class="p">))</span>       <span class="c1"># determinant is zero</span>
<span class="go">0.0</span>
</pre></div>
</div>
<p>This zero mode upsets <code class="xref py py-func docutils literal notranslate"><span class="pre">nonlinear_fit()</span></code>. If <code class="docutils literal notranslate"><span class="pre">a</span></code> and <code class="docutils literal notranslate"><span class="pre">b</span></code> are meant to
fluctuate together then an SVD cut as above will give correct results (with
<code class="docutils literal notranslate"><span class="pre">a</span></code> and <code class="docutils literal notranslate"><span class="pre">b</span></code> being forced equal to several decimal places, depending upon
the cut). Of course, simply replacing <code class="docutils literal notranslate"><span class="pre">b</span></code> by <code class="docutils literal notranslate"><span class="pre">a</span></code> in the fit function would
be even better. If, on the other hand, <code class="docutils literal notranslate"><span class="pre">a</span></code> and <code class="docutils literal notranslate"><span class="pre">b</span></code> were not meant to
fluctuate together, the prior should be redefined:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="n">prior</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">BufferDict</span><span class="p">(</span><span class="n">a</span><span class="o">=</span><span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="n">b</span><span class="o">=</span><span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">))</span>
</pre></div>
</div>
<p>where now each parameter has its own <code class="xref py py-class docutils literal notranslate"><span class="pre">gvar.GVar</span></code>.
This line can be rewritten</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="n">prior</span> <span class="o">=</span> <span class="n">gv</span><span class="o">.</span><span class="n">gvar</span><span class="p">(</span><span class="nb">dict</span><span class="p">(</span><span class="n">a</span><span class="o">=</span><span class="s1">&#39;1(1)&#39;</span><span class="p">,</span> <span class="n">b</span><span class="o">=</span><span class="s1">&#39;1(1)&#39;</span><span class="p">))</span>
</pre></div>
</div>
<p>which is slighlty more succinct.</p>
</section>
</section>


            <div class="clearer"></div>
          </div>
        </div>
      </div>
      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
        <div class="sphinxsidebarwrapper">
  <div>
    <h3><a href="index.html">Table of Contents</a></h3>
    <ul>
<li><a class="reference internal" href="#">Overview and Tutorial</a><ul>
<li><a class="reference internal" href="#introduction">Introduction</a></li>
<li><a class="reference internal" href="#gaussian-random-variables-and-error-propagation">Gaussian Random Variables and Error Propagation</a></li>
<li><a class="reference internal" href="#basic-fits">Basic Fits</a></li>
<li><a class="reference internal" href="#chained-fits-large-data-sets">Chained Fits; Large Data Sets</a></li>
<li><a class="reference internal" href="#x-has-errors"><code class="docutils literal notranslate"><span class="pre">x</span></code> has Errors</a></li>
<li><a class="reference internal" href="#correlated-parameters-gaussian-bayes-factor">Correlated Parameters; Gaussian Bayes Factor</a></li>
<li><a class="reference internal" href="#y-has-no-error-marginalization"><code class="docutils literal notranslate"><span class="pre">y</span></code> has No Error; Marginalization</a></li>
<li><a class="reference internal" href="#svd-cuts-and-roundoff-error">SVD Cuts and Roundoff Error</a></li>
<li><a class="reference internal" href="#svd-cuts-and-inadequate-statistics">SVD Cuts and Inadequate Statistics</a></li>
<li><a class="reference internal" href="#y-has-unknown-errors"><code class="docutils literal notranslate"><span class="pre">y</span></code> has Unknown Errors</a></li>
<li><a class="reference internal" href="#tuning-priors-with-the-empirical-bayes-criterion">Tuning Priors with the Empirical Bayes Criterion</a></li>
<li><a class="reference internal" href="#positive-parameters-non-gaussian-priors">Positive Parameters; Non-Gaussian Priors</a></li>
<li><a class="reference internal" href="#faster-fitters">Faster Fitters</a></li>
<li><a class="reference internal" href="#debugging-and-troubleshooting">Debugging and Troubleshooting</a></li>
</ul>
</li>
</ul>

  </div>
  <div>
    <h4>Previous topic</h4>
    <p class="topless"><a href="index.html"
                          title="previous chapter">lsqfit Documentation</a></p>
  </div>
  <div>
    <h4>Next topic</h4>
    <p class="topless"><a href="testing.html"
                          title="next chapter">Non-Gaussian Behavior; Testing Fits</a></p>
  </div>
  <div role="note" aria-label="source link">
    <h3>This Page</h3>
    <ul class="this-page-menu">
      <li><a href="_sources/overview.rst.txt"
            rel="nofollow">Show Source</a></li>
    </ul>
   </div>
<div id="searchbox" style="display: none" role="search">
  <h3 id="searchlabel">Quick search</h3>
    <div class="searchformwrapper">
    <form class="search" action="search.html" method="get">
      <input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
      <input type="submit" value="Go" />
    </form>
    </div>
</div>
<script>document.getElementById('searchbox').style.display = "block"</script>
        </div>
      </div>
      <div class="clearer"></div>
    </div>
    <div class="related" role="navigation" aria-label="related navigation">
      <h3>Navigation</h3>
      <ul>
        <li class="right" style="margin-right: 10px">
          <a href="genindex.html" title="General Index"
             >index</a></li>
        <li class="right" >
          <a href="py-modindex.html" title="Python Module Index"
             >modules</a> |</li>
        <li class="right" >
          <a href="testing.html" title="Non-Gaussian Behavior; Testing Fits"
             >next</a> |</li>
        <li class="right" >
          <a href="index.html" title="lsqfit Documentation"
             >previous</a> |</li>
        <li class="nav-item nav-item-0"><a href="index.html">lsqfit 13.0.1 documentation</a> &#187;</li>
        <li class="nav-item nav-item-this"><a href="">Overview and Tutorial</a></li> 
      </ul>
    </div>
    <div class="footer" role="contentinfo">
        &#169; Copyright 2009-2019, G. P. Lepage.
      Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 5.0.2.
    </div>
  </body>
</html>