I have a system of linear equations that is derived partially from experimental data. Theoretically, the system should have a single, exact solution; however, experimental error causes it to not have an exact solution. Is there a good method to find an approximate solution to this system? My idea for a naive implementation was to solve all pairs of two equations and average the value for the solutions to each variable, but this doesn't extend well as the number of equations increases. Is there any known algorithm to do this?

  • 4
    $\begingroup$ least squares might be a good candidate? $\endgroup$ – James Nov 26 '15 at 6:19
  • 3
    $\begingroup$ It's important to specify whether the linear system is in fact invertible (meaning you do get a single solution, although it might not be exactly the one you're looking for), underdetermined (you get multiple solutions, but might be able to pick a reasonable one), or overdetermined (there is no solution unless you hit the exact data). $\endgroup$ – Christian Clason Nov 26 '15 at 9:12
  • 3
    $\begingroup$ Also, do the experimental error enter into the matrix (i.e., coefficients of the linear system) or the right-hand side? $\endgroup$ – Christian Clason Nov 26 '15 at 9:13
  • $\begingroup$ And how many equations and variables are there? How do you solve all pairs of equations? Least squares (or are there only two variables?). $\endgroup$ – Dirk Jan 26 '16 at 5:23

It seems you are looking for an optimization (here, specifically a fit).

You have some experimental data and you want to have a simple model explaining it.

The simplest way is using the Ordinary Least Squares. The idea is to minimize the standard deviation or the error you make on the vertical axis.

Another one would be the PCA where you try to find an angle to view your variables to find the maximum information. It's based on the eigenvectors of your system. The advantage is that it only tries to find the best solution to minimize the orthogonal error.

Many other solutions are possibles but it would be preferable to have more details about what you're exactly looking for and the link between variables.

  • $\begingroup$ so this is essentially linear regression? $\endgroup$ – Akababa Mar 28 '19 at 2:54

You can use any of the numerical methods that is for linear equation systems. Some of them are;

  • Gauss-Jordan Elimination
  • Jacobi Method
  • Gauss-Seidel
  • Successive Over Relaxation(SOR)

For more info, you can look at this

  • 6
    $\begingroup$ I'm sorry, but this is bad advice. If the linear system does not have an exact solution, trying to compute it (no matter how) will necessarily blow up in your face. $\endgroup$ – Christian Clason Nov 26 '15 at 9:10

Your Answer

