{"action":"opened","issue":{"url":"https://api.github.com/repos/ISA-tools/stato/issues/32","labels_url":"https://api.github.com/repos/ISA-tools/stato/issues/32/labels{/name}","comments_url":"https://api.github.com/repos/ISA-tools/stato/issues/32/comments","events_url":"https://api.github.com/repos/ISA-tools/stato/issues/32/events","html_url":"https://github.com/ISA-tools/stato/issues/32","id":58273456,"number":32,"title":"Basic linear model estimation methods","user":{"login":"nicholst","id":5155907,"avatar_url":"https://avatars.githubusercontent.com/u/5155907?v=3","gravatar_id":"","url":"https://api.github.com/users/nicholst","html_url":"https://github.com/nicholst","followers_url":"https://api.github.com/users/nicholst/followers","following_url":"https://api.github.com/users/nicholst/following{/other_user}","gists_url":"https://api.github.com/users/nicholst/gists{/gist_id}","starred_url":"https://api.github.com/users/nicholst/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nicholst/subscriptions","organizations_url":"https://api.github.com/users/nicholst/orgs","repos_url":"https://api.github.com/users/nicholst/repos","events_url":"https://api.github.com/users/nicholst/events{/privacy}","received_events_url":"https://api.github.com/users/nicholst/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-02-19T21:35:26Z","updated_at":"2015-02-19T21:35:26Z","closed_at":null,"body":"Following on discussions with @cmaumet, it seems notable that there are no definitions for the major flavours of estimation methods for linear models (aka multiple linear regression).  I'm referring to OLS, WLS, and GLS:\r\n\r\n* **Ordinary Least Squares**.  Estimation method for a linear regression model when the errors are uncorrelated and equal in variance.  Is the Best Linear Unbiased Estimation (BLUE) method under these assumptions, Uniformly Minimum-Variance Unbiased Estimator (UMVUE) with addition of a Gaussian assumption.  http://en.wikipedia.org/wiki/Ordinary_least_squares .\r\n* **Weighted Least Squares**.  Estimation method for a linear regression model with errors that independent but have heterogeneous variance.  Difficult to use use in practice, as weights must be set based on the variance which is usually unknown.   If true variance is known, it is the Best Linear Unbiased Estimation (BLUE) method under these assumptions, Uniformly Minimum-Variance Unbiased Estimator (UMVUE) with addition of a Gaussian assumption. http://en.wikipedia.org/wiki/Least_squares#Weighted_least_squares .  See also practical implementation as Iteratively Reweighted Least Squares.  http://en.wikipedia.org/wiki/Iteratively_reweighted_least_squares .\r\n* **Generalised Least Squares**. Estimation method for a linear regression model with errors that are dependent and (possibly) have heterogeneous variance. Difficult to use use in practice, as covariance matrix of the errors must known to \"whiten\" data and model.   If true covariance is known, it is the Best Linear Unbiased Estimation (BLUE) method under these assumptions, Uniformly Minimum-Variance Unbiased Estimator (UMVUE) with addition of a Gaussian assumption. http://en.wikipedia.org/wiki/Generalized_least_squares .  See also practical implementation as Feasible Generalized Least Squares  http://en.wikipedia.org/wiki/Generalized_least_squares#Feasible_generalized_least_squares .\r\n\r\nSince these reference yet 2 other methods, I might as well define those too...\r\n\r\n* **Iteratively Reweighted Least Squares**. Practical implementation of Weighted Least Squares, where the heterogeneous variances of the errors are estimated from the residuals of the regression model, providing an estimate for the weights. Each successive estimate of the weights improves the estimation of the regression parameters, which in turn are used to compute residuals and update the weights. http://en.wikipedia.org/wiki/Iteratively_reweighted_least_squares .\r\n* **Feasible Generalized Least Squares**.  Practical implementation Generalised Least Squares, where the covariance of the errors is estimated from the residuals of the regression model, providing the information needed to whiten the data and model. Each successive estimate of the whitening matrix improves the estimation of the regression parameters, which in turn are used to compute residuals and update the whitening matrix.  http://en.wikipedia.org/wiki/Generalized_least_squares#Feasible_generalized_least_squares .\r\n\r\nWhat do you think @proccaserra?  GLS is important for NIDM as fMRI time series models use it routinely."}}