{"action":"created","issue":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4119","labels_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4119/labels{/name}","comments_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4119/comments","events_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4119/events","html_url":"https://github.com/scikit-learn/scikit-learn/pull/4119","id":54700690,"number":4119,"title":"[WIP] Constrained Regression Estimator","user":{"login":"xbhsu","id":7433754,"avatar_url":"https://avatars.githubusercontent.com/u/7433754?v=3","gravatar_id":"","url":"https://api.github.com/users/xbhsu","html_url":"https://github.com/xbhsu","followers_url":"https://api.github.com/users/xbhsu/followers","following_url":"https://api.github.com/users/xbhsu/following{/other_user}","gists_url":"https://api.github.com/users/xbhsu/gists{/gist_id}","starred_url":"https://api.github.com/users/xbhsu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/xbhsu/subscriptions","organizations_url":"https://api.github.com/users/xbhsu/orgs","repos_url":"https://api.github.com/users/xbhsu/repos","events_url":"https://api.github.com/users/xbhsu/events{/privacy}","received_events_url":"https://api.github.com/users/xbhsu/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2015-01-18T16:02:09Z","updated_at":"2015-01-21T08:09:08Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/pulls/4119","html_url":"https://github.com/scikit-learn/scikit-learn/pull/4119","diff_url":"https://github.com/scikit-learn/scikit-learn/pull/4119.diff","patch_url":"https://github.com/scikit-learn/scikit-learn/pull/4119.patch"},"body":"sklearn currently doesn’t support constrained linear regression models and allow people to fit linear models where the coefficients follow some constraint. One use case is using feature importance scores to constrain the value of the coefficients so that their net effects reflect their relative importances. A good amount of literature has been written on this (see below). This pull request aims to provide a constrained regression linear estimator and includes the a model where fit parameters are constrained based on a variance decomposition (Shapley values). It would add an alternative to the ridge regression methods already in sklearn. Hopefully other people find it useful to have as well. \r\n\r\nExample usage follows the linear_model API's:\r\n```python\r\nfrom sklearn.linear_model import ShapleyRegression\r\nmodel = ShapleyRegression.fit(X, y)\r\ncoef = model.coef_\r\nintercept = model.intercept_\r\n```\r\n\r\nReferences:\r\n\r\n1. Zuber, V. and Strimmer, K. (2010). Variable importance and model selection by decorrelation. [1007.5516](http://arxiv.org/abs/1007.5516)\r\n2. Grömping, U. (2007). Estimators of Relative Importance in Linear Regression Based on Variance Decomposition. The American Statistician 61, 139-147.\r\n3. Genizi, A. (1993). Decomposition of R2 in multiple regression with correlated regressors. Statistica Sinica 3, 407-420.\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/comments/70799053","html_url":"https://github.com/scikit-learn/scikit-learn/pull/4119#issuecomment-70799053","issue_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4119","id":70799053,"user":{"login":"agramfort","id":161052,"avatar_url":"https://avatars.githubusercontent.com/u/161052?v=3","gravatar_id":"","url":"https://api.github.com/users/agramfort","html_url":"https://github.com/agramfort","followers_url":"https://api.github.com/users/agramfort/followers","following_url":"https://api.github.com/users/agramfort/following{/other_user}","gists_url":"https://api.github.com/users/agramfort/gists{/gist_id}","starred_url":"https://api.github.com/users/agramfort/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/agramfort/subscriptions","organizations_url":"https://api.github.com/users/agramfort/orgs","repos_url":"https://api.github.com/users/agramfort/repos","events_url":"https://api.github.com/users/agramfort/events{/privacy}","received_events_url":"https://api.github.com/users/agramfort/received_events","type":"User","site_admin":false},"created_at":"2015-01-21T08:09:08Z","updated_at":"2015-01-21T08:09:08Z","body":"> > I can not really judge this one and I'd rather wait on input from\n> > @agramfort or @GaelVaroquaux who are much more familiar with these\n> > kinds of methods\n>\n> I am not convinced that it is widely-used enough.\n\nsame here but could you share results plots as in :\n\nhttp://scikit-learn.org/stable/auto_examples/linear_model/plot_lasso_and_elasticnet.html\nor\nhttp://scikit-learn.org/stable/auto_examples/linear_model/plot_ard.html\n\nto illustrates what it brings\n\n?"}}