Finite Sample Confidence Regions for Linear Regression Parameters Using
  Arbitrary Predictors
Charles Guille-Escuret Eugene Ndiaye
We explore a novel methodology for constructing confidence regions for
parameters of linear models, using predictions from any arbitrary predictor.
Our framework requires minimal assumptions on the noise and can be extended to
functions deviating from strict linearity up to some adjustable threshold,
thereby accommodating a comprehensive and pragmatically relevant set of
functions. The derived confidence regions can be cast as constraints within a
Mixed Integer Linear Programming framework, enabling optimisation of linear
objectives. This representation enables robust optimization and the extraction
of confidence intervals for specific parameter coordinates. Unlike previous
methods, the confidence region can be empty, which can be used for hypothesis
testing. Finally, we validate the empirical applicability of our method on
synthetic data.