Overparameterized Multiple Linear Regression as Hyper-Curve Fitting
E. Atza N. Budko
The paper shows that the application of the fixed-effect multiple linear
regression model to an overparameterized dataset is equivalent to fitting the
data with a hyper-curve parameterized by a single scalar parameter. This
equivalence allows for a predictor-focused approach, where each predictor is
described by a function of the chosen parameter. It is proven that a linear
model will produce exact predictions even in the presence of nonlinear
dependencies that violate the model assumptions. Parameterization in terms of
the dependent variable and the monomial basis in the predictor function space
are applied here to both synthetic and experimental data. The hyper-curve
approach is especially suited for the regularization of problems with noise in
predictor variables and can be used to remove noisy and "improper" predictors
from the model.