Symbolic Regression as Feature Engineering Method for Machine and Deep
  Learning Regression Tasks
Assaf Shmuel Oren Glickman Teddy Lazebnik
In the realm of machine and deep learning regression tasks, the role of
effective feature engineering (FE) is pivotal in enhancing model performance.
Traditional approaches of FE often rely on domain expertise to manually design
features for machine learning models. In the context of deep learning models,
the FE is embedded in the neural network's architecture, making it hard for
interpretation. In this study, we propose to integrate symbolic regression (SR)
as an FE process before a machine learning model to improve its performance. We
show, through extensive experimentation on synthetic and real-world
physics-related datasets, that the incorporation of SR-derived features
significantly enhances the predictive capabilities of both machine and deep
learning regression models with 34-86% root mean square error (RMSE)
improvement in synthetic datasets and 4-11.5% improvement in real-world
datasets. In addition, as a realistic use-case, we show the proposed method
improves the machine learning performance in predicting superconducting
critical temperatures based on Eliashberg theory by more than 20% in terms of
RMSE. These results outline the potential of SR as an FE component in
data-driven models.