Incorporating LLM Priors into Tabular Learners
Max Zhu Siniša Stanivuk Andrija Petrovic Mladen Nikolic Pietro Lio
We present a method to integrate Large Language Models (LLMs) and traditional
tabular data classification techniques, addressing LLMs challenges like data
serialization sensitivity and biases. We introduce two strategies utilizing
LLMs for ranking categorical variables and generating priors on correlations
between continuous variables and targets, enhancing performance in few-shot
scenarios. We focus on Logistic Regression, introducing MonotonicLR that
employs a non-linear monotonic function for mapping ordinals to cardinals while
preserving LLM-determined orders. Validation against baseline models reveals
the superior performance of our approach, especially in low-data scenarios,
while remaining interpretable.