Fairer and More Accurate Tabular Models Through NAS
Richeek Das Samuel Dooley
Making models algorithmically fairer in tabular data has been long studied,
with techniques typically oriented towards fixes which usually take a neural
model with an undesirable outcome and make changes to how the data are
ingested, what the model weights are, or how outputs are processed. We employ
an emergent and different strategy where we consider updating the model's
architecture and training hyperparameters to find an entirely new model with
better outcomes from the beginning of the debiasing procedure. In this work, we
propose using multi-objective Neural Architecture Search (NAS) and
Hyperparameter Optimization (HPO) in the first application to the very
challenging domain of tabular data. We conduct extensive exploration of
architectural and hyperparameter spaces (MLP, ResNet, and FT-Transformer)
across diverse datasets, demonstrating the dependence of accuracy and fairness
metrics of model predictions on hyperparameter combinations. We show that
models optimized solely for accuracy with NAS often fail to inherently address
fairness concerns. We propose a novel approach that jointly optimizes
architectural and training hyperparameters in a multi-objective constraint of
both accuracy and fairness. We produce architectures that consistently Pareto
dominate state-of-the-art bias mitigation methods either in fairness, accuracy
or both, all of this while being Pareto-optimal over hyperparameters achieved
through single-objective (accuracy) optimization runs. This research
underscores the promise of automating fairness and accuracy optimization in
deep learning models.