Basis Function Encoding of Numerical Features in Factorization Machines
  for Improved Accuracy
Alex Shtoff Elie Abboud Rotem Stram Oren Somekh
Factorization machine (FM) variants are widely used for large scale real-time
content recommendation systems, since they offer an excellent balance between
model accuracy and low computational costs for training and inference. These
systems are trained on tabular data with both numerical and categorical
columns. Incorporating numerical columns poses a challenge, and they are
typically incorporated using a scalar transformation or binning, which can be
either learned or chosen a-priori. In this work, we provide a systematic and
theoretically-justified way to incorporate numerical features into FM variants
by encoding them into a vector of function values for a set of functions of
one's choice.
  We view factorization machines as approximators of segmentized functions,
namely, functions from a field's value to the real numbers, assuming the
remaining fields are assigned some given constants, which we refer to as the
segment. From this perspective, we show that our technique yields a model that
learns segmentized functions of the numerical feature spanned by the set of
functions of one's choice, namely, the spanning coefficients vary between
segments. Hence, to improve model accuracy we advocate the use of functions
known to have strong approximation power, and offer the B-Spline basis due to
its well-known approximation power, availability in software libraries, and
efficiency. Our technique preserves fast training and inference, and requires
only a small modification of the computational graph of an FM model. Therefore,
it is easy to incorporate into an existing system to improve its performance.
Finally, we back our claims with a set of experiments, including synthetic,
performance evaluation on several data-sets, and an A/B test on a real online
advertising system which shows improved performance.