Towards Graph Foundation Models for Personalization
Andreas Damianou Francesco Fabbri Paul Gigioli Marco De Nadai Alice Wang Enrico Palumbo Mounia Lalmas
In the realm of personalization, integrating diverse information sources such
as consumption signals and content-based representations is becoming
increasingly critical to build state-of-the-art solutions. In this regard, two
of the biggest trends in research around this subject are Graph Neural Networks
(GNNs) and Foundation Models (FMs). While GNNs emerged as a popular solution in
industry for powering personalization at scale, FMs have only recently caught
attention for their promising performance in personalization tasks like ranking
and retrieval. In this paper, we present a graph-based foundation modeling
approach tailored to personalization. Central to this approach is a
Heterogeneous GNN (HGNN) designed to capture multi-hop content and consumption
relationships across a range of recommendable item types. To ensure the
generality required from a Foundation Model, we employ a Large Language Model
(LLM) text-based featurization of nodes that accommodates all item types, and
construct the graph using co-interaction signals, which inherently transcend
content specificity. To facilitate practical generalization, we further couple
the HGNN with an adaptation mechanism based on a two-tower (2T) architecture,
which also operates agnostically to content type. This multi-stage approach
ensures high scalability; while the HGNN produces general purpose embeddings,
the 2T component models in a continuous space the sheer size of user-item
interaction data. Our comprehensive approach has been rigorously tested and
proven effective in delivering recommendations across a diverse array of
products within a real-world, industrial audio streaming platform.