Foundation Models Meet Visualizations: Challenges and Opportunities
Weikai Yang Mengchen Liu Zheng Wang Shixia Liu
Recent studies have indicated that foundation models, such as BERT and GPT,
excel in adapting to a variety of downstream tasks. This adaptability has
established them as the dominant force in building artificial intelligence (AI)
systems. As visualization techniques intersect with these models, a new
research paradigm emerges. This paper divides these intersections into two main
areas: visualizations for foundation models (VIS4FM) and foundation models for
visualizations (FM4VIS). In VIS4FM, we explore the primary role of
visualizations in understanding, refining, and evaluating these intricate
models. This addresses the pressing need for transparency, explainability,
fairness, and robustness. Conversely, within FM4VIS, we highlight how
foundation models can be utilized to advance the visualization field itself.
The confluence of foundation models and visualizations holds great promise, but
it also comes with its own set of challenges. By highlighting these challenges
and the growing opportunities, this paper seeks to provide a starting point for
continued exploration in this promising avenue.