Multimodal Deep Learning of Word-of-Mouth Text and Demographics to
  Predict Customer Rating: Handling Consumer Heterogeneity in Marketing
Junichiro Niimi
In the marketing field, understanding consumer heterogeneity, which is the
internal or psychological difference among consumers that cannot be captured by
behavioral logs, has long been a critical challenge. However, a number of
consumers today usually post their evaluation on the specific product on the
online platform, which can be the valuable source of such unobservable
differences among consumers. Several previous studies have shown the validity
of the analysis on text modality, but on the other hand, such analyses may not
necessarily demonstrate sufficient predictive accuracy for text alone, as they
may not include information readily available from cross-sectional data, such
as consumer profile data. In addition, recent advances in machine learning
techniques, such as large-scale language models (LLMs) and multimodal learning
have made it possible to deal with the various kind of dataset simultaneously,
including textual data and the traditional cross-sectional data, and the joint
representations can be effectively obtained from multiple modalities.
Therefore, this study constructs a product evaluation model that takes into
account consumer heterogeneity by multimodal learning of online product reviews
and consumer profile information. We also compare multiple models using
different modalities or hyper-parameters to demonstrate the robustness of
multimodal learning in marketing analysis.