Large Scale Generative Multimodal Attribute Extraction for E-commerce
  Attributes
Anant Khandelwal Happy Mittal Shreyas Sunil Kulkarni Deepak Gupta
E-commerce websites (e.g. Amazon) have a plethora of structured and
unstructured information (text and images) present on the product pages.
Sellers often either don't label or mislabel values of the attributes (e.g.
color, size etc.) for their products. Automatically identifying these attribute
values from an eCommerce product page that contains both text and images is a
challenging task, especially when the attribute value is not explicitly
mentioned in the catalog. In this paper, we present a scalable solution for
this problem where we pose attribute extraction problem as a question-answering
task, which we solve using \textbf{MXT}, consisting of three key components:
(i) \textbf{M}AG (Multimodal Adaptation Gate), (ii) \textbf{X}ception network,
and (iii) \textbf{T}5 encoder-decoder. Our system consists of a generative
model that \emph{generates} attribute-values for a given product by using both
textual and visual characteristics (e.g. images) of the product. We show that
our system is capable of handling zero-shot attribute prediction (when
attribute value is not seen in training data) and value-absent prediction (when
attribute value is not mentioned in the text) which are missing in traditional
classification-based and NER-based models respectively. We have trained our
models using distant supervision, removing dependency on human labeling, thus
making them practical for real-world applications. With this framework, we are
able to train a single model for 1000s of (product-type, attribute) pairs, thus
reducing the overhead of training and maintaining separate models. Extensive
experiments on two real world datasets show that our framework improves the
absolute recall@90P by 10.16\% and 6.9\% from the existing state of the art
models. In a popular e-commerce store, we have deployed our models for 1000s of
(product-type, attribute) pairs.