Context-Enhanced Relational Operators with Vector Embeddings
Viktor Sanca Manos Chatzakis Anastasia Ailamaki
Collecting data, extracting value, and combining insights from relational and
context-rich multi-modal sources in data processing pipelines presents a
challenge for traditional relational DBMS. While relational operators allow
declarative and optimizable query specification, they are limited to data
transformations unsuitable for capturing or analyzing context. On the other
hand, representation learning models can map context-rich data into embeddings,
allowing machine-automated context processing but requiring imperative data
transformation integration with the analytical query.
  To bridge this dichotomy, we present a context-enhanced relational join and
introduce an embedding operator composable with relational operators. This
enables hybrid relational and context-rich vector data processing, with
algebraic equivalences compatible with relational algebra and corresponding
logical and physical optimizations. We investigate model-operator interaction
with vector data processing and study the characteristics of the E-join
operator. Using an example of string embeddings, we demonstrate enabling hybrid
context-enhanced processing on relational join operators with vector
embeddings. The importance of holistic optimization, from logical to physical,
is demonstrated in an order of magnitude execution time improvement.