Bridging the Domain Gaps in Context Representations for k-Nearest
  Neighbor Neural Machine Translation
Zhiwei Cao Baosong Yang Huan Lin Suhang Wu Xiangpeng Wei Dayiheng Liu Jun Xie Min Zhang Jinsong Su
$k$-Nearest neighbor machine translation ($k$NN-MT) has attracted increasing
attention due to its ability to non-parametrically adapt to new translation
domains. By using an upstream NMT model to traverse the downstream training
corpus, it is equipped with a datastore containing vectorized key-value pairs,
which are retrieved during inference to benefit translation. However, there
often exists a significant gap between upstream and downstream domains, which
hurts the retrieval accuracy and the final translation quality. To deal with
this issue, we propose a novel approach to boost the datastore retrieval of
$k$NN-MT by reconstructing the original datastore. Concretely, we design a
reviser to revise the key representations, making them better fit for the
downstream domain. The reviser is trained using the collected
semantically-related key-queries pairs, and optimized by two proposed losses:
one is the key-queries semantic distance ensuring each revised key
representation is semantically related to its corresponding queries, and the
other is an L2-norm loss encouraging revised key representations to effectively
retain the knowledge learned by the upstream NMT model. Extensive experiments
on domain adaptation tasks demonstrate that our method can effectively boost
the datastore retrieval and translation quality of $k$NN-MT.\footnote{Our code
is available at \url{https://github.com/DeepLearnXMU/RevisedKey-knn-mt}.}