Prompt2Gaussia: Uncertain Prompt-learning for Script Event Prediction
Shiyao Cui Xin Cong Jiawei Sheng Xuebin Wang Tingwen Liu Jinqiao Shi
Script Event Prediction (SEP) aims to predict the subsequent event for a
given event chain from a candidate list. Prior research has achieved great
success by integrating external knowledge to enhance the semantics, but it is
laborious to acquisite the appropriate knowledge resources and retrieve the
script-related knowledge. In this paper, we regard public pre-trained language
models as knowledge bases and automatically mine the script-related knowledge
via prompt-learning. Still, the scenario-diversity and label-ambiguity in
scripts make it uncertain to construct the most functional prompt and label
token in prompt learning, i.e., prompt-uncertainty and verbalizer-uncertainty.
Considering the innate ability of Gaussian distribution to express uncertainty,
we deploy the prompt tokens and label tokens as random variables following
Gaussian distributions, where a prompt estimator and a verbalizer estimator are
proposed to estimate their probabilistic representations instead of
deterministic representations. We take the lead to explore prompt-learning in
SEP and provide a fresh perspective to enrich the script semantics. Our method
is evaluated on the most widely used benchmark and a newly proposed large-scale
one. Experiments show that our method, which benefits from knowledge evoked
from pre-trained language models, outperforms prior baselines by 1.46\% and
1.05\% on two benchmarks, respectively.