Anticipating Technical Expertise and Capability Evolution in Research
  Communities using Dynamic Graph Transformers
Sameera Horawalavithana Ellyn Ayton Anastasiya Usenko Robin Cosbey Svitlana Volkova
The ability to anticipate technical expertise and capability evolution trends
globally is essential for national and global security, especially in
safety-critical domains like nuclear nonproliferation (NN) and rapidly emerging
fields like artificial intelligence (AI). In this work, we extend traditional
statistical relational learning approaches (e.g., link prediction in
collaboration networks) and formulate a problem of anticipating technical
expertise and capability evolution using dynamic heterogeneous graph
representations. We develop novel capabilities to forecast collaboration
patterns, authorship behavior, and technical capability evolution at different
granularities (e.g., scientist and institution levels) in two distinct research
fields. We implement a dynamic graph transformer (DGT) neural architecture,
which pushes the state-of-the-art graph neural network models by (a)
forecasting heterogeneous (rather than homogeneous) nodes and edges, and (b)
relying on both discrete -- and continuous -- time inputs. We demonstrate that
our DGT models predict collaboration, partnership, and expertise patterns with
0.26, 0.73, and 0.53 mean reciprocal rank values for AI and 0.48, 0.93, and
0.22 for NN domains. DGT model performance exceeds the best-performing static
graph baseline models by 30-80% across AI and NN domains. Our findings
demonstrate that DGT models boost inductive task performance, when previously
unseen nodes appear in the test data, for the domains with emerging
collaboration patterns (e.g., AI). Specifically, models accurately predict
which established scientists will collaborate with early career scientists and
vice-versa in the AI domain.