Divide et Impera: Multi-Transformer Architectures for Complex NLP-Tasks
Solveig Helland Elena Gavagnin Alexandre de Spindler
The growing capabilities of transformer models pave the way for solving
increasingly complex NLP tasks. A key to supporting application-specific
requirements is the ability to fine-tune. However, compiling a fine-tuning
dataset tailored to complex tasks is tedious and results in large datasets,
limiting the ability to control transformer output. We present an approach in
which complex tasks are divided into simpler subtasks. Multiple transformer
models are fine-tuned to one subtask each, and lined up to accomplish the
complex task. This simplifies the compilation of fine-tuning datasets and
increases overall controllability. Using the example of reducing gender bias as
a complex task, we demonstrate our approach and show that it performs better
than using a single model.