Concurrent Classifier Error Detection (CCED) in Large Scale Machine
  Learning Systems
Pedro Reviriego Ziheng Wang Alvaro Alonso Zhen Gao Farzad Niknia Shanshan Liu Fabrizio Lombardi
The complexity of Machine Learning (ML) systems increases each year, with
current implementations of large language models or text-to-image generators
having billions of parameters and requiring billions of arithmetic operations.
As these systems are widely utilized, ensuring their reliable operation is
becoming a design requirement. Traditional error detection mechanisms introduce
circuit or time redundancy that significantly impacts system performance. An
alternative is the use of Concurrent Error Detection (CED) schemes that operate
in parallel with the system and exploit their properties to detect errors. CED
is attractive for large ML systems because it can potentially reduce the cost
of error detection. In this paper, we introduce Concurrent Classifier Error
Detection (CCED), a scheme to implement CED in ML systems using a concurrent ML
classifier to detect errors. CCED identifies a set of check signals in the main
ML system and feeds them to the concurrent ML classifier that is trained to
detect errors. The proposed CCED scheme has been implemented and evaluated on
two widely used large-scale ML models: Contrastive Language Image Pretraining
(CLIP) used for image classification and Bidirectional Encoder Representations
from Transformers (BERT) used for natural language applications. The results
show that more than 95 percent of the errors are detected when using a simple
Random Forest classifier that is order of magnitude simpler than CLIP or BERT.
These results illustrate the potential of CCED to implement error detection in
large-scale ML models.