On Extending the Automatic Test Markup Language (ATML) for Machine
  Learning
Tyler Cody Bingtong Li Peter A. Beling
This paper addresses the urgent need for messaging standards in the
operational test and evaluation (T&E) of machine learning (ML) applications,
particularly in edge ML applications embedded in systems like robots,
satellites, and unmanned vehicles. It examines the suitability of the IEEE
Standard 1671 (IEEE Std 1671), known as the Automatic Test Markup Language
(ATML), an XML-based standard originally developed for electronic systems, for
ML application testing. The paper explores extending IEEE Std 1671 to encompass
the unique challenges of ML applications, including the use of datasets and
dependencies on software. Through modeling various tests such as adversarial
robustness and drift detection, this paper offers a framework adaptable to
specific applications, suggesting that minor modifications to ATML might
suffice to address the novelties of ML. This paper differentiates ATML's focus
on testing from other ML standards like Predictive Model Markup Language (PMML)
or Open Neural Network Exchange (ONNX), which concentrate on ML model
specification. We conclude that ATML is a promising tool for effective, near
real-time operational T&E of ML applications, an essential aspect of AI
lifecycle management, safety, and governance.