The Grand Illusion: The Myth of Software Portability and Implications
  for ML Progress
Fraser Mince Dzung Dinh Jonas Kgomo Neil Thompson Sara Hooker
Pushing the boundaries of machine learning often requires exploring different
hardware and software combinations. However, the freedom to experiment across
different tooling stacks can be at odds with the drive for efficiency, which
has produced increasingly specialized AI hardware and incentivized
consolidation around a narrow set of ML frameworks. Exploratory research can be
restricted if software and hardware are co-evolving, making it even harder to
stray away from mainstream ideas that work well with popular tooling stacks.
While this friction increasingly impacts the rate of innovation in machine
learning, to our knowledge the lack of portability in tooling has not been
quantified. In this work, we ask: How portable are popular ML software
frameworks? We conduct a large-scale study of the portability of mainstream ML
frameworks across different hardware types. Our findings paint an uncomfortable
picture -- frameworks can lose more than 40% of their key functions when ported
to other hardware. Worse, even when functions are portable, the slowdown in
their performance can be extreme and render performance untenable.
Collectively, our results reveal how costly straying from a narrow set of
hardware-software combinations can be - and suggest that specialization of
hardware impedes innovation in machine learning research.