Venn: Resource Management Across Federated Learning Jobs
Jiachen Liu Fan Lai Ding Ding Yiwen Zhang Mosharaf Chowdhury
In recent years, federated learning (FL) has emerged as a promising approach
for machine learning (ML) and data science across distributed edge devices.
With the increasing popularity of FL, resource contention between multiple FL
jobs training on the same device population is increasing as well. Scheduling
edge resources among multiple FL jobs is different from GPU scheduling for
cloud ML because of the ephemeral nature and planetary scale of participating
devices as well as the overlapping resource requirements of diverse FL jobs.
Existing resource managers for FL jobs opt for random assignment of devices to
FL jobs for simplicity and scalability, which leads to poor performance. In
this paper, we present Venn, an FL resource manager, that efficiently schedules
ephemeral, heterogeneous devices among many FL jobs, with the goal of reducing
their average job completion time (JCT). Venn formulates the Intersection
Resource Scheduling (IRS) problem to identify complex resource contention among
multiple FL jobs. Then, Venn proposes a contention-aware scheduling heuristic
to minimize the average scheduling delay. Furthermore, it proposes a
resource-aware device-to-job matching heuristic that focuses on optimizing
response collection time by mitigating stragglers. Our evaluation shows that,
compared to the state-of-the-art FL resource managers, Venn improves the
average JCT by up to 1.88X.