CroSSHeteroFL: Cross-Stratified Sampling Composition-Fitting to Federated Learning for Heterogeneous Clients

Abstract

In the large-scale deployment of federated learning (FL) systems, the heterogeneity of clients, such as mobile phones and Internet of Things (IoT) devices with different configurations, constitutes a significant problem regarding fairness, training performance, and accuracy. Such system heterogeneity leads to an inevitable trade-off between model complexity and data accessibility as a bottleneck. To avoid this situation and to achieve resource-adaptive FL, we introduce CrossHeteroFL to deal with heterogeneous clients equipped with different computational and communication capabilities. Our solution enables the training of heterogeneous local models with additional computational complexity and still generates a single global inference model. We demonstrate several CrossHeteroFL training scenarios and conduct extensive empirical evaluation, covering four levels of the computational complexity of three-model architectures on two datasets. The proposed mechanism provides the system with non-elementary access to a scattered fit among clients. However, the proposed method generalizes soft handover-based solutions by adjusting the model width according to clients’ capabilities and a tiered balance of data-source overviews to assess clients’ interests accurately. The evaluation results indicate our method solves the challenges in previous studies and produces greater top-1 accuracy and consistent performance under heterogeneous client conditions.