Regulating workers in federated learning by Yardstick competition

Sarıkaya, Yunus and Erçetin, Özgür (2020) Regulating workers in federated learning by Yardstick competition. In: 13th EAI International Conference on Performance Evaluation Methodologies and Tools (VALUETOOLS 2020), Tsukuba, Japan

[thumbnail of 3388831.3388843.pdf] PDF
3388831.3388843.pdf
Restricted to Registered users only

Download (528kB) | Request a copy

Abstract

Due to the large size of the training data, distributed learning approaches such as federated learning have gained attention recently. However, the convergence rate of distributed learning suffers from heterogeneous worker performance. In this paper, we consider an incentive mechanism for workers to mitigate the delays in completion of each batch. To motivate the workers to perform at their best by assigning higher computational resources to the learning task, we use a yardstick of average desired delay to complete each mini-batch calculation. The rewards are determined by how much each worker deviates from this yardstick. We analytically obtain the optimum equilibrium strategy of the workers as well as the optimal reward function of the model owner that achieves the average desired delay while minimizing the cost of operation. Our numerical results indicate that by adjusting budget parameters, the model owner should judiciously decide on the number of workers due to trade off between the diversity provided by the number of workers and the latency of completing the training.
Item Type: Papers in Conference Proceedings
Subjects: T Technology > TK Electrical engineering. Electronics Nuclear engineering > TK5101-6720 Telecommunication
Divisions: Faculty of Engineering and Natural Sciences > Academic programs > Electronics
Faculty of Engineering and Natural Sciences
Depositing User: Özgür Erçetin
Date Deposited: 17 Sep 2020 12:20
Last Modified: 02 Aug 2023 12:44
URI: https://research.sabanciuniv.edu/id/eprint/40194

Actions (login required)

View Item
View Item