An Adaptive Federated Learning Algorithm with Attenuated Memory on Non-IID and Long-Tail Data

Data sharing and privacy protection in the business sector present dual challenges, especially in the financial and medical fields. We propose an adaptive federated learning algorithm with attenuated memory (AFLAM) to address three critical problems in federated learning: nonindependently and nonidentically distributed (non-IID), long-tail distribution, and privacy. AFLAM dynamically recalculates each client’s weight with attenuated memory of the gradient history to mitigate bias from non-IID data, enhancing the modeling ability of tail data. AFLAM protects privacy by transmitting scaled instead of true gradient information. Two AFLAM algorithms, client-based and parameter-based, are proposed. AFLAM performs better with non-IID and long-tail distribution than existing state-of-the-art methods, improving accuracy by up to 5.71%. Our algorithm provides a novel approach to data sharing and privacy protection challenges.

History: Accepted by Ram Ramesh, Area Editor for Data Science & Machine Learning.

Funding: H. Yang is grateful for financial support from the National Natural Science Foundation of China [Grant 71771006] and the Beijing Jianlong Heavy Industry Program [Grant 20251202].

Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information (https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2024.0765) as well as from the IJOC GitHub software repository (https://github.com/INFORMSJoC/2024.0765). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/.