An Adaptive Federated Learning Algorithm with Attenuated Memory on Non-IID and Long-Tail Data

References

• Chen HY, Chao WL (2020) FedBE: Making Bayesian model ensemble applicable to federated learning. Preprint, submitted September 4, https://arxiv.org/abs/2009.01974.Google Scholar
• Cui Y, Jia M, Lin TY, Song Y, Belongie S (2019) Class-balanced loss based on effective number of samples. Proc. IEEE/CVF Conf. Comput. Vision Pattern Recognition (IEEE, Piscataway, NJ), 9268–9277.Google Scholar
• Dal Pozzolo A, Caelen O, Johnson RA, Bontempi G (2015) Calibrating probability with undersampling for unbalanced classification. IEEE Sympos. Ser. Comput. Intelligence (IEEE, Piscataway, NJ), 159–166.Google Scholar
• Elgabli A, Issaid CB, Bedi AS, Rajawat K, Bennis M, Aggarwal V (2022) FedNew: A communication-efficient and privacy-preserving Newton-type method for federated learning. 39th ICML (PMLR, New York), 5861–5877.Google Scholar
• Geiping J, Bauermeister H, Dröge H, Moeller M (2020) Inverting gradients-how easy is it to break privacy in federated learning? Adv. Neural Inform. Processing Systems, vol. 33 (Curran Associates Inc., Red Hook, NY), 16937–16947.Google Scholar
• Haddadpour F, Mahdavi M (2019) On the convergence of local descent methods in federated learning. Preprint, submitted October 31, https://arxiv.org/abs/1910.14425.Google Scholar
• Hsu TMH, Qi H, Brown M (2019) Measuring the effects of non-identical data distribution for federated visual classification. Preprint, submitted September 13, https://arxiv.org/abs/1909.06335.Google Scholar
• Huang Z, Yang H, Wu H, Yang C (2026) An adaptive federated learning algorithm with attenuated memory on non-IID and long-tail data. https://doi.org/10.1287/ijoc.2024.0765.cd, https://github.com/INFORMSJoC/2024.0765.Google Scholar
• Institute of Medical Technology (2025) Federated learning platform for multi-center pediatric pneumonia risk prediction. https://imt.bjmu.edu.cn/en/NEWS_new/930fb091c2c04b598e94aa262257fd4e.htm.Google Scholar
• Kang B, Xie S, Rohrbach M, Yan Z, Gordo A, Feng J, Kalantidis Y (2019) Decoupling representation and classifier for long-tailed recognition. Preprint, submitted October 21, https://arxiv.org/abs/1910.09217.Google Scholar
• Li T, Hu S, Beirami A, Smith V (2021a) Ditto: Fair and robust federated learning through personalization. Proc. 38th Internat. Conf. Machine Learn. (PMLR, New York), 6357–6368.Google Scholar
• Li Z, Lin T, Shang X, Wu C (2023) Revisiting weighted aggregation in federated learning with neural networks. Internat. Conf. Machine Learn. (ACM, New York), 19767–19788.Google Scholar
• Li T, Sanjabi M, Beirami A, Smith V (2019a) Fair resource allocation in federated learning. Preprint, submitted May 25, https://arxiv.org/abs/1905.10497.Google Scholar
• Li X, Huang K, Yang W, Wang S, Zhang Z (2019b) On the convergence of FedAvg on non-IID data. Preprint, submitted July 4, https://arxiv.org/abs/1907.02189.Google Scholar
• Li X, Jiang M, Zhang X, Kamp M, Dou Q (2021b) FedBN: Federated learning on non-IID features via local batch normalization. Preprint, submitted February 15, https://arxiv.org/abs/2102.07623.Google Scholar
• Li T, Sahu AK, Zaheer M, Sanjabi M, Talwalkar A, Smith V (2020) Federated optimization in heterogeneous networks. Proc. Machine Learn. Systems (OpenReview), 429–450.Google Scholar
• Lin T, Kong L, Stich SU, Jaggi M (2020) Ensemble distillation for robust model fusion in federated learning. Proc. 34th Internat. Conf. Neural Inform. Processing Systems (Curran Associates Inc., Red Hook, NY), 2351–2363.Google Scholar
• Luo M, Chen F, Hu D, Zhang Y, Liang J, Feng J (2021) No fear of heterogeneity: Classifier calibration for federated learning with non-IID data. Adv. Neural Inform. Processing Systems (ACM, New York), 5972–5984.Google Scholar
• Lyu L, Yu H, Yang Q (2020) Threats to federated learning: A survey. Preprint, submitted March 4, https://arxiv.org/abs/2003.02133. Google Scholar
• McMahan B, Moore E, Ramage D, Hampson S, Agüera y Arcas B (2017) Communication-efficient learning of deep networks from decentralized data. Internat. Conf. Artificial Intelligence Statist. (PMLR, New York), 1273–1282.Google Scholar
• Park S, Lim J, Jeon Y, Choi JY (2021) Influence-balanced loss for imbalanced visual classification. Proc. IEEE/CVF Internat. Conf. Comput. Vision (IEEE, Piscataway, NJ), 735–744.Google Scholar
• Sarkar D, Narang A, Rai S (2020) Fed-focal loss for imbalanced data classification in federated learning. Preprint, submitted November 12, https://arxiv.org/abs/2011.06283.Google Scholar
• Shang X, Lu Y, Huang G, Wang H (2022) Federated learning on heterogeneous and long-tailed data via classifier re-training with federated features. 31st Internat. Joint Conf. Artificial Intelligence (IEEE Computer Society, Washington, DC), 2218–2224.Google Scholar
• Sheller MJ, Edwards B, Reina GA, Martin J, Pati S, Kotrotsou A, Milchenko M, et al. (2020) Federated learning in medicine: Facilitating multi-institutional collaborations without sharing patient data. Sci. Rep. 10(1):12598.Crossref, Google Scholar
• Stich SU (2019) Local SGD converges fast and communicates little. Internat. Conf. Learn. Representations (OpenReview), 1–17.Google Scholar
• Stich SU, Cordonnier JB, Jaggi M (2018) Sparsified SGD with memory. Proc. 32nd Internat. Conf. Neural Inform. Processing Systems (Curran Associates Inc., Red Hook, NY), 4452–4463.Google Scholar
• Swift (2024) Swift to launch Ai-powered fraud defence to enhance cross-border payments. https://www.swift.com/news-events/press-releases/swift-launch-ai-powered-fraud-defence-enhance-cross-border-payments.Google Scholar
• Tan AZ, Yu H, Cui L, Yang Q (2022) Towards personalized federated learning. IEEE Trans. Neural Networks Learn. Systems 34(12):9587–9603.Crossref, Google Scholar
• Tan J, Lu X, Zhang G, Yin C, Li Q (2021) Equalization loss V2: A new gradient balance approach for long-tailed object detection. Proc. IEEE/CVF Conf. Comput. Vision Pattern Recognition (IEEE, Piscataway, NJ), 1685–1694.Google Scholar
• Wang L, Xu S, Wang X, Zhu Q (2021) Addressing class imbalance in federated learning. 35th AAAI Conf. Artificial Intelligence (AAAI Press, Palo Alto, CA), 10165–10173.Google Scholar
• Wang J, Liu Q, Liang H, Joshi G, Poor HV (2020) Tackling the objective inconsistency problem in heterogeneous federated optimization. 34th Internat. Conf. Neural Inform. Processing Systems (Curran Associates Inc., Red Hook, NY), 7611–7623.Google Scholar
• Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM (2017) Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. Proc. IEEE Conf. Comput. Vision Pattern Recognition (IEEE, Piscataway, NJ), 2097–2106.Google Scholar
• Wu H, Wang P (2021) Fast-convergent federated learning with adaptive weighting. IEEE Trans. Cognitive Comm. Networking 7(4):1078–1088.Crossref, Google Scholar
• Xiao Z, Chen Z, Liu S, Wang H, Feng Y, Hao J, Zhou JT, Wu J, Yang HH, Liu Z (2023) Fed-GraB: Federated long-tailed learning with self-adjusting gradient balancer. Preprint, submitted October 11, https://arxiv.org/abs/2310.07587.Google Scholar
• Yue X, Kontar R (2024) Federated Gaussian process: Convergence, automatic personalization and multi-fidelity modeling. IEEE Trans. Pattern Anal. Machine Intelligence 46(6):4246–4261.Crossref, Google Scholar
• Yue X, Nouiehed M, Al Kontar R (2023) GIFAIR-FL: A framework for group and individual fairness in federated learning. INFORMS J. Data Sci. 2(1):10–23.Link, Google Scholar
• Zeng Y, Chen H, Lee K (2021) Improving fairness via federated learning. Preprint, submitted October 29, https://arxiv.org/abs/2110.15545.Google Scholar
• Zhang Y, Kang B, Hooi B, Yan S, Feng J (2023) Deep long-tailed learning: A survey. IEEE Trans. Pattern Anal. Machine Intelligence 45(9):10795–10816.Crossref, Google Scholar
• Zhao X, Shen D (2024) FedSW: Federated learning with adaptive sample weights. Inform. Sci. 654:119873.Crossref, Google Scholar
• Zhao Y, Li M, Lai L, Suda N, Civin D, Chandra V (2018) Federated learning with non-IID data. Preprint, submitted June 2, https://arxiv.org/abs/1806.00582.Google Scholar
• Zhu L, Liu Z, Han S (2019) Deep leakage from gradients. Adv. Neural Inform. Processing Systems (Curran Associates Inc., Red Hook, NY), 32:1–11.Google Scholar