Toward Graph Data Collaboration in a Data-Sharing-Free Manner: A Novel Privacy-Preserving Graph Pretraining Model

References

• Abadi M, Chu A, Goodfellow I, McMahan HB, Mironov I, Talwar K, Zhang L (2016) Deep learning with differential privacy. Weippl E, Katzenbeisser S, Kruegel C, Myers A, Halevi S, eds. Proc. 2016 ACM SIGSAC Conf. Comput. Comm. Security (Association for Computing Machinery, New York), 308–318.Google Scholar
• Arora R, Upadhyay J (2019) On differentially private graph sparsification and applications. Wallach H, Larochelle H, Beygelzimer A, d’Alché-Buc F, Fox E, Garnett R, eds. Advances in Neural Information Processing Systems, vol. 32 (Curran Associates, Red Hook, NY), 13411–13422.Google Scholar
• Barabási AL (2013) Network science. Philos. Trans. Roy. Soc. A Math. Phys. Engrg. Sci. 371(1987):20120375.Crossref, Google Scholar
• Barabási AL, Albert R (1999) Emergence of scaling in random networks. Science 286(5439):509–512.Crossref, Google Scholar
• Bauer M, van Dinther C, Kiefer D (2020) Machine learning in SME: An empirical study on enablers and success factors. Anderson B, Thatcher J, Meservy R, Chudoba K, Fadel K, Brown S, eds. AMCIS 2020 Proc. Adv. Inform. Systems Res. (Association for Information Systems, Atlanta), 1–10.Google Scholar
• Benevenuto F, Rodrigues T, Cha M, Almeida V (2012) Characterizing user navigation and interactions in online social networks. Inform. Sci. 195:1–24.Crossref, Google Scholar
• Cao Y, Xu J, Yang C, Wang J, Zhang Y, Wang C, Chen L, Yang Y (2023) When to pre-train graph neural networks? From data generation perspective! Singh A, Sun Y, Akoglu L, Gunopulos D, Yan X, Kumar R, Ozcan F, Ye J, eds. Proc. 29th ACM SIGKDD Conf. Knowledge Discovery Data Mining (Association for Computing Machinery, New York), 142–153.Google Scholar
• Chen Y, Zhang Y, Bian Y, Yang H, Kaili M, Xie B, Liu T, Han B, Cheng J (2022) Learning causally invariant representations for out-of-distribution generalization on graphs. Koyejo S, Mohamed S, Agarwal A, Belgrave D, Cho K, Oh A, eds. Advances in Neural Information Processing Systems, vol. 35 (Curran Associates, Red Hook, NY), 22131–22148.Google Scholar
• Feng F, He X, Tang J, Chua TS (2019) Graph adversarial training: Dynamically regularizing based on graph structure. IEEE Trans. Knowledge Data Engrg. 33(6):2493–2504.Crossref, Google Scholar
• Grover A, Leskovec J (2016) node2vec: Scalable feature learning for networks. Krishnapuram B, Shah M, Smola A, Aggarwal C, Shen D, Rastogi R, eds. Proc. 22nd ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining (Association for Computing Machinery, New York), 855–864.Google Scholar
• Hamilton WL, Ying R, Leskovec J (2017) Inductive representation learning on large graphs. Guyon I, Von Luxburg U, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R, eds. Advances in Neural Information Processing Systems, vol. 30 (Curran Associates, Red Hook, NY), 1025–1035.Google Scholar
• Han X, Yang Y, Wang L, Wu J (2024) Privacy-preserving network embedding against private link inference attacks. IEEE Trans. Dependable Secure Comput. 21(2):847–859.Crossref, Google Scholar
• Han X, Zhang Z, Ding N, Gu Y, Liu X, Huo Y, Qiu J, et al. (2021) Pre-trained models: Past, present and future. AI Open 2:225–250.Crossref, Google Scholar
• Hu H, Cheng L, Vap JP, Borowczak M (2022) Learning privacy-preserving graph convolutional network with partially observed sensitive attributes. Laforest F, Troncy R, Simperl E, Agarwal D, Gionis A, Herman I, Médini L, eds. Proc. ACM Web Conf. 2022 (Association for Computing Machinery, New York), 3552–3561.Google Scholar
• Hu Z, Dong Y, Wang K, Chang KW, Sun Y (2020a) GPT-GNN: Generative pre-training of graph neural networks. Gupta R, Liu Y, Shah M, Rajan S, Tang J, Prakash BA, eds. Proc. 26th ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining (Association for Computing Machinery, New York), 1857–1867.Google Scholar
• Hu W, Liu B, Gomes J, Zitnik M, Liang P, Pande VS, Leskovec J (2020b) Strategies for pre-training graph neural networks. 8th Internat. Conf. Learn. Representations, ICLR 2020 (Addis Ababa, Ethiopia).Google Scholar
• Huang R, Xu J, Jiang X, Pan C, Yang Z, Wang C, Yang Y (2024) Measuring task similarity and its implication in fine-tuning graph neural networks. Wooldridge M, Dy J, Natarajan S, eds. Proc. 38th AAAI Conf. Artificial Intelligence 36th Conf. Innovative Applications Artificial Intelligence 14th Sympos. Ed. Adv. Artificial Intelligence (AAAI Press, Washington, DC), 12617–12625.Google Scholar
• Jeh G, Widom J (2002) SimRank: A measure of structural-context similarity. Zaïane OR, Goebel R, Hand D, Keim D, Ng R, eds. Proc. Eighth ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining (Association for Computing Machinery, New York), 538–543.Google Scholar
• Kipf TN, Welling M (2016) Variational graph auto-encoders. Preprint, submitted November 21, https://arxiv.org/abs/1611.07308.Google Scholar
• Kruskal JB (1956) On the shortest spanning subtree of a graph and the traveling salesman problem. Proc. Amer. Math. Soc. 7(1):48–50.Crossref, Google Scholar
• Li T, Li N (2009) On the tradeoff between privacy and utility in data publishing. Elder J, Fogelman FS, Flach P, Zaki M, eds. Proc. 15th ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining (Association for Computing Machinery, New York), 517–526.Google Scholar
• Li H, Zhang Z, Wang X, Zhu W (2022) Learning invariant graph representations for out-of-distribution generalization. Koyejo S, Mohamed S, Agarwal A, Belgrave D, Cho K, Oh A, eds. Advances in Neural Information Processing Systems, vol. 35 (Curran Associates, Red Hook, NY), 11828–11841.Crossref, Google Scholar
• Liao P, Zhao H, Xu K, Jaakkola T, Gordon GJ, Jegelka S, Salakhutdinov R (2021) Information obfuscation of graph neural networks. Meila M, Zhang T, eds. Proc. 38th Internat. Conf. Machine Learn. (JMLR.org), 6600–6610.Google Scholar
• Lü L, Zhou T (2011) Link prediction in complex networks: A survey. Physica A Statist. Mechanics Appl. 390(6):1150–1170.Crossref, Google Scholar
• Newman M (2010) Networks: An Introduction (Oxford University Press, New York).Crossref, Google Scholar
• Orekondy T, Schiele B, Fritz M (2019) Knockoff nets: Stealing functionality of black-box models. Davis L, Torr P, Zhu SC, eds. Proc. IEEE/CVF Conf. Comput. Vision Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, NJ), 4954–4963.Google Scholar
• Qiu J, Chen Q, Dong Y, Zhang J, Yang H, Ding M, Wang K, Tang J (2020) GCC: Graph contrastive coding for graph neural network pre-training. Gupta R, Liu Y, Shah M, Rajan S, Tang J, Prakash BA, eds. Proc. 26th ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining (Association for Computing Machinery, New York), 1150–1160.Google Scholar
• Rigaki M, Garcia S (2023) A survey of privacy attacks in machine learning. ACM Comput. Surv. 56(4):1–34.Crossref, Google Scholar
• Rong Y, Bian Y, Xu T, Xie W, Wei Y, Huang W, Huang J (2020) Self-supervised graph transformer on large-scale molecular data. Larochelle H, Ranzato M, Hadsell R, Balcan MF, Lin H, eds. Advances in Neural Information Processing Systems, vol. 33 (Curran Associates, Red Hook, NY), 12559–12571.Google Scholar
• Rozemberczki B, Sarkar R (2020) Characteristic functions on graphs: Birds of a feather, from statistical descriptors to parametric models. d’Aquin M, Dietze S, Hauff C, Curry E, Mauroux PC, eds. Proc. 29th ACM Internat. Conf. Inform. Knowledge Management (Association for Computing Machinery, New York), 1325–1334.Google Scholar
• Rozemberczki B, Allen C, Sarkar R (2021) Multi-scale attributed node embedding. J. Complex Networks 9(2):cnab014.Crossref, Google Scholar
• Rozemberczki B, Davies R, Sarkar R, Sutton C (2019) GEMSEC: Graph embedding with self clustering. Spezzano F, Chen W, Xiao X, eds. Proc. 2019 IEEE/ACM Internat. Conf. Adv. Soc. Networks Anal. Mining 2019 (Association for Computing Machinery, New York), 65–72.Google Scholar
• Sajadmanesh S, Shamsabadi AS, Bellet A, Gatica-Perez D (2023) Gap: Differentially private graph neural networks with aggregation perturbation. Calandrino J, Troncoso C, eds. Proc. 32nd USENIX Conf. Security Sympos (USENIX Association, Berkeley, CA), 3223–3240.Google Scholar
• Smith A, Anderson M (2018) Social media use in 2018. Accessed September 16, 2023, http://www.pewinternet.org/2018/03/01/social-media-use-in-2018.Google Scholar
• Tong H, Faloutsos C, Pan JY (2006) Fast random walk with restart and its applications. Clifton CW, Zhong N, Liu J, Wah BW, Wu X, eds. Proc. Sixth Internat. Conf. Data Mining (Institute of Electrical and Electronics Engineers, Piscataway, NJ), 613–622.Google Scholar
• van den Oord A, Li Y, Vinyals O (2018) Representation learning with contrastive predictive coding. Preprint, submitted July 10, https://arxiv.org/abs/1807.03748.Google Scholar
• Wang B, Guo J, Li A, Chen Y, Li H (2021) Privacy-preserving representation learning on graphs: A mutual information perspective. Zhu F, Ooi BC, Miao C, Wang H, Skrypnyk I, Hsu W, Chawla S, eds. Proc. 27th ACM SIGKDD Conf. Knowledge Discovery Data Mining (Association for Computing Machinery, New York), 1667–1676.Google Scholar
• Wu F, Long Y, Zhang C, Li B (2022) LINKTELLER: Recovering private edges from graph neural networks via influence analysis. Holz T, Ristenpart T, eds. IEEE Sympos. Security Privacy (Institute of Electrical and Electronics Engineers, Piscataway, NJ), 2005–2024.Google Scholar
• Xie H, Ma J, Xiong L, Yang C (2021) Federated graph classification over non-IID graphs. Ranzato M, Beygelzimer A, Dauphin Y, Liang PS, Wortman Vaughan J, eds. Advances in Neural Information Processing Systems, vol. 34 (Curran Associates, Red Hook, NY), 18839–18852.Google Scholar
• Xu K, Hu W, Leskovec J, Jegelka S (2019) How powerful are graph neural networks? 7th Internat. Conf. Learn. Representations ICLR 2019 (New Orleans, LA).Google Scholar
• Xu J, Wang J, Zhou Z, Lu T (2025) Toward graph data collaboration in a data-sharing-free manner: A novel privacy-preserving graph pretraining model. https://doi.org/10.1287/ijoc.2023.0115.cd, https://github.com/INFORMSJoC/2023.0115.Google Scholar
• Xu J, Huang R, Jiang X, Cao Y, Yang C, Wang C, Yang Y (2024) Better with less: A data-active perspective on pre-training graph neural networks. Oh A, Naumann T, Globerson A, Saenko K, Hardt M, Levine S, eds. Proc. 37th Internat. Conf. Neural Inform. Processing Systems (Curran Associates, Red Hook, NY), 56946–56978.Google Scholar
• You Y, Chen T, Sui Y, Chen T, Wang Z, Shen Y (2020) Graph contrastive learning with augmentations. Larochelle H, Ranzato M, Hadsell R, Balcan MF, Lin H, eds. Advances in Neural Information Processing Systems, vol. 33 (Curran Associates, Red Hook, NY), 5812–5823.Google Scholar
• Zhang J, Philip SY (2015) Integrated anchor and social link predictions across social networks. Yang Q, Wooldridge M, eds. Proc. 24th Internat. Joint Conf. Artificial Intelligence (AAAI Press, Washington, DC), 2125–2131.Google Scholar
• Zhang S, Tong H, Tang J, Xu J, Fan W (2020) Incomplete network alignment: Problem definitions and fast solutions. ACM Trans. Knowledge Discovery Data 14(4):1–26.Crossref, Google Scholar
• Zhou T, Lü L, Zhang YC (2009) Predicting missing links via local information. Eur. Phys. J. B. 71(4):623–630.Crossref, Google Scholar
• Zhou F, Zhang K, Xie S, Luo X (2020) Learning to correlate accounts across online social networks: An embedding-based approach. INFORMS J. Comput. 32(3):714–729.Link, Google Scholar
• Zhou Z, Zhou C, Li X, Yao J, Yao Q, Han B (2023) On strengthening and defending graph reconstruction attack with Markov chain approximation. Krause A, Brunskill E, Cho K, Engelhardt B, Sabato S, Scarlett J, eds. Proc. 40th Internat. Conf. Machine Learn. (JMLR.org), 42843–42877.Google Scholar