Denoising Diffusion Probabilistic Models Are Optimally Adaptive to Unknown Low Dimensionality

References

• [1] Anderson BD (1982) Reverse-time diffusion equation models. Stochastic Processes Appl. 12(3):313–326.Crossref, Google Scholar
• [2] Azangulov I, Deligiannidis G, Rousseau J (2024) Convergence of diffusion models under the manifold hypothesis in high-dimensions. Preprint, submitted September 27, https://arxiv.org/abs/2409.18804v1.Google Scholar
• [3] Benton J, Bortoli VD, Doucet A, Deligiannidis G (2024) Nearly d-linear convergence bounds for diffusion models via stochastic localization. 12th Internat. Conf. Learn. Representations, https://openreview.net/forum?id=r5njV3BsuD.Google Scholar
• [4] Block A, Jia Z, Polyanskiy Y, Rakhlin A (2022) Intrinsic dimension estimation using Wasserstein distance. J. Machine Learn. Res. 23(313):1–37.Google Scholar
• [5] Chen S, Daras G, Dimakis AG (2023c) Restoration-degradation beyond linear diffusions: A non-asymptotic analysis for DDIM-type samplers. Preprint, submitted March 6, https://arxiv.org/abs/2303.03384.Google Scholar
• [6] Chen H, Lee H, Lu J (2022a) Improved analysis of score-based generative modeling: User-friendly bounds under minimal smoothness assumptions. Preprint, submitted November 3, https://arxiv.org/abs/2211.01916v1.Google Scholar
• [7] Chen H, Lee H, Lu J (2023a) Improved analysis of score-based generative modeling: User-friendly bounds under minimal smoothness assumptions. Proc. 40th Internat. Conf. Machine Learn. (PMLR, New York), 4735–4763.Google Scholar
• [8] Chen M, Huang K, Zhao T, Wang M (2023b) Score approximation, estimation and distribution recovery of diffusion models on low-dimensional data. Proc. 40th Internat. Conf. Machine Learn. (PMLR, New York), 4672–4712.Google Scholar
• [9] Chen M, Mei S, Fan J, Wang M (2024a) Opportunities and challenges of diffusion models for generative AI. Natl. Sci. Rev. 11(12):nwae348.Crossref, Google Scholar
• [10] Chen S, Chewi S, Lee H, Li Y, Lu J, Salim A (2023c) The probability flow ODE is provably fast. Preprint, submitted May 19, https://arxiv.org/abs/2305.11798.Google Scholar
• [11] Chen S, Chewi S, Li J, Li Y, Salim A, Zhang AR (2022b) Sampling is as easy as learning the score: Theory for diffusion models with minimal data assumptions. Preprint, submitted September 22, https://arxiv.org/abs/2209.11215v1.Google Scholar
• [12] Chen S, Zhang H, Guo M, Lu Y, Wang P, Qu Q (2024b) Exploring low-dimensional subspaces in diffusion models for controllable image editing. Preprint, submitted September 10, https://arxiv.org/abs/2409.02374.Google Scholar
• [13] Dasgupta S, Freund Y (2008) Random projection trees and low dimensional manifolds. Proc. 40th Annual ACM Sympos. Theory Comput. (Association for Computing Machinery, New York), 537–546.Google Scholar
• [14] Dhariwal P, Nichol A (2021) Diffusion models beat GANs on image synthesis. Adv. Neural Inform. Processing Systems 34:8780–8794.Google Scholar
• [15] Dou Z, Kotekal S, Xu Z, Zhou HH (2024) From optimal score matching to optimal sampling. Preprint, submitted September 11, https://arxiv.org/abs/2409.07032.Google Scholar
• [16] Efron B (2011) Tweedie’s formula and selection bias. J. Amer. Statist. Assoc. 106(496):1602–1614.Crossref, Google Scholar
• [17] El Alaoui A, Montanari A (2022) An information-theoretic view of stochastic localization. IEEE Trans. Inform. Theory 68(11):7423–7426.Crossref, Google Scholar
• [18] Eldan R (2020) Taming correlations through entropy-efficient measure decompositions with applications to mean-field approximation. Probab. Theory Related Fields 176(3):737–755.Crossref, Google Scholar
• [19] Falconer K, Fraser J, Käenmäki A (2023) Minkowski dimension for measures. Proc. Amer. Math. Soc. 151:779–794.Crossref, Google Scholar
• [20] Gao X, Zhu L (2024) Convergence analysis for general probability flow ODEs of diffusion models in Wasserstein distances. Preprint, submitted January 31, https://arxiv.org/abs/2401.17958v1.Google Scholar
• [21] Haussmann UG, Pardoux E (1986) Time reversal of diffusions. Ann. Probab. 14(4):1188–1205.Crossref, Google Scholar
• [22] Ho J, Jain A, Abbeel P (2020) Denoising diffusion probabilistic models. Adv. Neural Inform. Processing Systems 33:6840–6851.Google Scholar
• [23] Ho J, Salimans T, Gritsenko A, Chan W, Norouzi M, Fleet DJ (2022) Video diffusion models. Adv. Neural Inform. Processing Systems 35:8633–8646.Google Scholar
• [24] Huang DZ, Huang J, Lin Z (2024) Convergence analysis of probability flow ODE for score-based generative models. Preprint, submitted April 15, https://arxiv.org/abs/2404.09730v1.Google Scholar
• [25] Kadkhodaie Z, Guth F, Simoncelli EP, Mallat S (2023) Generalization in diffusion models arises from geometry-adaptive harmonic representation. Preprint, submitted October 4, https://arxiv.org/abs/2310.02557v1.Google Scholar
• [26] Kpotufe S, Dasgupta S (2012) A tree-based regressor that adapts to intrinsic dimension. J. Comput. System Sci. 78(5):1496–1515.Crossref, Google Scholar
• [27] Lee JM (2018) Introduction to Riemannian Manifolds, Graduate Texts in Mathematics, vol. 2 (Springer, Cham, Switzerland).Crossref, Google Scholar
• [28] Lee H, Lu J, Tan Y (2022) Convergence for score-based generative modeling with polynomial complexity. Adv. Neural Inform. Processing Systems 35:22870–22882.Crossref, Google Scholar
• [29] Lee H, Lu J, Tan Y (2023) Convergence of score-based generative modeling for general data distributions. 34th Internat. Conf. Algorithmic Learn. Theory (PMLR, New York), 946–985.Google Scholar
• [30] Le Gall J-F (2016) Brownian Motion, Martingales, and Stochastic Calculus (Springer, Cham, Switzerland).Crossref, Google Scholar
• [31] Levina E, Bickel P (2004) Maximum likelihood estimation of intrinsic dimension. Adv. Neural Inform. Processing Systems 17:777–784.Google Scholar
• [32] Li G, Jiao Y (2024) Improved convergence rate for diffusion probabilistic models. Preprint, submitted October 17, https://arxiv.org/abs/2410.13738v1.Google Scholar
• [33] Li G, Yan Y (2024a) Adapting to unknown low-dimensional structures in score-based diffusion models. Preprint, submitted December 31, https://arxiv.org/abs/2405.14861.Google Scholar
• [34] Li G, Yan Y (2024b) O(d/T) convergence theory for diffusion probabilistic models under minimal assumptions. Preprint, submitted September 27, https://arxiv.org/abs/2409.18959v1.Google Scholar
• [35] Li G, Huang Z, Wei Y (2024a) Towards a mathematical theory for consistency training in diffusion models. Preprint, submitted February 12, https://arxiv.org/abs/2402.07802.Google Scholar
• [36] Li G, Wei Y, Chen Y, Chi Y (2023) Towards faster non-asymptotic convergence for diffusion-based generative models. Preprint, submitted June 15, https://arxiv.org/abs/2306.09251v1.Google Scholar
• [37] Li G, Wei Y, Chi Y, Chen Y (2024b) A sharp convergence theory for the probability flow ODEs of diffusion models. Preprint, submitted August 5, https://arxiv.org/abs/2408.02320.Google Scholar
• [38] Li G, Zhou Y, Wei Y, Chen Y (2025) Faster diffusion models via higher-order approximation. Preprint, submitted August 13, https://arxiv.org/abs/2506.24042.Google Scholar
• [39] Li G, Huang Y, Efimov T, Wei Y, Chi Y, Chen Y (2024c) Accelerating convergence of score-based diffusion models, provably. Preprint, submitted March 6, https://arxiv.org/abs/2403.03852.Google Scholar
• [40] Liang J, Huang Z, Chen Y (2025) Low-dimensional adaptation of diffusion models: Convergence in total variation. Proc. 38th Conf. Learn. Theory (PMLR, New York), 3723–3729.Google Scholar
• [41] Liang Y, Ju P, Liang Y, Shroff N (2024) Broadening target distributions for accelerated diffusion models via a novel analysis approach. Preprint, submitted October 1, https://arxiv.org/abs/2402.13901v3.Google Scholar
• [42] Mbacke SD, Rivasplata O (2023) A note on the convergence of denoising diffusion probabilistic models. Preprint, submitted December 10, https://arxiv.org/abs/2312.05989v1.Google Scholar
• [43] Moon KR, Stanley JS III, Burkhardt D, van Dijk D, Wolf G, Krishnaswamy S (2018) Manifold learning-based methods for analyzing single-cell RNA-sequencing data. Curr. Opin. Systems Biol. 7:36–46.Crossref, Google Scholar
• [44] Øksendal B (2003) Stochastic Differential Equations: An Introduction with Applications (Springer Berlin, Heidelberg, Germany).Crossref, Google Scholar
• [45] Pope P, Zhu C, Abdelkader A, Goldblum M, Goldstein T (2021) The intrinsic dimension of images and its impact on learning. Preprint, submitted April 18, https://arxiv.org/abs/2104.08894.Google Scholar
• [46] Potaptchik P, Azangulov I, Deligiannidis G (2024) Linear convergence of diffusion models under the manifold hypothesis. Preprint, submitted October 11, https://arxiv.org/abs/2410.09046v1.Google Scholar
• [47] Ramesh A, Dhariwal P, Nichol A, Chu C, Chen M (2022) Hierarchical text-conditional image generation with CLIP latents. Preprint, submitted April 13, https://arxiv.org/abs/2204.06125.Google Scholar
• [48] Rombach R, Blattmann A, Lorenz D, Esser P, Ommer B (2022) High-resolution image synthesis with latent diffusion models. IEEE/CVF Conf. Comput. Vision Pattern Recognition (IEEE, Piscataway, NJ), 10684–10695.Google Scholar
• [49] Simoncelli EP, Olshausen BA (2001) Natural image statistics and neural representation. Annual Rev. Neurosci. 24:1193–1216.Crossref, Google Scholar
• [50] Sohl-Dickstein J, Weiss E, Maheswaranathan N, Ganguli S (2015) Deep unsupervised learning using nonequilibrium thermodynamics. Proc. 32nd Internat. Conf. Machine Learn. (PMLR, New York), 2256–2265.Google Scholar
• [51] Song Y, Sohl-Dickstein J, Kingma DP, Kumar A, Ermon S, Poole B (2021) Score-based generative modeling through stochastic differential equations. Internat. Conf. Learn. Representations (OpenReview).Google Scholar
• [52] Tang W (2023) Diffusion probabilistic models. Preprint, submitted August 26, https://www.columbia.edu/~wt2319/DPM.pdf.Google Scholar
• [53] Tang R, Yang Y (2024) Adaptivity of diffusion models to manifold structures. Proc. 27th Internat. Conf. Artificial Intelligence Statist. (PMLR, New York), 238:1648–1656.Google Scholar
• [54] Tang W, Zhao H (2024) Score-based diffusion models via stochastic differential equations—A technical tutorial. Preprint, submitted February 12, https://arxiv.org/abs/2402.07487v1.Google Scholar
• [55] Vershynin R (2009) On the role of sparsity in compressed sensing and random matrix theory. 2009 3rd IEEE Internat. Workshop Comput. Adv. Multi-Sensor Adaptive Processing CAMSAP (IEEE, Piscataway, NJ), 189–192.Google Scholar
• [56] Vershynin R (2018) High-Dimensional Probability: An Introduction with Applications in Data Science, Cambridge Series in Statistical and Probabilistic Mathematics, vol. 47 (Cambridge University Press, Cambridge, UK).Crossref, Google Scholar
• [57] Wainwright MJ (2019) High-Dimensional Statistics: A Non-Asymptotic Viewpoint, Cambridge Series in Statistical and Probabilistic Mathematics, vol. 48 (Cambridge University Press, Cambridge, UK).Crossref, Google Scholar
• [58] Wang Y, He Y, Tao M (2024b) Evaluating the design space of diffusion-based generative models. Preprint, submitted October 27, https://arxiv.org/abs/2406.12839.Google Scholar
• [59] Wang P, Zhang H, Zhang Z, Chen S, Ma Y, Qu Q (2024a) Diffusion models learn low-dimensional distributions via subspace clustering. Preprint, submitted September 4, https://arxiv.org/abs/2409.02426v1.Google Scholar
• [60] Watson JL, Juergens D, Bennett NR, Trippe BL, Yim J, Eisenach HE, Ahern W, et al. (2023) De novo design of protein structure and function with RFdiffusion. Nature 620(7976):1089–1100.Crossref, Google Scholar
• [61] Wu Y, Chen Y, Wei Y (2024) Stochastic Runge-Kutta methods: Provable acceleration of diffusion models. Preprint, submitted October 7, https://arxiv.org/abs/2410.04760.Google Scholar
• [62] Xia Q, Vershynina A (2010) On the transport dimension of measures. SIAM J. Math. Anal. 41(6):2407–2430.Crossref, Google Scholar
• [63] Zhu G, Deng W, Hu H, Ma R, Zhang S, Yang J, Peng J, Kaplan T, Zeng J (2018) Reconstructing spatial organizations of chromosomes through manifold learning. Nucleic Acids Res. 46(8):e50.Crossref, Google Scholar