Stochastic Inertial Dynamics via Time Scaling and Averaging

1. Introduction

1.1. Problem Statement

We consider the minimization problem

where $\mathbb{H}$ is a separable real Hilbert space, and the objective F satisfies the following standing assumptions:

1.1.1. First-Order in Time Systems.

To solve (P) when $g\equiv 0$, a fundamental dynamic is the gradient flow system:

The gradient system (GF) is a dissipative dynamical system, whose study dates back to Cauchy Cauchy (1847) in finite dimension. It plays a fundamental role in optimization: It transforms the problem of minimizing f into the study of the asymptotic behavior of the trajectories of (GF). This example was the precursor to the rich connection between continuous dissipative dynamical systems and optimization. It is well known since the founding papers of Brezis, Baillon, and Bruck in the 1970s that, if the solution set $\text{arg}\hspace{0.17em}\text{min}(f)$ of (P) is nonempty, then each solution trajectory of (GF) converges weakly, and its (weak) limit belongs to $\text{arg}\hspace{0.17em}\text{min}(f)$. Moreover, this dynamic is known to yield a convergence rate of $\mathcal{O}(t^{-1})$ (in fact even $o(t^{-1})$) on the values.

The Euler forward (a.k.a. Euler-Maruyama) discretization (GF), with stepsize sequence $h_k>0$, is the celebrated gradient descent scheme

Under (H₀), and for ${(h_k)}_{k\in \mathbb{N}}\subset ]0,2/L[$, then we have both the convergence of the values $f(x_k)-\min f=\mathcal{O}(1/k)$ (in fact even $o(1/k)$), and the weak convergence of iterates ${(x_k)}_{k\in \mathbb{N}}$ to a point in $\text{arg}\hspace{0.17em}\text{min}(f)$. This convergence rate can be refined under various additional geometrical properties on the objective f such as error bounds (and the closely related Kurdyka-Łojasiewicz property in the convex case; Bolte et al. 2016).

1.1.2. Second-Order In-Time Systems: Key Role of Inertia.

Second-order in-time inertial dynamical systems have been introduced to provably accelerate the convergence behavior compared with (GF). An abundant literature has been devoted to the study of inertial dynamics:

The importance of working with an asymptotically vanishing viscosity coefficient $\gamma (t)$ to obtain acceleration was stressed by several authors (Cabot et al. 2009, Attouch and Cabot 2017). Most of the literature focuses on the case $\gamma \left(t\right)=\alpha /t$, originating from the seminal work of Su et al. (2016) who showed the rate of convergence $\mathcal{O}(1/t^2)$ of the values for $\alpha =3$, thus making the link with the Nesterov accelerated gradient method (Nesterov 1983). Since then, an important body of literature has been devoted to this important case and the subtle tuning of parameter $\alpha$. Taking $\alpha \ge 3$ is mandatory for provable accelerated rate $\mathcal{O}(1/t^2)$ of the values (Attouch et al. 2018b), and $\alpha >3$ provides an even better rate $o(1/t^2)$ and weak convergence of the trajectory (Attouch and Peypouquet 2016, May 2017). On the other hand, $\alpha <3$ necessarily leads to a slower rate $\mathcal{O}(1/t^{2\alpha /3})$ (Apidopoulos et al. 2018, Attouch et al. 2019). Another remarkable instance of (IGS_γ) corresponds to the well-known heavy ball with friction (HBF) method, where $\gamma (t)$ is a constant, first introduced by Polyak (1964). When the objective is strongly convex, it was shown that the trajectory converges exponentially with an optimal convergence rate for a properly chosen constant $\gamma$ (Attouch et al. 2011).

1.1.3. Geometric Hessian-Driven Damping.

Because of the inertial aspects, and the asymptotic vanishing viscous damping coefficient, (IGS_γ) may exhibit many small oscillations that are not desirable from an optimization point of view. To remedy this, a powerful tool consists in introducing a geometric damping driven by the Hessian of f into the dynamic. This yields the inertial system with explicit Hessian-driven damping (Attouch et al. 2016, 2022b):

and the inertial system with implicit Hessian-driven damping (Alecsa et al. 2021, Muehlebach and Jordan 2021):

The rationale behind the use of the term “implicit” in (ISIHD) comes from a Taylor expansion of the gradient term (as $t\to +\infty$ we expect $\dot{x}(t)\to 0$). Following the physical interpretation of these Ordinary Differential Equations, we call the nonnegative parameters $\gamma$ and $\beta$ as the viscous and geometric damping parameters, respectively.

At first glance, the presence of the Hessian in (ISEHD) may seem to entail numerical difficulties. However, this is not the case because the term ${\nabla }^{2}f(x(t))\dot{x}(t)$ is nothing but the time derivative of $t\mapsto \nabla f(x(t))$. This explains why the time discretization of this dynamic provides efficient first-order algorithms (Attouch et al. 2022b). On the other hand, (ISEHD) can be argued to be truly of second-order nature, that is, close to Newton’s and Levenberg-Marquardt’s dynamics (Castera et al. 2024). This understanding suggests that (ISIHD) may reflect the nature of first-order algorithms more faithfully than (ISEHD). Moreover, when it will come to our stochastic setting, which is the focus of this paper, the approach we propose will only make sense for the implicit form of the Hessian-driven damping. Indeed, in our stochastic setting, we do not have direct access to the evaluation of the gradient of f. Instead, we model the associated errors with a continuous Itô martingale (denoted as $M(t)$). But for the explicit form of the Hessian-driven damping, this would entail a term involving the time derivative of $\nabla f(X(t))+M(t)$. This is meaningless because (nonconstant) martingales are not differentiable almost sure (a.s.) This is why from now on, we will solely consider (ISIHD).

1.2. Motivations

In the following, $\mathbb{H},\mathbb{K}$ are real separable Hilbert spaces. In many practical situations, the (sub)gradient evaluation is subject to stochastic errors. This is, for example, the case if the cost per iteration is very high, and thus cheap and random approximations of the (sub)gradient are necessary. These errors can also be due to some other exogenous factor. The continuous-time approach through stochastic differential equations (SDEs) is a powerful way to model these errors in a unified way, and stochastic algorithms can then be viewed as time discretizations. In fact, several recent works have used the SDE perspective to model Stochastic Gradient Descent (SGD)-type algorithms motivated by various reasons; (Li et al. 2017, 2021; Mertikopoulos and Staudigl 2018; Soatto and Chaudhari 2018; Hu et al. 2019b; Orvieto and Lucchi 2019; Shi et al. 2023; Dambrine et al. 2024; Maulen-Soto et al. 2024, 2025). The continuous-time perspective offers a deep insight and unveils the key properties of the dynamic without being tied to a specific discretization.

In this context, the first SDE that comes to mind is

defined over a complete filtered probability space $(\mathrm{\Omega },\mathcal{F},{({\mathcal{F}}_t)}_{t\ge t_0},\mathbb{P})$, where the diffusion (volatility) term $\sigma :{\mathbb{R}}^{+}\times \mathbb{H}\to {\mathcal{L}}_2(\mathbb{K};\mathbb{H})$ is a measurable function, W is a ${\mathcal{F}}_t$-adapted $\mathbb{K}$-valued cylindrical Brownian motion, and the initial data $X_0$ is an ${\mathcal{F}}_{t_0}-$measurable $\mathbb{H}$-valued random variable. All these notations and concepts are introduced in Section 2. (SGF) is a stochastic counterpart of (GF) where the error is modeled as a stochastic integral with respect to the measure defined by a continuous Itô martingale.

1.2.1. SDE Modeling of SGD.

To simplify the discussion, let us focus in this section on the finite-dimensional case ($\mathbb{H}={\mathbb{R}}^d$, $\mathbb{K}={\mathbb{R}}^m$). In various areas of science and engineering, in particular, in machine learning, the SGD is a powerful alternative to gradient descent and consists in replacing the full gradient computation by a cheap random version, serving as an unbiased estimator. The SGD updates the iterates according to

where $h\in {\mathbb{R}}_{+}$ is the stepsize and $e_k$ is the random noise term on the gradient at the kth iteration. As such, (SGD) can be viewed as instance of the Robbins-Monro stochastic approximation algorithm (Robins and Monro 1951). When the objective takes the form $f(x)=\mathbb{E}[\widehat{f}(x,\xi )]$, where the expectation is with respect to the random variable $\xi$, the single-batch version of SGD readswhere ${({\xi }_k)}_{k\in \mathbb{N}}$ are independent and identically distributed random variables with the same distribution as $\xi$. Of course, (SGD_SB) is an instance of (SGD) with $e_k=\nabla \widehat{f}(x_k,{\xi }_k)-\nabla f(x_k)$.

The SDE continuous-time approach is motivated by its close relation to (SGD) or (SGD_SB). We first note that when the noise $e_k$ in (SGD) is $\mathcal{N}(0,{\sigma }_k{I}_d)$, (SGF) is a better continuous-time model for (SGD) than (GF), as has been shown recently in Dambrine et al. (2024, proposition 2.1). There, the sequence ${(x_k)}_{k\in \mathbb{N}}$ provided by (SGD), with $e_k\sim \mathcal{N}(0,{\sigma }_k{I}_d)$, was proved to be accurately approximated by (SGF) with $\sigma (t,X(t))=\sqrt{h}\sigma (t)$ and $\sigma (kh)={\sigma }_k{I}_d$.

For the standard single-batch SGD (SGD_SB), the argument is more involved. Actually, many recent works (Mandt et al. 2016; Li et al. 2017, 2021, 2024; Soatto and Chaudhari 2018; Hu et al. 2019b; Orvieto and Lucchi 2019; Latz 2021; Xie et al. 2021; Shi et al. 2023) have linked algorithm (SGD_SB) with continuous-time first-order stochastic diffusion dynamics such as (SGF). These works show either empirically or theoretically under which conditions (appropriate drift and diffusion terms, regularity of f, etc.) (SGF) can be seen as a good approximation model of (SGD_SB) for fixed stepsize. By a good model, we mean that (SGF) is a continuum limit of (SGD_SB) as the stepsize h goes to zero, or equivalently that the approximation of (SGD_SB) via the diffusion process (SGF) is precise in some weak sense (Li et al. 2017, 2021; Hu et al. 2019b).

As a consequence, using (SGF) as a proxy of (SGD) or (SGD_SB) allows to capitalize on the wealth of results in the field of SDEs, Itô calculus, and measure theory, and this in turn opens the door to new insights in the behavior of (SGD) or (SGD_SB) and to transfer all the convergence results that one can prove for (SGF) to (SGD). Actually, this is one of the main messages we want to convey in this work. Our motivation and results are also complementary to those in the literature. Indeed, most, if not all, of works cited in the previous paragraph are primarily motivated by the fact that continuous-time SDE approximation of (SGD) is a crucial tool to study its escape behavior of bad saddle points (a.k.a. traps) in the nonconvex smooth case. Our standpoint, which is line with Maulen-Soto et al. (2024), Mertikopoulos and Staudigl (2018), Dambrine et al. (2024), and Maulen-Soto et al. (2025), is complementary, and we argue that the continuous-time perspective offers a deep insight and unveils the key properties of the dynamic, without being tied to a specific discretization. This enlightens the behavior of the sequence generated by some specific algorithm. In turn, studying the continuous-time SDE will allow to predict the convergence behavior of stochastic algorithms seen as a discretization of the corresponding continuous-time dynamics.

1.2.2. Second-Order SDE Modeling of Inertial SGD.

Using a lifting argument to get an equivalent first-order reformulation, a natural generalization of (ISIHD) to the nonsmooth case yields the differential inclusion

where $\partial F$ is the convex subdifferential of F. In this setting, keeping in mind that we want to give a rigorous meaning to (ISIHD_NS), we can model the associated errors using a stochastic integral with respect to the measure defined by a continuous Itô martingale. This entails the following stochastic differential inclusion (SDI), which is the stochastic counterpart of (ISIHD_NS):

This SDI is defined on a filtered probability space $(\mathrm{\Omega },\mathcal{F},{\{{\mathcal{F}}_t\}}_{t\ge 0},\mathbb{P})$, where $\sigma :[t_0,+\infty [\times \mathbb{H}\to {\mathcal{L}}_2(\mathbb{K};\mathbb{H})$ is a measurable function; and W is a $\mathbb{K}$-valued cylindrical Brownian motion. When $g\equiv 0$, this reduces to the stochastic counterpart of (ISIHD), given by the following SDE:

In the smooth but nonconvex case, and motivated again by strict saddle points avoidance, Hu et al. (2019a, b) study the convergence behavior of a stochastic discrete heavy ball method from its approximating SDE. The latter is a randomly perturbed nonlinear oscillator in Hu et al. (2019c) and a coupled system of nonlinear oscillators in Hu et al. (2019a). These SDEs are very different from the ones considered here. By contrast, other works do not employ SDE techniques and instead analyze the stochastic iteration directly; see, for instance, Barakat et al. (2021), Gadat et al. (2018), and Le (2024), with the last reference also addressing the nonsmooth case. There are several advantages of adopting the SDE perspective in our setting, among which are a more accurate approximation of the discrete stochastic dynamics than the ODE method and the ability to quantify stochastic fluctuations and describe the long-run distribution of the iterates; the approximation aspect is detailed in Appendix A.

Extending what we have discussed above for (SGF) as a good model of (SGD), we will show in Proposition A.1 (see Appendix A for details) that (S-ISIHD) is a good model of the natural stochastic inertial algorithm (A.1) obtained by simple Euler forward (a.k.a. Euler-Maruyama) discretization of (S-ISIHD). The corresponding convergence rate as a function of the time stepsize h is $\mathcal{O}(h)$ and shows that this is much better that the approximation rate of (ISIHD), which is only $\mathcal{O}(\sqrt{h})$. As a consequence, this justifies and motivates the continuous-time dynamics (S-ISIHD) (and (S-ISIHD_NS)) as a good proxy of stochastic inertial algorithms and opens the door to new insights in the behavior of such algorithms, and that their convergence properties can be easily derived from those of (S-ISIHD) with minimal effort. For instance, when f is smooth with Lipschitz-continuous gradient, it is easy to see from the descent lemma that

where $X_k$ are the iterates of (A.1) and $X(t)$ the solution to (S-ISIHD). This means that any rate proved on $\mathbb{E}[f(X(t))-\min f]$ can be directly transferred to $\mathbb{E}[f(X_k)-\min f]$.

1.3. Objectives and Contributions

In this work, our goal is to provide a general mathematical framework for analyzing the convergence properties of (S-ISIHD) and (S-ISIHD_NS). In this context, considering inertial dynamics with a time-dependent vanishing viscosity coefficient $\gamma$ is a key ingredient to obtain fast convergent methods. We will develop a systematic and unified way that transfers the properties of stochastic first-order dynamics recently studied by Maulen-Soto et al. (2024, 2025) to second-order ones. Our program will then rely on two pillars: time scaling and averaging, following the methodology recently developed in Attouch et al. (2024) for the deterministic case.

More precisely, we study the stochastic dynamics (S-ISIHD_NS) and its long-time behavior in order to solve (P). We conduct a new analysis using specific and careful arguments that are much more involved than in the deterministic case. To get some intuition, keeping the discussion informal at this stage, let us first identify the assumptions needed to expect that the position state of (S-ISIHD) “approaches” $\text{arg}\hspace{0.17em}\text{min}(f)$ in the long run. In the case where $\mathbb{H}=\mathbb{K}$, $\gamma (·)\equiv \gamma >0,\beta \equiv 0,$ and $\sigma =\tilde{\sigma }{I}_{\mathbb{H}}$, where $\tilde{\sigma }$ is a positive real constant; under mild assumptions one can show that (S-ISIHD) has a unique invariant distribution ${\pi }_{\tilde{\sigma }}$ in $(x,v)$ with density proportional to

(Pavliotis 2014, proposition 6.1). Clearly, as $\tilde{\sigma }\to 0^{+}$, ${\pi }_{\tilde{\sigma }}$ gets concentrated around $\text{arg}\hspace{0.17em}\text{min}(f)\times \{0_{\mathbb{H}}\}$, with ${\lim }_{\tilde{\sigma }\to 0^{+}}{\pi }_{\tilde{\sigma }}(\text{arg}\hspace{0.17em}\text{min}(f)\times \{0_{\mathbb{H}}\})=1$; see also Section 1.4 for further discussion. Motivated by these observations and the fact that we aim to exactly solve (P), our paper will then mainly focus on the case where $\sigma (·,x)$ vanishes fast enough as $t\to +\infty$ uniformly in x.

Our main contributions are summarized as follows:

• We show almost sure weak convergence of the trajectory (see Theorem 5) and convergence rates (see Theorem 6) in expectation in the case of time-dependent coefficients $\gamma (t)$ and a proper choice of $\beta (t)$ (depending on $\gamma$). For this analysis, we transfer the results from the Lyapunov analysis of the first-order in-time stochastic (sub)gradient system studied in Maulen-Soto et al. (2024, 2025) from which our inertial system is built through time scaling and averaging.

• We obtain almost sure and ergodic convergence results which correspond precisely to the best-known results in the deterministic case. In particular, if we let $\alpha >3, \gamma \left(t\right)=\alpha /t, \beta \left(t\right)=t/(\alpha -1)$, then under appropriate assumptions on the diffusion (volatility) term $\sigma$, we obtain the rate of convergence $o(1/t^2)$ of the values as well as fast convergence of the gradient in almost sure sense (see Corollary 1). This corresponds to the best known results of second-order in-time Hessian-driven damping dynamics in the deterministic case.

• As far as the interplay between viscous damping, geometric damping, and noise level, our results reveal that viscous damping, which allows for acceleration, enters in the control of the noise and may impose a more stringent summability condition on the diffusion coefficient (see, e.g., Theorem 5, Theorem 6 and Corollary 1). The geometric damping, which is designed to reduce oscillations, does not enter directly in this control and can be chosen in a more flexible way.

• We then turn to providing a local analysis with a local linear convergence rate under the Polyak-Łojasiewicz inequality (see Theorem 8). This is much more challenging in the stochastic case, and even more for second-order systems, as localizing the process in this case is very delicate. Leveraging time scaling and averaging offers an elegant framework to achieve such a local convergence analysis while it is barely possible otherwise. To the best of our knowledge, this is the first result of this kind for these systems in the literature.

• We also show almost sure strong convergence of the trajectory to the minimal norm solution when adding a Tikhonov regularization to our systems (see Theorem 9). Moreover, we show convergence rates in expectation for the objective and the trajectory for a particular Tikhonov regularizer (see Theorem 11).

It is worth observing that, because our approach is based on an averaging technique, it will involve Jensen’s inequality at some point to get fast convergence rates. In this respect, the convexity assumption on the objective function appears unavoidable, at least in this proof strategy. It is also worth stressing the fact that the approach only makes sense for the implicit form of the Hessian-driven damping. Indeed, as explained above, the explicit form of the Hessian-driven damping has a term involving the time derivative of the (sub)gradient at the trajectory. As the noise, modeled here as an Itô martingale, in practice stems from the (sub)gradient evaluation, this time derivative is meaningless with explicit Hessian-driven damping, as (nonconstant) martingales are a.s. nondifferentiable.

1.4. Relation to Prior Work

1.4.1. Kinetic Diffusion Dynamics for Sampling.

Let us consider (S-ISIHD) in the case where $\mathbb{H}=\mathbb{K}={\mathbb{R}}^n$, $\gamma (t)=\gamma >0,\beta \equiv 0$, and $\sigma =\sqrt{2\gamma }I$. Then one recovers the kinetic Langevin diffusion (or second-order Langevin process). In this case, the continuous-time Markov process $(X(t),V(t))$ is positive recurrent and has a unique invariant distribution that has the density $\propto \exp (-f\left(x\right)-\Vert v{\Vert }^2/2)$ with respect to the Lebesgue measure on ${\mathbb{R}}^{2n}$. Time-discretized versions of this Langevin diffusion process have been studied in the literature to (approximately) sample from $\propto \exp (-f(x))$ with asymptotic and nonasymptotic convergence guarantees in various topologies and under various conditions; see Cheng et al. (2018), Ma et al. (2021), and Dalalyan et al. (2019) and references therein. By rescaling the problem, relation between sampling and optimization with quantitative estimates has been investigated in, for example, Dalalyan and Karagulyan (2019) for the strongly convex case.

1.4.2. First-Order Stochastic (Sub)Gradient Systems.

In Maulen-Soto et al. (2024) (respectively, Maulen-Soto et al. (2025)), the authors studied (SGF) (respectively, its nonsmooth version as an SDI) as a proxy for (SGD). One of their main results is the almost sure weak convergence of the trajectory to the set of minimizers under integrability conditions on $\sigma$, as well as convergence rates in expectation and in almost sure sense of the dynamic under different geometries of f. Our goal here is to take these results to second-order inertial dynamics featuring both viscous and geometric dampings. This turns out to be a challenging task. We will show that the convergence rate on the objective can be achieved provided that the noise vanishes sufficiently fast.

1.4.3. Inexact Inertial Gradient Systems.

There is an abundant literature regarding the dynamics of (ISIHD) and (ISEHD), either in the exact case or with errors but only deterministic ones (Schmidt et al. 2011; Haraux and Jendoubi 2012; Villa et al. 2013; Dossal and Aujol 2015; Attouch et al. 2018a, b, 2022a, b, 2023a, b; Alecsa et al. 2021; Castera et al. 2021; Shi et al. 2022). Only a few papers have been devoted to studying stochastic versions of (IGS_γ), either with vanishing damping $\gamma (t)=\alpha /t$ or constant damping $\gamma (t)$ (stochastic HBF) (Gadat and Panloup 2014, Gadat et al. 2018, Orvieto et al. 2020, Dambrine et al. 2024). One of the advantages of considering the stochastic version of (ISIHD) is the possible reduction of oscillations thanks to the introduction of the geometric damping. We show that this effect could be preserved in the stochastic setting, hence obtaining more stable trajectory solutions than other stochastic second-order dynamics. Additionally, having the flexibility to choose the viscous damping would eventually allow us to achieve faster convergence results. For a stochastic version of (IGS_γ), Dambrine et al. (2024) provide asymptotic $\mathcal{O}(1/t^2)$ convergence rate on the objective values in expectation under integrability conditions on the diffusion term, as well as other rates under additional geometrical properties of the objective. These geometrical assumptions come in the form of global growth and flatness of the objective that is very restrictive. Rather, here, our geometrical assumptions on f will be only local. The corresponding stochastic algorithms for these two choices of $\gamma$, whose mathematical formulation and analysis are simpler, have been the subject of active research work (Frostig et al. 2015, Allen-Zhu 2017, Jain et al. 2017, Lin et al. 2017, Gadat et al. 2018, Yan 2018, Assran and Rabbat 2020, Laborde and Oberman 2020, Lan 2020, Loizou and Richtárik 2020, Defazio and Jelassi 2022, Driggs et al. 2022, Attouch et al. 2024, Hamadouche et al. 2024).

1.4.4. Time Scaling and Averaging.

Attouch et al. (2024) proposed time scaling and averaging to link (GF) and (ISIHD) with a general viscous damping function $\gamma$ and a properly adjusted geometric damping function $\beta$ (related to $\gamma$). Our aim is to extend the results of Attouch et al. (2024) to the stochastic case. Leveraging these techniques with a general function $\gamma$ and an appropriate $\beta$, we will be able to transfer all the results we obtained in Maulen-Soto et al. (2025) for a first-order SDI to the second-order one (S-ISIHD_NS). This avoids in particular to go through an intricate and a dedicated Lyapunov analysis for (S-ISIHD_NS). A local convergence analysis becomes also easily accessible through this lens, whereas it is barely possible otherwise. We also specialize our results to the standard case where $\gamma \left(t\right)=\alpha /t$ and $\beta \left(t\right)=t/(\alpha -1)$.

The idea of passing from a first-order system to a second-order one via time scaling is not new. In the smooth case ($g\equiv 0$), Cabot (2009) proposes time scaling and a tricky change of variables to show that (IGS_γ) is equivalent to an averaged gradient system, that is, the steepest gradient system (GF) where the instantaneous value of $\nabla f(x(t))$ is replaced by some average of the gradients $\nabla f(x(s))$ over all past positions $s\le t$. See also Goudou and Munier (2005) for more general gradient systems with memory terms involving kernels. This gives rise to an integro-differential equation whose asymptotic behavior and the equivalent second-order dynamic have been investigated in Cabot (2009). A stochastic version of this integro-differential equation has been studied in Gadat and Panloup (2014), where the long time behavior of the resulting process, in particular its invariant distribution and occupation measure, was investigated under ellipticity assumptions on f and $\sigma$ and proper behavior of the averaging gradient function. Clearly, the motivation of that work is not on the minimizing properties of the process, whereas it is our focus here. We remark that after simple integration on (S-ISIHD), we can get that

where $p(t)=\text{exp}({\int }_{t_0}^t\gamma (s)ds)$. This representation highlights the link with integro-differential equations.

1.5. Organization of the Paper

Section 2 introduces notations, definitions, and preliminaries that are essential to our exposition. Section 3 is the main part of our study. We develop the passage from the first-order system to the second-order inertial system by using the time scaling and averaging in a stochastic framework. Almost sure and ergodic convergence rates are provided under different geometric properties of the objective function, such as convexity and Polyak-Łojasiewicz geometry. Finally, we show a strong convergence result when adding a Tikhonov regularization. Additional technical results that are needed throughout the paper are gathered in Appendix B.

2. Notation and Preliminaries

2.1. Notation

We will use the following shorthand notations: Given $n\in \mathbb{N}$, $[n]\stackrel{\text{def}}{=}\{1,\dots ,n\}$. Consider $\mathbb{H},\mathbb{K}$ real separable Hilbert spaces endowed with the inner product ${〈·,·〉}_{\mathbb{H}}$ and ${〈·,·〉}_{\mathbb{K}}$, respectively, and norm $\Vert ·{\Vert }_{\mathbb{H}}=\sqrt{{〈·,·〉}_{\mathbb{H}}}$ and $\Vert ·{\Vert }_{\mathbb{K}}=\sqrt{{〈·,·〉}_{\mathbb{K}}}$, respectively (we will omit the subscripts $\mathbb{H}$ and $\mathbb{K}$ whenever it is clear from the context); $I_{\mathbb{H}}$ is the identity operator from $\mathbb{H}$ to $\mathbb{H}$; $\mathcal{L}(\mathbb{K};\mathbb{H})$ is the space of bounded linear operators from $\mathbb{K}$ to $\mathbb{H}$; ${\mathcal{L}}_1(\mathbb{K})$ is the space of trace-class operators; and ${\mathcal{L}}_2(\mathbb{K};\mathbb{H})$ is the space of bounded linear Hilbert-Schmidt operators from $\mathbb{K}$ to $\mathbb{H}$. For $M\in {\mathcal{L}}_1(\mathbb{K})$, the trace is defined by

where $I\subseteq \mathbb{N}$ and ${(e_i)}_{i\in I}$ is an orthonormal basis of $\mathbb{K}$. Besides, for $M\in \mathcal{L}(\mathbb{K};\mathbb{H})$, $M^{\star }\in \mathcal{L}(\mathbb{H};\mathbb{K})$ is the adjoint operator of M, and for $M\in {\mathcal{L}}_2(\mathbb{K};\mathbb{H})$,is its Hilbert-Schmidt norm (in the finite-dimensional case is equivalent to the Frobenius norm). We denote by w-lim (respectively, s-lim) the limit for the weak (respectively, strong) topology of $\mathbb{H}$. The notation $A:\mathbb{H}\rightrightarrows \mathbb{H}$ means that A is a set-valued operator from $\mathbb{H}$ to $\mathbb{H}$. For $f:\mathbb{H}\to \mathbb{R}$, the sublevel of f at height $r\in \mathbb{R}$ is denoted $[f\le r]\stackrel{\text{def}}{=}\{x\in \mathbb{H}:f(x)\le r\}$. For $1\le p\le +\infty$, ${\text{L}}^p([a,b])$ is the space of measurable functions $g:\mathbb{R}\to \mathbb{R}$ such that ${\int }_a^b|g(t){|}^{p}dt<+\infty$, with the usual adaptation when $p=+\infty$. On the probability space $(\mathrm{\Omega },\mathcal{F},\mathbb{P})$, ${\text{L}}^p(\mathrm{\Omega };\mathbb{H})$ denotes the (Bochner) space of $\mathbb{H}$-valued random variables whose pth moment (with respect to the measure $\mathbb{P}$) is finite. Other notations will be explained when they first appear.

Let us recall some important definitions and results from convex analysis; for a comprehensive coverage, we refer the reader to Rockafellar (1997).

We denote by ${\mathrm{\Gamma }}_0(\mathbb{H})$ the class of proper lsc and convex functions on $\mathbb{H}$ taking values in $\mathbb{R}\cup \{+\infty \}$. For $\mu >0$, ${\mathrm{\Gamma }}_{\mu }(\mathbb{H})\subset {\mathrm{\Gamma }}_0(\mathbb{H})$ is the class of $\mu$-strongly convex functions, that is, functions f such that $f-\mu \Vert ·{\Vert }^2/2$ is convex. We denote by $C^s(\mathbb{H})$ the class of s-times continuously differentiable functions on $\mathbb{H}$. For $L\ge 0$, $C_L^{1,1}(\mathbb{H})\subset C^1(\mathbb{H})$ is the set of functions on $\mathbb{H}$ whose gradient is L-Lipschitz continuous, and $C_L^2(\mathbb{H})$ is the subset of $C_L^{1,1}(\mathbb{H})$ whose functions are twice differentiable.

The subdifferential of a function $f\in {\mathrm{\Gamma }}_0(\mathbb{H})$ is the set-valued operator $\partial f:\mathbb{H}\rightrightarrows \mathbb{H}$ such that, for every x in $\mathbb{H}$,

which is nonempty for every point in the relative interior of the domain of f. When f is finite-valued, then f is continuous, and $\partial f(x)$ is a nonempty convex and compact set for every $x\in \mathbb{H}$. If f is differentiable, then $\partial f(x)=\{\nabla f(x)\}$. For every $x\in \mathbb{H}$ such that $\partial f(x)\ne \varnothing$, the minimum norm selection of $\partial f(x)$ is the unique element $\{{\partial }^{0}f(x)\}\stackrel{\text{def}}{=}\text{arg}\hspace{0.17em}{\text{min}}_{u\in \partial f(x)}\Vert u\Vert$. The projection of a point $x\in \mathbb{H}$ onto a nonempty closed convex set $C\subseteq \mathbb{H}$ is denoted by ${\text{P}}_C(x)$.

2.2. Assumptions on Volatility and Damping Parameters

Recall that our focus in this paper is on an optimization perspective, and as we argued in the Introduction, we will study the long time behavior of our SDEs and SDIs (in particular (S-ISIHD) and (S-ISIHD_NS)) as the diffusion term vanishes when $t\to +\infty$. Therefore, throughout the paper, we assume that the diffusion (volatility) term $\sigma$ satisfies

for some $l_0>0$ and for all $t\ge t_0,x,x^{\prime }\in \mathbb{H}$. The Lipschitz continuity assumption is mild and classical and will be required to ensure the well posedness of (S-ISIHD) and (S-ISIHD_NS). Let us also define ${\sigma }_{\infty }:[t_0,+\infty [\to {\mathbb{R}}_{+}$ as

Remark 1.

(Hσ) implies the existence of ${\sigma }_{*}>0$ such that

$$\Vert \sigma (t,x){\Vert }_{\text{HS}}^2=\text{tr}[\mathrm{\Sigma }(t,x)]\le {\sigma }_{*}^2,$$

for all $t\ge t_0,x\in \mathbb{H}$, where $\mathrm{\Sigma }\stackrel{\text{def}}{=}\sigma {\sigma }^{\star }$.

For $t_0>0$, let $\gamma :[t_0,+\infty [\to {\mathbb{R}}_{+}$ be a viscous damping and define

We assume that

Remark 2.

Let us notice that $\mathrm{\Gamma }$ satisfies the relation ${\mathrm{\Gamma }}^{\prime }=\gamma \mathrm{\Gamma }-1.$

2.3. Reminder of Main Previous Results

Before we delve into our core contributions, it is important to note that we will require some specific results gleaned from Maulen-Soto et al. (2024, 2025). These are the subject of the following paragraphs.

2.3.1. Results on (SGF).

Because we are going to show results in the smooth case, we rewrite (H₀) when $g\equiv 0$:

Theorem 1

(Maulen-Soto et al. 2024, Theorem 3.1). Consider f and $\sigma$ that satisfy Assumptions (H₀) and (H_σ). Let $\nu \ge 2$ and consider the SDE

$$\{\begin{array}{l}\begin{array}{l}dX(t)=-\nabla f(X(t))dt+\sigma (t,X(t))dW(t),\hfill \\ X(t_0)=X_0,\hfill \end{array}\hfill \end{array}$$(1)

where $X_0\in {\text{L}}^{\nu }(\mathrm{\Omega };\mathbb{H})$. Then, there exists a unique solution $X\in S_{\mathbb{H}}^{\nu }[t_0]$ (see Section 7.2 for the notation) of (1). Additionally, if ${\sigma }_{\infty }\in {\text{L}}^2([t_0,+\infty [)$, then

1. ${\sup }_{t\ge 0}\mathbb{E}[{\Vert X(t)\Vert }^2]<+\infty$.

2. $\forall x^{\star }\in \mathcal{S}$, ${\lim }_{t\to +\infty }\Vert X(t)-x^{\star }\Vert$ exists a.s. and ${\sup }_{t\ge 0}\Vert X(t)\Vert <+\infty$ a.s.

3. ${\lim }_{t\to \infty }\Vert \nabla f(X(t))\Vert =0$ a.s. As a result, ${\lim }_{t\to +\infty }f\left(X\right(t\left)\right)=\min f$ a.s.

4. There exists an $\mathcal{S}$-valued random variable $X^{\star }$ such that ${\text{w}-\text{lim}}_{t\to +\infty }X(t)=X^{\star }$ a.s.

Theorem 2

(Maulen-Soto et al. 2024, Theorem 3.4). Let $\nu \ge 2$ and consider the SDE (1) with initial data $X_0\in {\text{L}}^{\nu }(\mathrm{\Omega };\mathbb{H})$, where f and $\sigma$ satisfy Assumptions (H₀) and (H_σ). Moreover, we assume that $\sigma$ satisfies $t\mapsto t{\sigma }_{\infty }^2\left(t\right)\in {\text{L}}^1\left(\right[t_0,+\infty \left[\right)$ and that either $\mathbb{H}$ is finite-dimensional or $f\in C^2(\mathbb{H})$. Then, the solution trajectory $X\in S_{\mathbb{H}}^{\nu }[t_0]$ is unique and we have that

1. $\mathbb{E}[f(X(t))-\min f]=\mathcal{O}(t^{-1}).$

   Moreover, if $f\in C^2(\mathbb{H})$, then the following hold:

2. $t\mapsto t\Vert \nabla f(X(t)){\Vert }^2\in {\text{L}}^1([t_0,+\infty [)$ a.s.

3. $f(X(t))-\min f=o(t^{-1})$ a.s.

2.3.2. Results on (SGI).

The far more intricate nonsmooth version of (SGF) has been recently studied in Maulen-Soto et al. (2025). Let $F,\sigma$ satisfy (H₀) and (H_σ). We consider the stochastic differential inclusion

The following definition makes precise the notion of solution that we are interested in.

Definition 1.

A solution of (SGI) is a couple $(X,\eta )$ of ${\mathcal{F}}_t$-adapted processes such that almost surely

1. X is continuous with sample paths in the domain of $\partial g$.

2. $\eta :[t_0,+\infty [\to \mathbb{H}$ is absolutely continuous, such that $\eta (t_0)=0$, and $\forall T>t_0$, ${\eta }^{\prime }\in {\text{L}}^2([t_0,T];\mathbb{H})$, ${\eta }^{\prime }(t)\in \partial g(X(t))$ for almost all $t\ge t_0$;

3. For $t>t_0$,

   $$\begin{array}{l}\{\begin{array}{l}X(t)=X_0-{\int }_{t_0}^t\nabla f(X(s))ds-\eta (t)+{\int }_{t_0}^t\sigma (s,X(s))dW(s),\hfill \\ X(t_0)=X_0.\hfill \end{array}\hfill \end{array}$$(2)

For brevity, we sometimes omit the process $\eta$ and say that X is a solution of (SGI), meaning that, there exists a process $\eta$ such that $(X,\eta )$ satisfies the previous definition. The definition of uniqueness for the process X is given in Section 7.2.

In order to show the main results for (SGI), we consider the sequence of solutions ${(X_{\lambda })}_{\lambda >0}$ of the SDEs

$$\{\begin{array}{l}\begin{array}{l}d{X}_{\lambda }(t)=-\nabla (f+g_{\lambda })(X_{\lambda }(t))dt+\sigma (t,X_{\lambda }(t))dW(t),\hspace{1em}t>t_0,\hfill \\ X_{\lambda }(t_0)=X_0,\hfill \end{array}\hfill \end{array}$$(SDE_λ)

where $g_{\lambda }$ is the Moreau envelope of g with parameter $\lambda >0$. Observe that system (SDEλ) was also directly studied in Maulen-Soto et al. (2024) to solve (P), where a complete discussion is provided. It is worth emphasizing that the Moreau envelope of g, and the corresponding system (SDEλ) is only needed as a means to establish existence and uniqueness of solution of (SGI). This follows the same reasoning as in the deterministic case where the Moreau-Yosida regularization and nonlinear semigroup theory have proven very useful to show existence and uniqueness for differential inclusions Brézis (1973). However, in the convergence analysis, we never replace g by its Moreau envelope.

We define the integrability condition that for every $T>t_0$,

$$\underset{\lambda \downarrow 0}{\lim \sup }{\displaystyle {\int }_{t_0}^T\mathbb{E}}(\Vert \nabla g_{\lambda }(X_{\lambda }(t)){\Vert }^2)dt<+\infty ,$$(H_λ)

Remark 3.

Some conditions on g for (H_λ) to be satisfied are discussed in Pettersson (1995). Of particular interest, which covers a broad class of functions, is when $\partial g$ has full domain and there exists $C_0>0$ such that

$$\Vert {\partial }^{0}g(x)\Vert \le C_0(1+\Vert x\Vert ),\hspace{1em}\forall x\in \mathbb{H}.$$

This is, for instance, the case when g is Lipschitz continuous.

The following results on (SGI) were proved in Maulen-Soto et al. (2025).

Theorem 3

(Maulen-Soto et al. 2025). Consider $F=f+g$ and $\sigma$ satisfying (H₀) and (H_σ). Suppose further that g verifies (H_λ). Let $\nu \ge 2$, $t_0\ge 0$, and consider the dynamic (SGI) with initial data $X_0\in {\text{L}}^{\nu }(\mathrm{\Omega };\mathbb{H})$. Then, (SGI) has a unique solution in the sense of Definition 1 $(X,\eta )\in S_{\mathbb{H}}^{\nu }[t_0]\times C^1([t_0,+\infty [;\mathbb{H})$.

Moreover, if ${\sigma }_{\infty }\in {\text{L}}^2([t_0,+\infty [)$, then the following holds:

1. $\mathbb{E}[{\sup }_{t\ge t_0}{\Vert X(t)\Vert }^{\nu }]<+\infty$.

2. $\forall x^{\star }\in {\mathcal{S}}_F$, ${\lim }_{t\to +\infty }\Vert X(t)-x^{\star }\Vert$ exists a.s. and ${\sup }_{t\ge t_0}\Vert X(t)\Vert <+\infty$ a.s.

3. If g is continuous, then $\nabla f$ is constant on ${\mathcal{S}}_F$, there exists $x^{\star }\in {\mathcal{S}}_F$ such that ${\text{s}-\text{lim}}_{t\to +\infty }\nabla f\left(X\right(t\left)\right)=\nabla f\left(x^{\star }\right)$ a.s., and

   $${\int }_{t_0}^{+\infty }\left(F\right(X\left(t\right))-\min \mathit{\text{F)dt}}<+\infty \hspace{1em}a.s\mathrm{.}$$

4. There exists an ${\mathcal{S}}_F$-valued random variable $X^{\star }$ such that ${\text{w}-\text{lim}}_{t\to +\infty }X(t)=X^{\star }$.

2.3.3. Tikhonov Regularization.

Let us now turn to a Tikhonov regularization of (SGI), that is,

Solution existence and uniqueness for (SGI-TA) are established in Maulen-Soto et al. (2025, theorem 3.3). We also have the following.

Theorem 4

(Maulen-Soto et al. 2025, Theorem 4.1). Let $\nu \ge 2$ and consider the dynamic (SGI-TA) with initial data $X_0\in {\text{L}}^{\nu }(\mathrm{\Omega };\mathbb{H})$, where $F=f+g$ and $\sigma$ satisfy Assumptions (H₀) and (H_σ). Furthermore, assume that g satisfies (H_λ). Then, there exists a unique solution $X\in S_{\mathbb{H}}^{\nu }[t_0]$ of (SGI-TA). Let $x^{\star }={\text{P}}_{{\mathcal{S}}_F}(0)$ be the minimum norm solution, and for $\epsilon >0$ let $x_{\epsilon }$ be the unique minimizer of $F_{\epsilon }\left(x\right)\stackrel{\text{def}}{=}F\left(x\right)+\epsilon \Vert x{\Vert }^2/2$. Suppose that ${\sigma }_{\infty }\in {\text{L}}^2([t_0,+\infty [)$, and that $\epsilon :[t_0,+\infty [\to {\mathbb{R}}_{+}$ satisfies the conditions:

(T₁) $\epsilon (t)\to 0$ as $t\to +\infty$;

(T₂) ${\int }_{t_0}^{+\infty }\epsilon (t)dt=+\infty$;

(T₃) ${\int }_{t_0}^{+\infty }\epsilon (t)(\Vert x^{\star }{\Vert }^2-\Vert x_{\epsilon (t)}{\Vert }^2)dt<+\infty .$

Then, we have

1. ${\sup }_{t\ge t_0}\Vert X(t)\Vert <+\infty$ a.s., and

2. ${\text{s}-\text{lim}}_{t\to +\infty }X(t)=x^{\star }$ a.s.

This means that we can obtain almost sure strong convergence of the trajectory to a particular (nonrandom) solution: the one of minimal norm.

3. From First-Order to Second-Order Systems

3.1. Time Scaling and Averaging

We apply a time scaling and then an averaging technique to the system (SGI) to derive an insightful reparametrization of a particular case of our second-order system (S-ISIHD_NS), specifically, the case when $\beta \equiv \mathrm{\Gamma }$. The main advantage of this method is that the results of (SGI) directly carry over to obtain results on the convergence behavior of (S-ISIHD_NS) without passing through a dedicated Lyapunov analysis. However, as discussed in the introduction, the averaging technique is restricted to convex objectives, as it heavily relies on Jensen’s inequality.

Let $\nu \ge 2$, $s_0>0$. We consider the potential $F=f+g$ where g satisfies (H_λ). Let ${\sigma }_1$ be a diffusion term in the time parametrization by s. We will study the dynamic (SGI) in s, starting at $s_0$, with diffusion term ${\sigma }_1$ under Hypotheses (H₀) and (H_σ). Let ${\sigma }_{1_{*}}>0$ be such that

and ${\sigma }_{1_{\infty }}(s)\stackrel{\text{def}}{=}{\sup }_{x\in \mathbb{H}}\Vert {\sigma }_1(s,x){\Vert }_{\text{HS}}$. We rewrite (SGI) in a format adapted to our case,where $Z_0\in {\text{L}}^{\nu }([s_0,+\infty [;\mathbb{H})$.

Let us make the change of time $s=\tau (t)$ in the dynamic (3), where $\tau$ is an increasing function from $[t_0,+\infty [$ to $[s_0,+\infty [$, which is twice differentiable and which satisfies ${\lim }_{t\to +\infty }\tau (t)=+\infty$. Denote $Y(t)\stackrel{\text{def}}{=}Z(s)$ and $t_0$ be such that $s_0=\tau (t_0)$. By the chain rule and Öksendal 2003 (theorem 8.5.7), we have

Consider the smooth case, that is, when $g\equiv 0$ and the hypotheses of Theorem 2 ($f\in C_L^2(\mathbb{H})$ and ${\sigma }_1\in {\text{L}}^2([s_0,+\infty [)$), then we can conclude that the convergence rate of (4) (when $g\equiv 0$) is the following:

By introducing a function $\tau$ that grows faster than the identity ($\tau (t)\ge t$), we have accelerated the dynamic, passing from the asymptotic convergence rate $1/s$ for (3) to $1/\tau (t)$ for (4). The price to pay is that the drift term in (4) is nonautonomous; furthermore, when the coefficient in front of the gradient tends to infinity as $t\to +\infty$, it will preclude the use of an explicit discretization in time. To overcome this, we adapt from Attouch et al. (2024) the following approach, which is called averaging.

Consider (4) and define the two stochastic processes $X,V:\mathrm{\Omega }\times [t_0,+\infty [\to \mathbb{H}$ as

where $Y(t)$ is the process in (4), and $X_0,V_0\in {\text{L}}^{\nu }(\mathrm{\Omega };\mathbb{H})$ are initial data. This leads us to set $Z_0\stackrel{\text{def}}{=}{X}_0+{\tau }^{\prime }(t_0)V_0$ in order for the equations to fit. According to the averaging, the differential form of $Y(t)$ is

Combining the previous equation with (4), we have that

Using that $dX(t)=V(t)dt$ and dividing by ${\tau }^{\prime }$, we then have

Therefore, after the time scaling and averaging, we obtain the following dynamic:

Let $\gamma :[t_0,+\infty [\to {\mathbb{R}}_{+}$ satisfy (Hγ). We are going to determine $\tau$ in order to obtain a viscous damping coefficient equal to $\gamma$, that is,

Clearly, ${\tau }^{\prime }$ solves the following ODE in $\zeta$:

As observed in Remark 2, the function $\mathrm{\Gamma }$ also satisfies the same ODE, and thus we can adjust the initial condition of ${\tau }^{\prime }$ to obtain

We then integrate and take $\tau (t)=s_0+{\int }_{t_0}^t\mathrm{\Gamma }(u)du$ to get $\tau (t_0)=s_0$ as required. This is a valid selection of $\tau$ because $t\mapsto s_0+{\int }_{t_0}^t\mathrm{\Gamma }(u)du$ is an increasing function from $[t_0,+\infty [$ to $[s_0,+\infty [$, twice differentiable and $\mathrm{\Gamma }\notin {\text{L}}^1([t_0,+\infty [)$ because $\mathrm{\Gamma }$ is lower bounded by a nondecreasing function because $\gamma$ is upper bounded by a nonincreasing function (Attouch and Cabot 2017, proposition 2.2) by (Hγ). For this particular selection of $\tau$, and defining ${\tilde{\sigma }}_1(t,·)\stackrel{\text{def}}{=}{\sigma }_1\left(\tau \right(t),·)/\sqrt{\mathrm{\Gamma }\left(t\right)}$, we have that (ISIHD-S.1) is equivalent to

Clearly, (ISIHD-S.2) is nothing but (S-ISIHD_NS) when $\beta \equiv \mathrm{\Gamma }$ and $\sigma \equiv {\tilde{\sigma }}_1$.

In order to be able to transfer the convergence results on Z in (3) (via (4)) to X in (ISIHD-S.2), it remains to express $X$ in terms of $Y$ only. For this, let

Recalling the averaging in (6), we need to integrate the following equation:

Multiplying both sides by $e^{A(t)}$ and using (6), we get equivalently

Integrating and again using (6), we obtain

Then we can write

where ${\mu }_t$ is the probability measure on $[t_0,t]$ defined bywhere ${\delta }_{t_0}$ is the Dirac measure at $t_0$, $a(u)e^{A(u)-A(t)}du$ is the measure with density $a(·)e^{A(·)-A(t)}$ with respect to the Lebesgue measure on $[t_0,t]$, and $\xi (t)$ is a random process since $V_0$ is a random variable, that is,

3.2. Convergence of the Trajectory and Convergence Rates Under General $\gamma$, and $\beta \equiv \mathrm{\Gamma }$

We state the main results of this section. We first show almost sure convergence of the trajectory of (S-ISIHD_NS) to a random variable taking values in the set of minimizers of F. When $g\equiv 0$, we also provide convergence rates.

The following result imposes an integrability condition on the diffusion term, which is essential for ensuring the almost sure weak convergence of the trajectory. More precisely, this integrability condition allows to show that the trajectory asymptotically converges almost surely, in the weak topology, to a random variable that takes values in the set of minimizers of F.

Theorem 5.

Let $\nu \ge 2$ and consider the dynamic (S-ISIHD_NS) with initial data $X_0,V_0\in {\text{L}}^{\nu }(\mathrm{\Omega };\mathbb{H})$, where $\gamma :[t_0,+\infty [\to {\mathbb{R}}_{+}$ satisfies (H_γ), and $\beta \equiv \mathrm{\Gamma }$. Besides, $F=f+g$ and $\sigma$ satisfy Assumptions (H₀) and (H_σ). Moreover, suppose that g satisfies (H_λ). Then, there exists a unique solution $(X,V)\in S_{\mathbb{H}\times \mathbb{H}}^{\nu }[t_0]$ of (S-ISIHD_NS). Additionally, if $\mathrm{\Gamma }{\sigma }_{\infty }\in {\text{L}}^2([t_0,+\infty [)$, then there exists an ${\mathcal{S}}_F$-valued random variable $X^{\star }$ such that ${\text{w}-\text{lim}}_{t\to +\infty }X\left(t\right)=X^{\star }$ a.s. and ${\text{w}-\text{lim}}_{t\to +\infty }\mathrm{\Gamma }\left(t\right)V\left(t\right)=0.$ a.s.

Proof.

Let $\theta (t)\stackrel{\text{def}}{=}{\int }_{t_0}^t\mathrm{\Gamma }(u)du$, $\tilde{\sigma }(s,·)\stackrel{\text{def}}{=}\sigma ({\theta }^{-1}(s),·)\sqrt{\mathrm{\Gamma }({\theta }^{-1}(s))}$, and ${\tilde{\sigma }}_{\infty }(s)\stackrel{\text{def}}{=}{\sup }_{x\in \mathbb{H}}\Vert \tilde{\sigma }(s,x){\Vert }_{\text{HS}}$. Then $\mathrm{\Gamma }{\sigma }_{\infty }\in {\text{L}}^2([t_0,+\infty [)$ is equivalent to ${\tilde{\sigma }}_{\infty }\in {\text{L}}^2({\mathbb{R}}_{+})$. Consider the dynamic:

$$\{\begin{array}{l}dZ(s)\in -\partial F(Z(s))+\tilde{\sigma }(s,Z(s))dW(s),\hspace{1em}s>0,\hfill \\ Z(0)=X_0+\mathrm{\Gamma }(t_0)V_0.\hfill & \hfill \end{array}$$(12)

By Theorem 3, we have that there exists a unique solution $(Z,\eta )\in S_{\mathbb{H}}^{\nu }\times C^1({\mathbb{R}}_{+};\mathbb{H})$ of (12) and an ${\mathcal{S}}_F$-valued random variable $X^{\star }$ such that ${\text{w}-\text{lim}}_{s\to +\infty }Z\left(s\right)=X^{\star }$ a.s. Moreover, using the time scaling $\tau \equiv \theta$ and the averaging described in this section, we end up with the dynamic (S-ISIHD_NS) in the case where $\beta \equiv \mathrm{\Gamma }$.

It is direct to check that the time scaling and averaging preserves the uniqueness of a solution $(X,V)\in S_{\mathbb{H}\times \mathbb{H}}^0[t_0]$. Now let us validate $(X,V)\in S_{\mathbb{H}\times \mathbb{H}}^{\nu }[t_0]$. Because

$$\mathbb{E}\left(\underset{s\in [0,T]}{\sup }\Vert Z(s){\Vert }^{\nu }\right)<+\infty ,\hspace{1em}\forall T>0,$$

we directly obtain

$$\mathbb{E}\left(\underset{t\in [t_0,T]}{\sup }\Vert Y(t){\Vert }^{\nu }\right)<+\infty ,\hspace{1em}\forall T>t_0.$$

Thanks to Relation (9), the following holds:

$$\begin{array}{ll}\Vert X(t){\Vert }^{\nu }\hfill & \le \nu \left({\Vert X(t)-{\displaystyle {\int }_{t_0}^{t}Y(u)}d{\mu }_t(u)\Vert }^{\nu }+{\Vert {\displaystyle {\int }_{t_0}^{t}Y(u)}d{\mu }_t(u)\Vert }^{\nu }\right)\hfill \\ \hfill & \le \nu \left(\Vert \xi (t){\Vert }^{\nu }+{(t-t_0)}^{\nu -1}{\displaystyle {\int }_{t_0}^t\Vert Y(u){\Vert }^{\nu }}d{\mu }_t(u)\right).\hfill \end{array}$$

Let $T>t_0$ be arbitrary. Taking supremum over $[t_0,T]$ and then expectation at both sides, we obtain that

$$\mathbb{E}\left(\underset{t\in [t_0,T]}{\sup }\Vert X(t){\Vert }^{\nu }\right)\le \nu \left(\mathbb{E}(\Vert V_0{\Vert }^{\nu })\Vert \mathrm{\Gamma }(t_0){\Vert }^{\nu }+{(T-t_0)}^{\nu -1}\mathbb{E}\left(\underset{t\in [t_0,T]}{\sup }\Vert Y(t){\Vert }^{\nu }\right)\right)<+\infty .$$

Because $V(t)=\frac{Y(t)-X(t)}{\mathrm{\Gamma }(t)}$, we have

$$\Vert V(t){\Vert }^{\nu }\le \frac{\nu }{{\mathrm{\Gamma }}^{\nu }(t)}(\Vert Y(t){\Vert }^{\nu }+\Vert X(t){\Vert }^{\nu }).$$

Similar to before, we let $T>t_0$ be arbitrary, and take the supremum over $[t_0,T]$ and then expectation at both sides to obtain

$$\mathbb{E}\left(\underset{t\in [t_0,T]}{\sup }\Vert V(t){\Vert }^{\nu }\right)\le \nu \underset{t\in [t_0,T]}{\sup }\frac{1}{{\mathrm{\Gamma }}^{\nu }(t)}\left(\mathbb{E}\left(\underset{t\in [t_0,T]}{\sup }(\Vert Y(t){\Vert }^{\nu }+\Vert X(t){\Vert }^{\nu })\right)\right).$$

Because $\mathrm{\Gamma }$ is a continuous positive function, by the extreme value theorem, we have that there exists $t_T\in [t_0,T]$ such that ${\sup }_{t\in [t_0,T]}1/{\mathrm{\Gamma }}^{\nu }\left(t\right)=1/{\mathrm{\Gamma }}^{\nu }\left(t_T\right)<+\infty$, and we conclude that $(X,V)\in S_{\mathbb{H}\times \mathbb{H}}^{\nu }[t_0]$.

Now we prove that there exists an ${\mathcal{S}}_F$-valued random variable $X^{\star }$ such that ${\text{w}-\text{lim}}_{t\to +\infty }X\left(t\right)=X^{\star }$ a.s. By virtue of Theorem 3, there exists an ${\mathcal{S}}_F$-valued random variable $X^{\star }$ such that ${\text{w}-\text{lim}}_{s\to +\infty }Z\left(s\right)=X^{\star }$ a.s. We also notice that we have directly ${\text{w}-\text{lim}}_{t\to +\infty }Y\left(t\right)=X^{\star }$ a.s. Let $h\in \mathbb{H}$ be arbitrary and use Relation (9) as follows:

$$\begin{array}{ll}|〈X(t)-X^{\star },h〉|\hfill & \le |〈X(t)-{\displaystyle {\int }_{t_0}^{t}Y(u)}d{\mu }_t(u),h〉|+|〈{\displaystyle {\int }_{t_0}^{t}Y(u)}d{\mu }_t(u)-X^{\star },h〉|\hfill \\ \hfill & =|〈\xi (t),h〉|+|〈{\displaystyle {\int }_{t_0}^t(Y(u)-X^{\star })}d{\mu }_t(u),h〉|\hfill \\ \hfill & =|〈\xi (t),h〉|+|{\displaystyle {\int }_{t_0}^t〈Y(u)-X^{\star },h〉}d{\mu }_t(u)|\hfill \\ \hfill & \le \Vert \xi (t)\Vert \Vert h\Vert +{\displaystyle {\int }_{t_0}^t|}〈Y(u)-X^{\star },h〉|d{\mu }_t(u),\hfill \end{array}$$

where the second equality comes from the dominated convergence theorem, because ${\sup }_{s>t_0}\Vert Y(s)\Vert <+\infty$ a.s. (by (ii) of Theorem 3).

Now let $a\left(t\right)=1/\mathrm{\Gamma }\left(t\right)$ and $A\left(t\right)={\int }_{{\mathit{t}}_0}^{t}du/\mathrm{\Gamma }\left(u\right)$. By Lemma 3 (defined in Section 7.1) and the fact that $V_0\in {\text{L}}^{\nu }(\mathrm{\Omega };\mathbb{H})$, we have that ${\lim }_{t\to +\infty }\Vert \xi \left(t\right)\Vert =0$ a.s. On the other hand, it holds that

$$\begin{array}{ll}{\displaystyle {\int }_{t_0}^t|}〈Y(u)-X^{\star },h〉|d{\mu }_t(u)\hfill & \le e^{-A(t)}|〈Y(t_0)-X^{\star },h〉|\hfill \\ \hfill & +e^{-A(t)}{\displaystyle {\int }_{t_0}^{t}a}(u)e^{A(u)}|〈Y(u)-X^{\star },h〉|du.\hfill \end{array}$$

Now let $b(u)=|〈Y(u)-X^{\star },h〉|$. Because we already proved that ${\lim }_{u\to +\infty }b(u)=0$ a.s., and we have that $a\notin {\text{L}}^1([t_0,+\infty [)$ by Lemma 3, we utilize Lemma 2 (also defined in Section 7.1) with our respective a, b functions. This let us conclude that

$$\underset{t\to +\infty }{\lim }|〈X(t)-X^{\star },h〉|=0\hspace{1em}\text{a}.\text{s}.$$

Thus, ${\text{w}-\text{lim}}_{t\to +\infty }X\left(t\right)=X^{\star }$ a.s. Finally, because

$$Y(t)=X(t)+\mathrm{\Gamma }(t)V(t),$$

and that $X$ and $Y$ have (a.s.) the same limit, we conclude that

$$\underset{t\to +\infty }{\text{w}-\lim }\mathrm{\Gamma }\left(t\right)V\left(t\right)=0\hspace{1em}\text{a}.\text{s}.\hspace{1em}\square$$

In the smooth case, we also have convergence rates on the objective value and the gradient. In particular, the following two theorems will provide general abstract convergence rates under the same integrability condition on the diffusion term. These results will be specialized to specific choice of the parameters in Section 3.3.

Theorem 6.

Let $\nu \ge 2$ and consider the dynamic (S-ISIHD) with initial data $X_0,V_0\in {\text{L}}^{\nu }(\mathrm{\Omega };\mathbb{H})$, such that f and $\sigma$ satisfy (H₀) and (H_γ), and in the case where $\gamma$ satisfies (Hγ), $\beta \equiv \mathrm{\Gamma }$. Moreover, suppose that either $\mathbb{H}$ is finite dimensional or $f\in C^2(\mathbb{H})$, and

$$t\mapsto \sqrt{\theta (t)}\mathrm{\Gamma }(t){\sigma }_{\infty }(t)\in {\text{L}}^2([t_0,+\infty [),$$

where $\theta (t)\stackrel{\text{def}}{=}{\int }_{t_0}^t\mathrm{\Gamma }(u)du$. Then the solution trajectory $(X,V)\in S_{\mathbb{H}\times \mathbb{H}}^{\nu }[t_0]$ is unique and satisfies

$$\mathbb{E}[f(X(t))-\min f]=\mathcal{O}\left(\max \{e^{-A(t)},I\left[\frac{1}{\theta }\right](t)\}\right),\hspace{1em}\forall t>t_0,$$

where $A(t)\stackrel{\text{def}}{=}{\int }_{t_0}^t\frac{du}{\mathrm{\Gamma }(u)}$ and we recall that $I\left[\frac{1}{\theta }\right](t)=e^{-A(t)}{\int }_{t_0}^t\frac{1}{\theta (u)}\frac{e^{A(u)}}{\mathrm{\Gamma }(u)}du$.

From Hypothesis (H_λ), we have that ${\lim }_{t\to +\infty }{e}^{-A(t)}=0$, and because $\mathrm{\Gamma }\notin {\text{L}}^1([t_0,+\infty [)$, we can use Lemma 2 to check that ${\lim }_{t\to +\infty }I\left[\frac{1}{\theta }\right]\left(t\right)=0$.

Remark 4.

If $\mathbb{H}$ is finite-dimensional and g is a L₀-Lipschitz function, we could obtain convergence rates for the nonsmooth setting (P) by considering (S-ISIHD) with potential $F_{\rho }\stackrel{\text{def}}{=}f+g_{\rho }$, where $g_{\rho }$ is the Moreau envelope of g with parameter $\rho >0$. With the same hypotheses on the initial conditions, $\sigma ,\gamma ,\beta$ as in Theorem 6, and proceeding as in Maulen-Soto et al. (2024, section 5), we could obtain

$$\mathbb{E}[F(X_{\rho }(t))-\min F]=\mathcal{O}\left(\max \{e^{-A(t)},I\left[\frac{1}{\theta }\right](t)\}\right)+\frac{L_0^2}{2}\rho ,\hspace{1em}\forall t>t_0,$$

where $(X_{\rho },V_{\rho })\in S_{\mathbb{H}\times \mathbb{H}}^{\nu }[t_0]$ is the solution of (S-ISIHD) with potential $F_{\rho }$.

Proof.

We will utilize the averaging technique used in Theorem 5 and Jensen’s inequality. First, we have

$$\begin{array}{ll}\mathbb{E}(f(X(t))-\min f)\hfill & =\mathbb{E}\left(f(X(t))-f\left({\displaystyle {\int }_{t_0}^{t}Y(u)}d{\mu }_t(u)\right)\right)\hfill \\ \hfill & +\mathbb{E}\left(f\left({\displaystyle {\int }_{t_0}^{t}Y(u)}d{\mu }_t(u)\right)-\min f\right).\hfill \end{array}$$

Let us recall that $\mathbb{E}({\sup }_{s\ge 0}\Vert Z(s)\Vert )<+\infty$, which implies that $\mathbb{E}({\sup }_{t\ge t_0}\Vert X(t)\Vert )<+\infty$. We bound the first term using the convexity of f and Cauchy-Schwarz inequality, that is, that for every $x,y\in \mathbb{H}$,

$$f(x)-f(y)\le 〈\nabla f(x),x-y〉\le \Vert \nabla f(x)\Vert \Vert y-x\Vert ,$$

we get that

$$\begin{array}{ll}f(X(t))-f\left({\displaystyle {\int }_{t_0}^{t}Y(u)}d{\mu }_t(u)\right)\hfill & \le \Vert \nabla f(X(t))\Vert \Vert \xi (t)\Vert \hfill \\ \hfill & \le \Vert \xi (t)\Vert (L\Vert X(t)\Vert +\Vert \nabla f(0)\Vert )\hfill \\ \hfill & \le \Vert \xi (t)\Vert \left(L\underset{t\ge t_0}{\sup }\Vert X(t)\Vert +\Vert \nabla f(0)\Vert \right),\hfill \end{array}$$

and we conclude that

$$\mathbb{E}\left(f(X(t))-f\left({\displaystyle {\int }_{t_0}^{t}Y(u)}d{\mu }_t(u)\right)\right)=\mathcal{O}(e^{-A(t)}).$$

For the second term, we use Jensen’s inequality to obtain

$$\begin{array}{ll}\mathbb{E}\left(f\left({\displaystyle {\int }_{t_0}^{t}Y(u)}d{\mu }_t(u)\right)-\min f\right)\hfill & \le {\displaystyle {\int }_{t_0}^t\mathbb{E}}[f(Y(u))-\min f]d{\mu }_t(u)\hfill \\ \hfill & \le e^{-A(t)}\mathbb{E}[f(Y(t_0))-\min f]\hfill \\ \hfill & +e^{-A(t)}{\displaystyle {\int }_{t_0}^t\frac{e^{A(u)}}{\mathrm{\Gamma }(u)}}\mathbb{E}(f(Y(u))-\min f)du.\hfill \end{array}$$

Because $\sqrt{\theta }\mathrm{\Gamma }{\sigma }_{\infty }\in {\text{L}}^2([t_0,+\infty [)$ is equivalent to $s\mapsto s{\tilde{\sigma }}_{\infty }^2(s)\in {\text{L}}^1({\mathbb{R}}_{+})$, by Theorem 2, we have that there exists $C>0$ such that $\mathbb{E}\left(f\right(Z\left(s\right))-\min f)\le C/s$. Then, we have $\mathbb{E}\left(f\right(Y\left(t\right))-\min f)\le C/\theta \left(t\right)$. Hence, there exists $C_0>0$ such that

$$\mathbb{E}(f(X(t))-\min f)\le C_0{e}^{-A(t)}+CI\left[\frac{1}{\theta }\right](t).\hspace{1em}\square$$

Theorem 7.

Let $\nu \ge 2$ and consider the dynamic (S-ISIHD) with initial data $X_0,V_0\in {\text{L}}^{\nu }(\mathrm{\Omega };\mathbb{H})$, such that f and $\sigma$ satisfy (H₀) and (H_σ), and in the case where $\gamma$ satisfies (H_γ), $\beta \equiv \mathrm{\Gamma }$. Moreover, suppose that $f\in C^2(\mathbb{H})$ and

$$t\mapsto \sqrt{\theta (t)}\mathrm{\Gamma }(t){\sigma }_{\infty }(t)\in {\text{L}}^2([t_0,+\infty [),$$

where $\theta (t)\stackrel{\text{def}}{=}{\int }_{t_0}^t\mathrm{\Gamma }(u)du$. Then the solution trajectory $(X,V)\in S_{\mathbb{H}\times \mathbb{H}}^{\nu }[t_0]$ is unique and satisfies

$${\int }_{{\mathit{t}}_0}^{+\infty }\theta \left(u\right)\mathrm{\Gamma }\left(u\right)\Vert \nabla f\left(X\right(u)+\mathrm{\Gamma }(u\left)V\right(u\left)\right){\Vert }^{2}du<+\infty \hspace{1em}\text{a}.\text{s}.$$(13)

Proof.

Consider (12) and the technique used in Theorem 5. We have that $t\mapsto \theta (t){\mathrm{\Gamma }}^2(t){\sigma }_{\infty }^2(t)\in {\text{L}}^1([t_0,+\infty [)$ is equivalent to $s\mapsto s{\tilde{\sigma }}_{\infty }^2(s)\in {\text{L}}^1({\mathbb{R}}_{+})$. Therefore, we can use Theorem 2 to state that

$${\int }_0^{+\infty }s\Vert \nabla f\left(Z\right(s\left)\right){\Vert }^{2}ds<+\infty \hspace{1em}\text{a}.\text{s}.$$

Using the time scaling $\tau \equiv \theta$ and making the change of variable $\theta (t)=s$ in the previous integral, we obtain

$${\int }_{{\mathit{t}}_0}^{+\infty }\theta \left(t\right)\mathrm{\Gamma }\left(t\right)\Vert \nabla f\left(Y\right(t\left)\right){\Vert }^{2}dt<+\infty \hspace{1em}\text{a}.\text{s}.$$

Recalling that in the averaging, we impose that $Y=X+\mathrm{\Gamma }V$, we conclude. □

3.3. Fast Convergence Under $\alpha >3, \gamma \left(t\right)=\alpha /t$ and $\beta \left(t\right)=t/(\alpha -1)$

In the following, we show fast convergence results in expectation and in almost sure sense. This result imposes an integrability condition on the diffusion term to ensure a convergence rate of $1/t^2$ for the function values. This is highly desirable, as it represents the fastest convergence rate we can expect to achieve for gradient-based second-order dynamics featuring inertia when applied to a general convex function. Besides, these results match those in Attouch et al. (2024) when there is no noise.

Corollary 1

(Case $\alpha /t$). Let $\nu \ge 2,\alpha >3$ and consider the dynamic (S-ISIHD) with initial data $X_0,V_0\in {\text{L}}^{\nu }(\mathrm{\Omega };\mathbb{H})$, in the case where $\gamma \left(t\right)=\alpha /t$ and $\beta \left(t\right)=t/(\alpha -1)$. Besides, consider that f and $\sigma$ satisfy (H₀) and (H_σ). Moreover, let $f\in C^2(\mathbb{H})$ and $t\mapsto t^2{\sigma }_{\infty }(t)\in {\text{L}}^2([t_0,+\infty [)$. Then the solution trajectory $(X,V)\in S_{\mathbb{H}\times \mathbb{H}}^{\nu }[t_0]$ is unique and satisfies

1. $f(X(t))-\min f=o(t^{-2})$ a.s.

2. $\mathbb{E}[f(X(t))-\min f]=\mathcal{O}(t^{-2})$.

   $${\int }_{{\mathit{t}}_0}^{+\infty }{t}^3{\Vert \nabla f\left(X\left(t\right)+\frac{t}{\alpha -1}V\left(t\right)\right)\Vert }^{2}dt<+\infty \hspace{1em}a.s.$$

Proof.

Consider (12) with $\mathrm{\Gamma }\left(t\right)=t/(\alpha -1)$ and $\theta \left(t\right)=(t^2-t_0^2)/\left(2\right(\alpha -1\left)\right)$. Let $\tilde{\sigma }(s,·)=\sigma ({\theta }^{-1}(s),·)\sqrt{\mathrm{\Gamma }({\theta }^{-1}(s))}$. Notice that $t\mapsto t^2{\sigma }_{\infty }(t)\in {\text{L}}^2([t_0,+\infty [)$ is equivalent to $s\mapsto s{\tilde{\sigma }}_{\infty }^2(s)\in {\text{L}}^1({\mathbb{R}}_{+})$. We apply Theorem 2 to deduce that

$$f(Z(s))-\min f=o(s^{-1}) \text{a}.\text{s}.$$

Using the time scaling $\tau \equiv \theta$ and then the averaging technique as in the proof of Theorem 5, we have that

$$f(Y(t))-\min f=o(t^{-2}) \text{a}.\text{s}.$$

Moreover, it holds that

$$X(t)={\displaystyle {\int }_{t_0}^{t}Y(u)}d{\mu }_t(u)+\xi (t).$$

1. Now we prove the first point in the following way:

   $$\begin{array}{ll}{t}^2(f(X(t))-\min f)\hfill & =t^2\left(f(X(t))-f\left({\displaystyle {\int }_{t_0}^{t}Y(u)}d{\mu }_t(u)\right)\right)\hfill \\ \hfill & +t^2\left(f\left({\displaystyle {\int }_{t_0}^{t}Y(u)}d{\mu }_t(u)\right)-\min f\right).\hfill \end{array}$$

   Let us bound the first term using the convexity of f:

   $$\begin{array}{ll}{t}^2\left(f(X(t))-f\left({\displaystyle {\int }_{t_0}^{t}Y(u)}d{\mu }_t(u)\right)\right)\hfill & \le t^2\Vert \nabla f(X(t))\Vert \Vert \xi (t)\Vert \hfill \\ \hfill & \le t^2\Vert \xi (t)\Vert (L\Vert X(t)\Vert +\Vert \nabla f(0)\Vert )\hfill \\ \hfill & \le t^2\Vert \xi (t)\Vert \left(L\underset{t\ge t_0}{\sup }\Vert X(t)\Vert +\Vert \nabla f(0)\Vert \right).\hfill \end{array}$$

   Let us recall that ${\sup }_{s\ge 0}\Vert Z(s)\Vert <+\infty$ a.s. Because of the time scaling and averaging, it is direct to check that ${\sup }_{t\ge t_0}\Vert X(t)\Vert <+\infty$ a.s. On the other hand, $\Vert \xi (t)\Vert =\mathcal{O}(t^{1-\alpha })$ a.s. Therefore, we have

   $$t^2\left(f(X(t))-f\left({\displaystyle {\int }_{t_0}^{t}Y(u)}d{\mu }_t(u)\right)\right)=\mathcal{O}(t^{3-\alpha })\hspace{1em}\text{a}.\text{s}.$$(14)

   Now let us bound the second term using Jensen’s inequality:

   $$\begin{array}{l}{t}^2\left(f\left({\displaystyle {\int }_{t_0}^{t}Y(u)}d{\mu }_t(u)\right)-\min f\right)\le t^2\left({\displaystyle {\int }_{t_0}^t}[f(Y(u))-\min f]d{\mu }_t(u)\right)\\ =\frac{t_0^{\alpha -1}}{t^{\alpha -3}}[f(Y(t_0))-\min f]+\frac{\alpha -1}{t^{\alpha -3}}{\displaystyle {\int }_{t_0}^t{u}^{\alpha -4}}(u^2(f(Y(u))-\min f))du.\end{array}$$

   In order to calculate the limit of this second term, let $a\left(t\right)=(\alpha -1)/t, b\left(u\right)=u^2\left(f\right(Y\left(u\right))-\min f)$; by Lemma 2, we have that

   $$\underset{t\to +\infty }{\lim }\frac{\alpha -1}{t^{\alpha -1}}{\int }_{{\mathit{t}}_0}^t{u}^{\alpha -2}b\left(u\right)du=0\hspace{1em}\text{a}.\text{s}.$$

   Because $\alpha >3$, we also have that

   $$\underset{t\to +\infty }{\lim }\frac{\alpha -3}{t^{\alpha -3}}{\int }_{{\mathit{t}}_0}^t{u}^{\alpha -4}b\left(u\right)du=0\hspace{1em}\text{a}.\text{s}.$$(15)

   Therefore, we conclude that

   $$\underset{t\to +\infty }{\lim }{t}^2\left(f\right(X\left(t\right))-\min f)=0\hspace{1em}\text{a}.\text{s}.$$

2. By Theorem 6, in the case $\gamma \left(t\right)=\alpha /t$, we have that $e^{-A(t)}=t_0^{\alpha -1}{t}^{1-\alpha }$ and $\theta \left(t\right)=(t^2-t_0^2)/\left(2\right(\alpha -1\left)\right)$. On the other hand,

   $$I\left[\frac{1}{\theta }\right](t)=2{(\alpha -1)}^2{t}^{1-\alpha }{\displaystyle {\int }_{t_0}^t\frac{u^{\alpha -2}}{u^2-t_0^2}}=\mathcal{O}(t^{1-\alpha }+t^{-2}).$$

   Because $\alpha >3$, we have that $\mathcal{O}(t^{1-\alpha })$ is also $\mathcal{O}(t^{-2})$, and we conclude that

   $$\mathbb{E}(f(X(t))-\min f)=\mathcal{O}(t^{-2}).$$

3. This point follows directly from Theorem 7 in the case $\gamma (t)=\frac{\alpha }{t}$. □

3.4. Convergence Rate Under Polyak-Łojasiewicz Inequality

In this section, we show a local convergence rate under Polyak-Łojasiewicz inequality. The Polyak-Łojasiewicz property is a special case of the Łojasiewicz property (Łojasiewicz 1963, 1965, 1984) and is commonly used to prove linear convergence of gradient descent algorithms.

Definition 2

(Polyak-Łojasiewicz Inequality). Let $f:\mathbb{H}\to \mathbb{R}$ be a differentiable function with $\mathcal{S}\ne \varnothing$. Then, f satisfies the Polyak-Łojasiewicz (PŁ) inequality on $\mathcal{S}$, if there exists $r>\min f$ and $\mu >0$ such that

$$2\mu (f(x)-\min f)\le {\Vert \nabla f(x)\Vert }^2,\hspace{1em}\forall x\in [\min f<f<r],$$(16)

and we will write $f\in \text{P}{Ł}_{\mu }(\mathcal{S})$.

Theorem 8.

Let $\nu \ge 2$ and consider the dynamic (S-ISIHD) with initial data $X_0,V_0\in {\text{L}}^{\nu }(\mathrm{\Omega };\mathbb{H})$, where f satisfies (${\text{H}}_0^{{}^{\prime }}$), and $\sigma$ satisfies (H_σ). Besides, $f\in \text{P}{Ł}_{\mu }(\mathcal{S})$ and suppose that either $\mathbb{H}$ is finite dimensional or $f\in C^2(\mathbb{H})$. Let also, $\gamma \equiv \sqrt{2\mu }$, $\beta \equiv \mathrm{\Gamma }\equiv 1/\sqrt{2\mu }$, and such that ${\sigma }_{\infty }\in {\text{L}}^2([t_0,+\infty [)$.

Then the solution trajectory $(X,V)\in S_{\mathbb{H}\times \mathbb{H}}^{\nu }[t_0]$ is unique. Moreover, letting $\delta >0$, then there exists ${\widehat{t}}_{\delta }>t_0,{\widehat{s}}_{\delta }>0,K_{\mu ,\delta },C_l,C_f>0$ such that

$$\mathbb{E}(f(X(t))-\min f)\le K_{\mu ,\delta }{e}^{-\frac{\mu }{2}(t-{\widehat{t}}_{\delta })}+\frac{1}{\mu }{l}_{\delta }\left(\frac{t+3{\widehat{t}}_{\delta }-4t_0}{4\mu }\right)+C_f\sqrt{\delta },\hspace{1em}\forall t>{\widehat{t}}_{\delta },$$(17)

where

$$l_{\delta }(s)=\frac{L}{2}{\sigma }_{\infty }^2(s)+C_l\sqrt{\delta }\frac{{\sigma }_{\infty }^2(s)}{2\sqrt{{\displaystyle {\int }_{{\widehat{s}}_{\delta }}^s{\sigma }_{\infty }^2}(u)du}}.$$

Besides, if $f\in \text{P}{Ł}_{\mu }(\mathcal{S})$ holds on the entire space (i.e., $r=+\infty$), then we have that there exists $K_{\mu }>0$ such that

$$\mathbb{E}(f(X(t))-\min f)\le K_{\mu }{e}^{-\frac{\mu }{2}(t-t_0)}+\frac{L}{2\mu }{\sigma }_{\infty }^2\left(\frac{t-t_0}{4\mu }\right),\hspace{1em}\forall t>t_0.$$(18)

Remark 5.

If f is μ-strongly convex, then $f\in \text{P}{Ł}_{\mu }(\mathcal{S})$ holds on the entire space (i.e., $r=+\infty$).

Remark 6.

For a proper discussion on the arbitrarily small (but nonzero) term $C_f\sqrt{\delta }$, we refer to (Maulen-Soto et al. 2024, section 4).

Proof.

Consider the dynamic (S-ISIHD) with $\gamma \equiv c, \beta \equiv \mathrm{\Gamma }\equiv 1/c$, where $c>0$ is a constant that will be fixed later.

Let us also define $\theta \left(t\right)\stackrel{\text{def}}{=}{\int }_{{\mathit{t}}_0}^t\mathrm{\Gamma }\left(u\right)du=(t-t_0)/c$ and $\tilde{\sigma }(s,·)\stackrel{\text{def}}{=}\sigma ({\theta }^{-1}(s),·)\sqrt{\mathrm{\Gamma }({\theta }^{-1}(s))}$. Then ${\sigma }_{\infty }\in {\text{L}}^2([t_0,+\infty [)$ is equivalent to ${\tilde{\sigma }}_{\infty }\in {\text{L}}^2({\mathbb{R}}_{+})$. Now consider the dynamic

$$\{\begin{array}{l}dZ(s)=-\nabla f(Z(s))+\tilde{\sigma }(s,Z(s))dW(s),\hspace{1em}s>0,\hfill \\ Z(0)=X_0+\mathrm{\Gamma }(t_0)V_0.\hfill \end{array}$$(19)

Let $\delta >0$ and apply the result of Maulen-Soto et al. (2024, theorem 4.5(i-b)) (with coefficient $\sqrt{2\mu }$); that is, there exists ${\widehat{s}}_{\delta }>0$ such that for every $\lambda \in ]0,1[$,

$$\begin{array}{ll}\mathbb{E}(f(Z(s))-\min f)\hfill & \le e^{-2\mu (s-{\widehat{s}}_{\delta })}\mathbb{E}(f(Z({\widehat{s}}_{\delta }))-\min f)\hfill \\ \hfill & +e^{-2\mu (1-\lambda )(s-{\widehat{s}}_{\delta })}\left(\frac{L{C}_{\infty }^2}{2}+C_l{C}_{\infty }\sqrt{\delta }\right)\hfill \\ \hfill & +\frac{l_{\delta }({\widehat{s}}_{\delta }+\lambda (s-{\widehat{s}}_{\delta }))}{2\mu }+C_f\sqrt{\delta },\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\forall s>{\widehat{s}}_{\delta },\hfill \end{array}$$(20)

where $C_{\infty },C_l,C_f>0$ and the establishment of $l_{\delta }$ are detailed in Maulen-Soto et al. (2024, section 4.2), in particular $C_{\infty }=\sqrt{{\int }_{t_0}^{+\infty }{\sigma }_{\infty }^2(s)ds}$.

Considering the time scaling $\tau \equiv \theta$, $Y(t)=Z(\theta (t))$ and ${\widehat{t}}_{\delta }>t_0$ such that $\theta ({\widehat{t}}_{\delta })={\widehat{s}}_{\delta }$ (i.e., ${\widehat{t}}_{\delta }=c{\widehat{s}}_{\delta }+t_0$), we have that

$$\begin{array}{ll}\mathbb{E}(f(Y(t))-\min f)\hfill & \le e^{-2\mu (\theta (t)-{\widehat{s}}_{\delta })}\mathbb{E}(f(Y({\widehat{t}}_{\delta }))-\min f)\hfill \\ \hfill & +e^{-2\mu (1-\lambda )(\theta (t)-{\widehat{s}}_{\delta })}\left(\frac{L{C}_{\infty }^2}{2}+C_l{C}_{\infty }\sqrt{\delta }\right)\hfill \\ \hfill & +\frac{l_{\delta }({\widehat{s}}_{\delta }+\lambda (\theta (t)-{\widehat{s}}_{\delta }))}{2\mu }+C_f\sqrt{\delta },\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\forall t>{\widehat{t}}_{\delta }.\hfill \end{array}$$(21)

Let $a(t)=c$ and $A(t)=c(t-{\widehat{t}}_{\delta })$. Now, we consider the averaging as in (8) but change the initial condition to ${\widehat{t}}_{\delta }$. Thus, we have

$$X(t)={\displaystyle {\int }_{{\widehat{t}}_{\delta }}^{t}Y(u)}d{\tilde{\mu }}_t(u)+\tilde{\xi }(t),$$(22)

where ${\tilde{\mu }}_t$ is the probability measure on $[{\widehat{t}}_{\delta },t]$ defined by

$${\tilde{\mu }}_t=e^{-c(t-{\widehat{t}}_{\delta })}{\delta }_{{\widehat{t}}_{\delta }}+c{e}^{c(u-t)}du,$$(23)

where ${\delta }_{{\widehat{t}}_{\delta }}$ is the Dirac measure at ${\widehat{t}}_{\delta }$ and

$$\tilde{\xi }(t)\stackrel{\text{def}}{=}-\frac{1}{c}{e}^{-c(t-{\widehat{t}}_{\delta })}V({\widehat{t}}_{\delta }).$$(24)

Then

$$\mathbb{E}(f(X(t))-\min f)=\mathbb{E}\left(f(X(t))-f\left({\displaystyle {\int }_{{\widehat{t}}_{\delta }}^{t}Y(u)}d{\mu }_t(u)\right)\right)+\mathbb{E}\left(f\left({\displaystyle {\int }_{{\widehat{t}}_{\delta }}^{t}Y(u)}d{\mu }_t(u)\right)-\min f\right).$$

We can bound the first term using convexity and Cauchy-Schwarz inequality in the following way:

$$\begin{array}{ll}\hfill & \mathbb{E}\left(f(X(t))-f\left({\displaystyle {\int }_{{\widehat{t}}_{\delta }}^{t}Y(u)}d{\tilde{\mu }}_t(u)\right)\right)\hfill \\ \hfill & \le \sqrt{\mathbb{E}(\Vert \nabla f(X(t)){\Vert }^2)}\sqrt{\mathbb{E}(\Vert \tilde{\xi }(t){\Vert }^2)}\hfill \\ \hfill & \le \frac{\sqrt{\mathbb{E}(\Vert V({\widehat{t}}_{\delta }){\Vert }^2)}}{c}\sqrt{2\Vert \nabla f(0){\Vert }^2+2L^2\mathbb{E}\left(\underset{t\ge t_0}{\sup }\Vert X(t){\Vert }^2\right)}{e}^{-c(t-{\widehat{t}}_{\delta })},\hfill \end{array}$$

where $\mathbb{E}({\sup }_{t\ge t_0}\Vert X(t){\Vert }^2)<+\infty$ as mentioned in Corollary 1.

On the other hand, we can bound the second term using Jensen’s inequality and then (21):

$$\begin{array}{ll}\hfill & \mathbb{E}\left(f\left({\displaystyle {\int }_{{\widehat{t}}_{\delta }}^{t}Y(u)}d{\tilde{\mu }}_t(u)\right)-\min f\right)\hfill \\ \hfill & \le {\displaystyle {\int }_{{\widehat{t}}_{\delta }}^t\mathbb{E}}(f(Y(u)-\min f))d{\tilde{\mu }}_t(u)\hfill \\ \hfill & \le {\displaystyle {\int }_{{\widehat{t}}_{\delta }}^t{e}^{-2\mu (\theta (u)-{\widehat{s}}_{\delta })}}\mathbb{E}(f(Y({\widehat{t}}_{\delta }))-\min f)d{\tilde{\mu }}_t(u)\hfill \\ \hfill & +{\displaystyle {\int }_{{\widehat{t}}_{\delta }}^t{e}^{-2\mu (1-\lambda )(\theta (u)-{\widehat{s}}_{\delta })}}\left(\frac{L{C}_{\infty }^2}{2}+C_l{C}_{\infty }\sqrt{\delta }\right)d{\tilde{\mu }}_t(u)\hfill \\ \hfill & +{\displaystyle {\int }_{{\widehat{t}}_{\delta }}^t\frac{l_{\delta }({\widehat{s}}_{\delta }+\lambda (\theta (u)-{\widehat{s}}_{\delta }))}{2\mu }}d{\tilde{\mu }}_t(u)+C_f\sqrt{\delta }\hfill \\ \hfill & =\left(\mathbb{E}(f(Y({\widehat{t}}_{\delta }))-\min f)+\left(\frac{L{C}_{\infty }^2}{2}+C_l{C}_{\infty }\sqrt{\delta }\right)+\frac{l_{\delta }({\widehat{s}}_{\delta })}{2\mu }\right)e^{-c(t-{\widehat{t}}_{\delta })}\hfill \\ \hfill & +c\mathbb{E}(f(Y({\widehat{t}}_{\delta }))-\min f)e^{-ct}{\displaystyle {\int }_{{\widehat{t}}_{\delta }}^t{e}^{-2\mu (\theta (u)-{\widehat{s}}_{\delta })}}{e}^{cu}du\hfill \\ \hfill & +c\left(\frac{L{C}_{\infty }^2}{2}+C_l{C}_{\infty }\sqrt{\delta }\right)e^{-ct}{\displaystyle {\int }_{{\widehat{t}}_{\delta }}^t{e}^{-2\mu (1-\lambda )(\theta (u)-{\widehat{s}}_{\delta })}}{e}^{cu}du\hfill \\ \hfill & +\frac{c}{2\mu }{e}^{-ct}{\displaystyle {\int }_{{\widehat{t}}_{\delta }}^t{l}_{\delta }}({\widehat{s}}_{\delta }+\lambda (\theta (u)-{\widehat{s}}_{\delta }))e^{cu}du+C_f\sqrt{\delta },\hspace{1em}\forall t>{\widehat{t}}_{\delta }.\hfill \end{array}$$

We bound the first integral as follows:

$$e^{-ct}{\displaystyle {\int }_{{\widehat{t}}_{\delta }}^t{e}^{-2\mu (\theta (u)-{\widehat{s}}_{\delta })}}{e}^{cu}du\le e^{2\mu \left(\frac{t_0}{c}+{\widehat{s}}_{\delta }\right)}{e}^{-\frac{2\mu }{c}t}.$$

The second integral is bounded in the same way:

$$e^{-ct}{\displaystyle {\int }_{{\widehat{t}}_{\delta }}^t{e}^{-2\mu (1-\lambda )(\theta (u)-{\widehat{s}}_{\delta })}}{e}^{cu}du\le e^{2\mu (1-\lambda )\left(\frac{t_0}{c}+{\widehat{s}}_{\delta }\right)}{e}^{-\frac{2\mu (1-\lambda )}{c}t}.$$

To treat the third integral, we are going to split the integral in two in order to find a useful convergence rate. Let us recall that $l_{\delta }\in {\text{L}}^1([{\widehat{s}}_{\delta },+\infty [)$ and that $l_{\delta }$ is decreasing. Let us define ${\phi }_{\lambda ,c,\delta }(t)\stackrel{\text{def}}{=}{\widehat{s}}_{\delta }+\lambda \left(\frac{t+{\widehat{t}}_{\delta }-2t_0}{2c}-{\widehat{s}}_{\delta }\right)$, then

$$\begin{array}{ll}{e}^{-ct}{\displaystyle {\int }_{{\widehat{t}}_{\delta }}^t{l}_{\delta }}({\widehat{s}}_{\delta }+\lambda (\theta (u)-{\widehat{s}}_{\delta }))e^{cu}du\hfill & =e^{-ct}{\displaystyle {\int }_{{\widehat{t}}_{\delta }}^{\frac{{\widehat{t}}_{\delta }+t}{2}}{l}_{\delta }}({\widehat{s}}_{\delta }+\lambda (\theta (u)-{\widehat{s}}_{\delta }))e^{cu}du\hfill \\ \hfill & +e^{-ct}{\displaystyle {\int }_{\frac{{\widehat{t}}_{\delta }+t}{2}}^t{l}_{\delta }}({\widehat{s}}_{\delta }+\lambda (\theta (u)-{\widehat{s}}_{\delta }))e^{cu}du\hfill \\ \hfill & \le \frac{c}{\lambda }{e}^{\frac{c{\widehat{t}}_{\delta }}{2}}{C}_{\infty }{e}^{-\frac{ct}{2}}+l_{\delta }({\phi }_{\lambda ,c,\delta }(t)).\hfill \end{array}$$

Now that we have bounded all the terms, we have the following bound:

$$\begin{array}{ll}\mathbb{E}(f(X(t))-\min f)\hfill & \le \frac{\sqrt{\mathbb{E}(\Vert V({\widehat{t}}_{\delta }){\Vert }^2)}}{c}\sqrt{2\Vert \nabla f(0){\Vert }^2+2L^2\mathbb{E}\left(\underset{t\ge t_0}{\sup }\Vert X(t){\Vert }^2\right)}{e}^{-c(t-{\widehat{t}}_{\delta })}\hfill \\ \hfill & +\left(\mathbb{E}(f(Y({\widehat{t}}_{\delta }))-\min f)+\left(\frac{L{C}_{\infty }^2}{2}+C_l{C}_{\infty }\sqrt{\delta }\right)+\frac{l_{\delta }({\widehat{s}}_{\delta })}{2\mu }\right)e^{-c(t-{\widehat{t}}_{\delta })}\hfill \\ \hfill & +c\mathbb{E}(f(Y({\widehat{t}}_{\delta }))-\min f)e^{2\mu \left(\frac{t_0+{\widehat{t}}_{\delta }}{c}+{\widehat{s}}_{\delta }\right)}{e}^{-\frac{2\mu }{c}(t-{\widehat{t}}_{\delta })}\hfill \\ \hfill & +c\left(\frac{L{C}_{\infty }^2}{2}+C_l{C}_{\infty }\sqrt{\delta }\right)e^{2\mu (1-\lambda )\left(\frac{t_0+{\widehat{t}}_{\delta }}{c}+{\widehat{s}}_{\delta }\right)}{e}^{-\frac{2\mu (1-\lambda )}{c}(t-{\widehat{t}}_{\delta })}\hfill \\ \hfill & +\frac{c}{2\mu }\left(\frac{c}{\lambda }{C}_{\infty }{e}^{-\frac{c(t-{\widehat{t}}_{\delta })}{2}}+l_{\delta }({\phi }_{\lambda ,c,\delta }(t))\right)+C_f\sqrt{\delta },\hspace{1em}\forall t>{\widehat{t}}_{\delta }.\hfill \end{array}$$

Letting $\lambda =1/2$ and $c=\sqrt{2\mu }$, we obtain

$$\begin{array}{ll}\mathbb{E}(f(X(t))-\min f)\hfill & \le \frac{\sqrt{\mathbb{E}(\Vert V({\widehat{t}}_{\delta }){\Vert }^2)}}{\sqrt{2\mu }}\sqrt{2\Vert \nabla f(0){\Vert }^2+2L^2\mathbb{E}\left(\underset{t\ge t_0}{\sup }\Vert X(t){\Vert }^2\right)}{e}^{-\sqrt{2\mu }(t-{\widehat{t}}_{\delta })}\hfill \\ \hfill & +\left(\mathbb{E}(f(Y({\widehat{t}}_{\delta }))-\min f)+\left(\frac{L{C}_{\infty }^2}{2}+C_l{C}_{\infty }\sqrt{\delta }\right)+\frac{l_{\delta }({\widehat{s}}_{\delta })}{2\mu }\right)e^{-\sqrt{2\mu }(t-{\widehat{t}}_{\delta })}\hfill \\ \hfill & +\sqrt{2\mu }\mathbb{E}(f(Y({\widehat{t}}_{\delta }))-\min f)e^{2\mu \left(\frac{t_0+{\widehat{t}}_{\delta }}{\sqrt{2\mu }}+{\widehat{s}}_{\delta }\right)}{e}^{-\sqrt{2\mu }(t-{\widehat{t}}_{\delta })}\hfill \\ \hfill & +\sqrt{2\mu }\left(\frac{L{C}_{\infty }^2}{2}+C_l{C}_{\infty }\sqrt{\delta }\right)e^{\mu \left(\frac{t_0+{\widehat{t}}_{\delta }}{\sqrt{2\mu }}+{\widehat{s}}_{\delta }\right)}{e}^{-\frac{\sqrt{2\mu }}{2}(t-{\widehat{t}}_{\delta })}\hfill \\ \hfill & +2C_{\infty }{e}^{-\frac{\sqrt{2\mu }(t-{\widehat{t}}_{\delta })}{2}}+\frac{1}{\sqrt{2\mu }}{l}_{\delta }({\phi }_{\frac{1}{2},\sqrt{2\mu },\delta }(t))+C_f\sqrt{\delta },\hspace{1em}\forall t>{\widehat{t}}_{\delta }.\hfill \end{array}$$

Letting $K_{\mu ,\delta }\stackrel{\text{def}}{=}\sqrt{2\mu }\left(\frac{L{C}_{\infty }^2}{2}+C_l{C}_{\infty }\sqrt{\delta }\right)e^{\mu \left(\frac{t_0+{\widehat{t}}_{\delta }}{\sqrt{2\mu }}+{\widehat{s}}_{\delta }\right)}+2C_{\infty }$, we conclude that

$$\mathbb{E}(f(X(t))-\min f)\le K_{\mu ,\delta }{e}^{-\frac{\sqrt{2\mu }}{2}(t-{\widehat{t}}_{\delta })}+\frac{1}{\sqrt{2\mu }}{l}_{\delta }({\phi }_{\frac{1}{2},\sqrt{2\mu },\delta }(t))+C_f\sqrt{\delta },\hspace{1em}\forall t>{\widehat{t}}_{\delta }.\hspace{1em}\square$$(25)

4. From Weak to Strong Convergence Under General $\gamma$ and $\beta \equiv \mathrm{\Gamma }$

4.1. General Result

We consider the Tikhonov regularization of the dynamic (S-ISIHD_NS), that is, for $t>0$,

We show some conditions (on $\gamma ,\beta ,\epsilon$) under which we can obtain strong convergence of the trajectory.

Theorem 9.

Consider that $\gamma :[t_0,+\infty [\to {\mathbb{R}}_{+}$ satisfies (Hγ). Besides, $F=f+g$ and $\sigma$ satisfy Assumptions (H₀) and (H_σ). Moreover, suppose that g satisfies (H_λ) and let $\nu \ge 2$. Consider (S-ISIHD_NS-TA) with $\beta \equiv \mathrm{\Gamma }$ and initial data $X_0,V_0\in {\text{L}}^{\nu }(\mathrm{\Omega };\mathbb{H})$.

Then, there exists a unique solution $(X,V)\in S_{\mathbb{H}\times \mathbb{H}}^{\nu }[t_0]$ of (S-ISIHD_NS-TA). Additionally, let $x^{\star }\stackrel{\text{def}}{=}{\text{P}}_{{\mathcal{S}}_F}(0)$ be the minimum norm solution, and for $\epsilon >0$, let $x_{\epsilon }$ be the unique minimizer of $F_{\epsilon }\left(x\right)\stackrel{\text{def}}{=}F\left(x\right)+\epsilon \Vert x{\Vert }^2/2$. If we suppose that $\mathrm{\Gamma }{\sigma }_{\infty }\in {\text{L}}^2([t_0,+\infty [)$, and that $\epsilon :[t_0,+\infty [\to {\mathbb{R}}_{+}$ satisfies the conditions:

($T_1^{\prime }$) $\epsilon (t)\to 0$ as $t\to +\infty$;

($T_2^{\prime }$) ${\int }_{t_0}^{+\infty }\epsilon (t)\mathrm{\Gamma }(t)dt=+\infty$; and

($T_3^{\prime }$) ${\int }_{t_0}^{+\infty }\epsilon (t)\mathrm{\Gamma }(t)(\Vert x^{\star }{\Vert }^2-\Vert x_{\epsilon (t)}{\Vert }^2)dt<+\infty .$

Then ${\text{s}-\text{lim}}_{t\to +\infty }X(t)=x^{\star }$ a.s., and $V(t)=o\left(\frac{1}{\mathrm{\Gamma }(t)}\right)$ a.s.

Proof.

Let $s_0>0$, $\theta (t)\stackrel{\text{def}}{=}{s}_0+{\int }_{t_0}^t\mathrm{\Gamma }(u)du$; $\tilde{\epsilon }(t)=\epsilon ({\theta }^{-1}(t))$; and $\tilde{\sigma }(s,·)\stackrel{\text{def}}{=}\sigma ({\theta }^{-1}(s),·)\sqrt{\mathrm{\Gamma }({\theta }^{-1}(s))}$. Then $\epsilon$ satisfying ($T_1^{\prime }$),($T_2^{\prime }$), and ($T_3^{\prime }$) is equivalent to $\tilde{\epsilon }$ satisfying ($T_1$), ($T_2$), and ($T_3$) defined in Theorem 4. Besides, $\mathrm{\Gamma }{\sigma }_{\infty }\in {\text{L}}^2([t_0,+\infty [)$ is equivalent to ${\tilde{\sigma }}_{\infty }\in {\text{L}}^2({\mathbb{R}}_{+})$. Consider the dynamic:

$$\{\begin{array}{l}dZ(s)\in -\partial F(Z(s))-\tilde{\epsilon }(s)Z(s)+\tilde{\sigma }(s,Z(s))dW(s),\hspace{1em}s>s_0,\hfill \\ Z(s_0)=X_0+\mathrm{\Gamma }(t_0)V_0.\hfill & \hfill \end{array}$$(26)

By Theorem 4, we have that there exists a unique solution $Z\in S_{\mathbb{H}}^{\nu }[s_0]$, and that ${\lim }_{s\to +\infty }Z\left(s\right)=x^{\star }$ a.s. (recall that $x^{\star }\stackrel{\text{def}}{=}{\text{P}}_{S_F}(0)$). Using the time scaling $\tau \equiv \theta$ and the averaging described at the beginning of this section, we end up with the dynamic (S-ISIHD_NS-TA) in the case where $\beta \equiv \mathrm{\Gamma }$. The existence and uniqueness of solution, and the fact that $(X,V)\in S_{\mathbb{H}\times \mathbb{H}}^{\nu }[t_0]$ goes analogously as in the proof of Theorem 5.

Now we prove the claim: Because ${\lim }_{s\to +\infty }Z\left(s\right)=x^{\star }$ a.s., this implies directly that ${\lim }_{t\to +\infty }Y\left(t\right)=x^{\star }$ a.s. Besides, we have Relation (9), that is,

$$X(t)={\displaystyle {\int }_{t_0}^{t}Y(u)}d{\mu }_t(u)+\xi (t),$$

where ${\mu }_t$ and $\xi$ are defined in (10) and (11), respectively. Consequently, we have

$$\begin{array}{ll}\Vert X(t)-x^{\star }\Vert \hfill & \le \Vert X(t)-{\displaystyle {\int }_{t_0}^{t}Y(u)}d{\mu }_t(u)\Vert +\Vert {\displaystyle {\int }_{t_0}^{t}Y(u)}d{\mu }_t(u)-x^{\star }\Vert \hfill \\ \hfill & \le \Vert \xi (t)\Vert +\Vert {\displaystyle {\int }_{t_0}^{t}Y(u)}d{\mu }_t(u)-x^{\star }\Vert .\hfill \end{array}$$

Let $a\left(t\right)=1/\mathrm{\Gamma }\left(t\right)$ and $A\left(t\right)={\int }_{{\mathit{t}}_0}^{t}du/\mathrm{\Gamma }\left(u\right)$. By Lemma 3, we have that ${\lim }_{t\to +\infty }\Vert \xi \left(t\right)\Vert =0$. On the other hand,

$$\begin{array}{ll}\Vert {\displaystyle {\int }_{t_0}^{t}Y(u)}d{\mu }_t(u)-x^{\star }\Vert \hfill & =\Vert {\displaystyle {\int }_{t_0}^t(Y(u)-x^{\star })}d{\mu }_t(u)\Vert \hfill \\ \hfill & \le {\displaystyle {\int }_{t_0}^t\Vert Y(u)-x^{\star }\Vert }d{\mu }_t(u)\hfill \\ \hfill & =e^{-A(t)}\Vert Y(t_0)-x^{\star }\Vert +e^{-A(t)}{\displaystyle {\int }_{t_0}^{t}a}(u)e^{A(u)}\Vert Y(u)-x^{\star }\Vert du.\hfill \end{array}$$

Let $b(u)=\Vert Y(u)-x^{\star }\Vert$. Because we already proved that ${\lim }_{u\to +\infty }b\left(u\right)=0$ a.s., and we have that $a\notin {\text{L}}^1([t_0,+\infty [)$ by Lemma 3, we utilize Lemma 2 with our respective a, b functions. This let us conclude that

$$\underset{t\to +\infty }{\lim }\Vert {\int }_{t_0}^{t}Y\left(u\right)d{\mu }_t\left(u\right)-x^{\star }\Vert =0\hspace{1em}\text{a}.\text{s}.$$

Thus, ${\lim }_{t\to +\infty }X\left(t\right)=x^{\star }$ a.s. Finally, because

$$Y(t)=X(t)+\mathrm{\Gamma }(t)V(t),$$

and the fact that $X$ and $Y$ have (a.s.) the same limit, we conclude that

$$\underset{t\to +\infty }{\lim }\mathrm{\Gamma }\left(t\right)V\left(t\right)=0\hspace{1em}\text{a}.\text{s}.\hspace{1em}\square$$

4.2. Concrete Cases

In order to give some conditions when ($T_1^{\prime }$), ($T_2^{\prime }$), and ($T_3^{\prime }$) of Theorem 9 are satisfied, we need to introduce the following definition.

Definition 3

(Hölderian Error Bound). Let $f:\mathbb{H}\to \mathbb{R}$ be a proper function such that $\mathcal{S}\ne \varnothing$; f satisfies a Hölderian (or power-type) error bound inequality on $\mathcal{S}$ with exponent $p\ge 1$, if there exists $\kappa >0$ and $r>\min f$ such that

$$f(x)-\min f\ge \kappa \text{dist}{(x,\mathcal{S})}^p,\hspace{1em}\forall x\in [\min f\le f\le r],$$(27)

and we will write $f\in {\text{EB}}^p(\mathcal{S})$.

Remark 7.

Let $f:\mathbb{H}\to \mathbb{R}$ be a differentiable function such that $\mathcal{S}\ne \varnothing$. If f satisfies the P-Ł inequality on $\mathcal{S}$, then f satisfies a Hölderian error bound inequality with exponent $p=2$.

Theorem 10.

Consider the setting of Theorem 9 and suppose that $F=f+g\in {\text{EB}}^p({\mathcal{S}}_F)$. Let $s_0>0$ and denote $\theta (t)\stackrel{\text{def}}{=}{s}_0+{\int }_{t_0}^t\mathrm{\Gamma }(s)ds$, then taking the Tikhonov parameter $\epsilon \left(t\right)=1/{\theta }^r\left(t\right)$ with

$$1\ge r>\frac{2p}{2p+1},$$

the three conditions ($T_1^{\prime }$), ($T_2^{\prime }$), and ($T_3^{\prime }$) of Theorem 9 are satisfied simultaneously. In particular, for any solution $(X,V)\in S_{\mathbb{H}\times \mathbb{H}}^{\nu }[t_0]$ of (9), we get almost sure (strong) convergence of $X(t)$ to the minimal norm solution named $x^{\star }=P_{{\mathcal{S}}_F}(0)$ and that $V\left(t\right)=o(1/\mathrm{\Gamma }(t\left)\right)$.

Proof.

We proceed as in the proof of Theorem 9 and arrive to the dynamic (26); because $\tilde{\epsilon }\left(t\right)=\epsilon \left({\theta }^{-1}\right(t\left)\right)=1/t^r$, the proof goes as in Maulen-Soto et al. (2025, theorem 4.8). □

Theorem 11.

Let $\nu \ge 2$, $f\in {\mathrm{\Gamma }}_0(\mathbb{H})\cap C_L^2(\mathbb{H})$ such that $\mathcal{S}$ is nonempty, and also $f\in {\text{EB}}^p(\mathcal{S})$, $\sigma$ satisfying (H_σ), and $\mathrm{\Gamma }{\sigma }_{\infty }\in {\text{L}}^2([t_0,+\infty [)$ and is nonincreasing. Let us consider $\epsilon \left(t\right)=1/t^r$, where $0<r<1$, then we evaluate (S-ISIHD_NS-TA) in the case where $\gamma$ satisfies (H_γ), $g\equiv 0$, $\beta \equiv \mathrm{\Gamma }$, and with initial data $X_0,V_0\in {\text{L}}^{\nu }(\mathrm{\Omega };\mathbb{H})$, that is, for $t>t_0$,

$$\{\begin{array}{l}dX(t)=V(t)dt,\hfill \\ dV(t)=-\gamma (t)V(t)dt-\nabla f(X(t)+\mathrm{\Gamma }(t)V(t))dt-\epsilon (t)(X(t)+\mathrm{\Gamma }(t)V(t))\hfill \\ +\sigma (t,X(t)+\mathrm{\Gamma }(t)V(t))dW(t),\hfill \\ X(t_0)=X_0,\hspace{1em}V(t_0)=V_0.\hfill \end{array}$$(28)

For $\epsilon >0$, let us define $f_{\epsilon }\left(x\right)\stackrel{\text{def}}{=}f\left(x\right)+\epsilon \Vert x{\Vert }^2/2$, and let $x_{\epsilon }$ be the unique minimizer of $f_{\epsilon }$. Moreover, let $s_0>0$ and for $s_1>s_0$ consider

$$R(s)\stackrel{\text{def}}{=}{e}^{-\frac{s^{1-r}}{1-r}}{\displaystyle {\int }_{s_1}^s{e}^{\frac{u^{1-r}}{1-r}}}{\sigma }_{\infty }^2({\theta }^{-1}(u))\mathrm{\Gamma }({\theta }^{-1}(u))du,$$(29)

where $\theta (t)\stackrel{\text{def}}{=}{s}_0+{\int }_{t_0}^t\mathrm{\Gamma }(u)du$. Let also $x^{\star }\stackrel{\text{def}}{=}{\text{P}}_{\mathcal{S}}(0)$, $A\left(s\right)\stackrel{\text{def}}{=}{\int }_{s_1}^{s}du/\mathrm{\Gamma }\left(u\right)$, and $t_1\stackrel{\text{def}}{=}{\theta }^{-1}(s_1)$. Then, the solution trajectory $(X,V)\in S_{\mathbb{H}\times \mathbb{H}}^{\nu }[t_0]$ is unique, and we have that

1. $R(\theta (t))\to 0 \text{as} \hspace{0.17em}t\to +\infty .$

2. Let $\overline{\sigma }(t)=\mathrm{\Gamma }(t){\sigma }_{\infty }^2(t)$, then

   $$R(\theta (t))=\mathcal{O}\left(\text{exp}(-{\theta }^r(t)(1-2^{-r}))+{\theta }^r(t)\overline{\sigma }\left(\frac{s_1+\theta (t)}{2}\right)\right).$$

   Moreover, if $\overline{\sigma }(t)=\mathcal{O}({\theta }^{-\mathrm{\Delta }}(t))$ for $\mathrm{\Delta }>1$, then $R(\theta (t))=\mathcal{O}({\theta }^{r-\mathrm{\Delta }}(t))$.

   Besides, we have the following convergence rate in expectation:

3. For the values, we have

   $$\mathbb{E}[f(X(t))-\min (f)]=\mathcal{O}(\max \{e^{-A(t)},I[h_1](t)\}),$$

   where $h_1\left(t\right)=1/{\theta }^r\left(t\right)+R\left(\theta \right(t\left)\right)$.

4. Also, for the trajectory, we obtain

   $$\mathbb{E}[\Vert X(t)-x^{\star }|{|}^2]=\mathcal{O}(\max \{e^{-A(t)},I[h_2](t)\}),$$

where $h_2(t)={\theta }^{r-1}(t)+{\theta }^{-\frac{r}{p}}(t)+{\theta }^r(t)R(\theta (t))$.

Proof.

We proceed as in the proof of Theorem 5 and define analogously $\tilde{\sigma }$, $\tilde{\epsilon }$; we also consider the dynamic (12). By Maulen-Soto et al. (2025, theorem 4.11) we obtain that

$$R(s)=e^{-\frac{s^{1-r}}{1-r}}{\displaystyle {\int }_{s_1}^s{e}^{\frac{u^{1-r}}{1-r}}}{\tilde{\sigma }}_{\infty }^2(u)du,$$

where ${\tilde{\sigma }}_{\infty }^2\in {\text{L}}^2([s_0,+\infty [)$, satisfies the following:

• $R(s)\to +\infty$ as $t\to +\infty$.

• $R(s)=\mathcal{O}\left(\text{exp}(-s^r(1-2^{-r}))+s^r{\tilde{\sigma }}_{\infty }^2\left(\frac{s_1+s}{2}\right)\right)$. Moreover if ${\tilde{\sigma }}_{\infty }^2(s)=\mathcal{O}(s^{-\mathrm{\Delta }})$ for $\mathrm{\Delta }>1$, then $R(s)=\mathcal{O}(s^{r-\mathrm{\Delta }})$.

Also, evaluating at $s=\theta (t)$, we obtain the first two items of the theorem. For the third and fourth items, we used that

• $\mathbb{E}[f(Z(s))-\min (f)]=\mathcal{O}\left(\frac{1}{s^r}+R(s)\right)$.

• $\mathbb{E}[\Vert Z(s)-x^{\star }|{|}^2]=\mathcal{O}\left(\frac{1}{s^{1-r}}+\frac{1}{s^{\frac{r}{p}}}+s^{r}R(s)\right).$

Then, proceeding as in the proof of Theorem 6, we obtain the desired results. □

Corollary 2.

Consider Theorem 10 in the case where $\gamma \left(t\right)=\alpha /t$ for $\alpha >1$, $\beta \left(t\right)=t/(\alpha -1)$; then, we have that

1. If ${\sigma }_{\infty }^2(t)=\mathcal{O}(t^{-2(\mathrm{\Delta }+1)})$ for $\mathrm{\Delta }>1$, and $\alpha \ne \{1+2r,1+2(\mathrm{\Delta }-r)\}$, then

   $$\mathbb{E}[f(X(t))-\min (f)]=\mathcal{O}(\max \{t^{-(\alpha -1)},t^{-2r},t^{-2(\mathrm{\Delta }-r)}\}).$$

   In particular, if $\alpha >3$,

2. If ${\sigma }_{\infty }^2(t)=\mathcal{O}(t^{-2(\mathrm{\Delta }+1)})$ for $\mathrm{\Delta }>\max \{1,2r\}$, and $\alpha \ne \{3-2r,1+2r/p,1+2(2r-\mathrm{\Delta }\left)\right\}$, then

   $$\mathbb{E}[\Vert X(t)-x^{\star }|{|}^2]=\mathcal{O}(\max \{t^{-(\alpha -1)},t^{-2(1-r)},t^{-\frac{2r}{p}},t^{-2(2r-\mathrm{\Delta })}\}).$$

5. Conclusion

This work uncovers the global and local convergence properties of trajectories of the inertial system with implicit Hessian-driven damping under stochastic errors both in the smooth and nonsmooth setting. The aim is to solve convex optimization problems with noisy gradient input with vanishing variance. We have shed light on these properties and provided a comprehensive local and global complexity analysis both in the case where the Hessian damping parameter $\beta$ was dependent on the geometric damping $\gamma$ and when it was zero. We believe that this work, along with the technique of time scaling and averaging, paves the way for important extensions and research avenues. Among them, we mention extension to the situation where the drift term is a nonpotential cocoercive operator.