https://dudusoar.github.io/tags/convergence/Recent content in Convergence on Yuchen DuHugo -- 0.164.0en-usFri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/01-learning-from-a-noisy-target/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/01-learning-from-a-noisy-target/Stochastic approximation begins when the true target is hidden and each update can use only a noisy estimate of it.https://dudusoar.github.io/posts/stochastic-approximation/02-model-error-as-distance-to-target/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/02-model-error-as-distance-to-target/Why convergence should track distance to the true target rather than the size of the latest update.https://dudusoar.github.io/posts/stochastic-approximation/03-what-convergence-requires/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/03-what-convergence-requires/A compact map of the step-size and noise assumptions behind stochastic-approximation convergence.https://dudusoar.github.io/posts/stochastic-approximation/12-robbins-monro-conditions/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/12-robbins-monro-conditions/The step sizes must remain large enough to keep correcting error and become small enough to suppress accumulated noise.https://dudusoar.github.io/posts/stochastic-approximation/13-why-step-sizes-must-sum-to-infinity/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/13-why-step-sizes-must-sum-to-infinity/A finite total step size can make an iteration stop moving before its error has disappeared.https://dudusoar.github.io/posts/stochastic-approximation/18-expanding-the-error-recursion/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/18-expanding-the-error-recursion/The explicit error formula separates the residual initial condition from the weighted sum of all past disturbances.https://dudusoar.github.io/posts/stochastic-approximation/20-how-noise-variance-decays/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/20-how-noise-variance-decays/Each disturbance is controlled twice: by its entry coefficient and by every contraction that follows it.https://dudusoar.github.io/posts/stochastic-approximation/22-two-proof-routes-to-dvoretzky-convergence/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/22-two-proof-routes-to-dvoretzky-convergence/Expectation and variance explain distributional concentration; the energy and martingale route proves almost-sure convergence.