Variance on Yuchen Duhttps://dudusoar.github.io/tags/variance/Recent content in Variance on Yuchen DuHugo -- 0.164.0en-usFri, 17 Jul 2026 00:00:00 +0000Why Alpha Cannot Equal One Under Persistent Noisehttps://dudusoar.github.io/posts/stochastic-approximation/06-why-alpha-cannot-equal-one/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/06-why-alpha-cannot-equal-one/With alpha equal to one, every update discards history and inherits the full error of the latest noisy target.Expectation and Variance Explain Why History Stabilizes an Updatehttps://dudusoar.github.io/posts/stochastic-approximation/09-expectation-and-variance-of-two-updates/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/09-expectation-and-variance-of-two-updates/A direct derivation compares memoryless and history-preserving updates under zero-mean finite-variance noise.What the Noise Assumptions Actually Buy Ushttps://dudusoar.github.io/posts/stochastic-approximation/10-what-noise-assumptions-buy-us/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/10-what-noise-assumptions-buy-us/Zero mean, finite variance, independence, and identical distribution each support a different part of noise averaging.Why the Squared Step Sizes Must Be Summablehttps://dudusoar.github.io/posts/stochastic-approximation/14-why-squared-step-sizes-must-be-summable/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/14-why-squared-step-sizes-must-be-summable/For a changing step-size sequence, square summability bounds the total variance injected by noisy observations.A Second Proof Route Starts from Expectation and Variancehttps://dudusoar.github.io/posts/stochastic-approximation/17-expectation-variance-proof-route/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/17-expectation-variance-proof-route/Expanding the recursion explains how initial error and historical noise decay, but usually yields convergence in probability rather than almost-sure convergence.How Historical Noise Is Attenuatedhttps://dudusoar.github.io/posts/stochastic-approximation/20-how-noise-variance-decays/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/20-how-noise-variance-decays/Each disturbance is controlled twice: by its entry coefficient and by every contraction that follows it.