https://dudusoar.github.io/tags/step-size/Recent content in Step Size on Yuchen DuHugo -- 0.164.0en-usFri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/03-what-convergence-requires/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/03-what-convergence-requires/A compact map of the step-size and noise assumptions behind stochastic-approximation convergence.https://dudusoar.github.io/posts/stochastic-approximation/04-what-the-step-size-means/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/04-what-the-step-size-means/The step size is simultaneously a movement ratio, a trust coefficient, and a weight on new information.https://dudusoar.github.io/posts/stochastic-approximation/05-why-the-step-size-stays-below-one/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/05-why-the-step-size-stays-below-one/Step sizes between zero and one are the only range that combines monotone error contraction with attenuation of new noise.https://dudusoar.github.io/posts/stochastic-approximation/06-why-alpha-cannot-equal-one/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/06-why-alpha-cannot-equal-one/With alpha equal to one, every update discards history and inherits the full error of the latest noisy target.https://dudusoar.github.io/posts/stochastic-approximation/07-noiseless-versus-noisy-updates/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/07-noiseless-versus-noisy-updates/Alpha equal to one is optimal for an exact target but accepts the full disturbance when the target is noisy.https://dudusoar.github.io/posts/stochastic-approximation/11-alpha-one-requires-vanishing-noise/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/11-alpha-one-requires-vanishing-noise/If every update fully replaces the current position, convergence requires the river-location judgment error itself to disappear.https://dudusoar.github.io/posts/stochastic-approximation/12-robbins-monro-conditions/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/12-robbins-monro-conditions/The step sizes must remain large enough to keep correcting error and become small enough to suppress accumulated noise.https://dudusoar.github.io/posts/stochastic-approximation/13-why-step-sizes-must-sum-to-infinity/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/13-why-step-sizes-must-sum-to-infinity/A finite total step size can make an iteration stop moving before its error has disappeared.https://dudusoar.github.io/posts/stochastic-approximation/14-why-squared-step-sizes-must-be-summable/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/14-why-squared-step-sizes-must-be-summable/For a changing step-size sequence, square summability bounds the total variance injected by noisy observations.