Why the Step Size Must Stay Below One under Noise
Series: Stochastic Approximation from First Principles · Part 5 of 22 Why should $\alpha_k$ lie between zero and one? The answer has two parts: geometry explains overshooting, and the error recursion explains noise amplification. Geometry Without Noise If $\eta_k=0$, then: $$ w_{k+1}=(1-\alpha_k)w_k+\alpha_k w^*. $$The result depends on the value of $\alpha_k$: Step size Position of $w_{k+1}$ Behavior $\alpha_k\in(0,1)$ Between $w_k$ and $w^*$ Moves steadily toward the target $\alpha_k=1$ Exactly at $w^*$ Reaches the target in one step $\alpha_k>1$ Beyond $w^*$ Overshoots and may oscillate In the river analogy, $\alpha_k>1$ means walking past the estimated river location. With no noise, that merely overshoots the river. With noise, it overshoots a location that may already be wrong. ...