Series: Stochastic Approximation from First Principles · Part 11 of 22
If $\alpha=1$, the traveler moves directly to the latest inferred location:
$$ w_{k+1}=\text{target}_k. $$The analyst can then substitute $\text{target}_k=w^*+\eta_k$:
$$ w_{k+1}=w^*+\eta_k. $$For $w_k$ to converge to $w^*$, the disturbance itself must converge to zero:
$$ \eta_k\to0. $$For mean-square convergence, this requires:
$$ \mathbb{E}[\eta_k^2]\to0. $$If the errors are mean zero, this reduces to $\operatorname{Var}(\eta_k)\to0$.
By contrast, a history-preserving mechanism can work with persistent noise whose variance remains finite:
$$ \operatorname{Var}(\eta_k)=\sigma^2<\infty. $$One design demands that noise disappear. The other permits noise to remain and limits how much of it enters each update.
Compare the Modeling Choices
| Design | What is controlled | What is allowed | Practical implication |
|---|---|---|---|
| Standard stochastic approximation | The step-size sequence $\alpha_k$ | Persistent finite-variance noise | Usually realistic |
| Keep $\alpha=1$ | The noise must approach zero | Full replacement at every step | Strong environmental requirement |
| Ideal limit | $\eta_k\equiv0$ | $\alpha=1$ | Noise-free abstraction |
In the forest analogy, insisting on $\alpha=1$ means that every later sound-based judgment must become almost perfectly accurate. Wind, leaves, and echoes would have to stop affecting the inferred location. A more realistic strategy accepts that errors will persist and reduces how strongly each one changes the route.
Suppose the next judgment can still miss the river by $20$ meters no matter how many times the traveler has listened before. Another iteration does not make that raw judgment exact. With $\alpha=1$, the traveler simply jumps to the latest noisy location, so the position error remains as noisy as the judgment error.
Control the Mechanism, Not the Environment
This comparison reveals a broader modeling principle:
Place weaker assumptions on factors you cannot control, then design the controllable mechanism to tolerate them.
Noise belongs to the environment. The step-size rule belongs to the algorithm. Demanding a perfect environment so that the mechanism can remain unconstrained reverses those roles.
Controlling $\alpha_k$ buys the ability to converge under persistent uncertainty.