A Memoryless Update Is Not the Same as Markov Sufficiency
Series: Stochastic Approximation from First Principles · Part 8 of 22 When $\alpha=1$, the update $$ w_{k+1}=w^*+\eta_k $$looks memoryless. That resemblance can be confused with the Markov property, but the two ideas are not equivalent. A Memoryless Update Fully Trusts the Current Observation At $\alpha_k=1$: $$ w_{k+1}=\text{target}_k=w^*+\eta_k. $$The update: ignores the previous state $w_k$; gives the previous position zero weight in the new position; depends only on the current noisy target. In the river analogy, you completely trust the latest sound and move directly to the location it suggests. The difficulty is that $\text{target}_k$ is not the river. It is an estimate contaminated by location error. ...