Series: Stochastic Approximation from First Principles · Part 4 of 22

In the update

$$ w_{k+1}=w_k+\alpha_k(\text{target}_k-w_k), $$

what does $\alpha_k$ actually represent?

Separate Direction from Distance

Treat the update as a vector movement:

$$ w_{k+1} =w_k+ \underbrace{\alpha_k\|\text{target}_k-w_k\|}_{\text{actual distance moved}} \cdot \underbrace{ \frac{\text{target}_k-w_k} {\|\text{target}_k-w_k\|} }_{\text{unit direction}}. $$

The vector $\text{target}_k-w_k$ contains both a direction and a distance:

  • $\|\text{target}_k-w_k\|$ is the distance suggested by the latest observation.
  • $\alpha_k\|\text{target}_k-w_k\|$ is the distance the update actually moves.
  • $\alpha_k$ is the fraction of the suggested movement that the update accepts.

In the river analogy, you walk an $\alpha_k$ fraction of the estimated distance toward the location implied by the current sound. The step size is therefore both a movement ratio and a degree of trust in the latest estimate.

Why the Direction Is Not Normalized

A natural alternative is to normalize $\text{target}_k-w_k$ into a unit vector and then multiply it by $\alpha_k$. That would make every step have length $\alpha_k$, regardless of how far the estimate is from the target.

Normalization discards useful distance information. Without normalization:

  • Far from the estimated target, the update naturally takes a larger step.
  • Near the estimated target, it naturally takes a smaller step.

The distance already provides free adaptation. Normalizing the vector would force $\alpha_k$ to control the entire movement length by itself.

The Algebra Also Uses the Full Difference

Convergence analysis tracks:

$$ \Delta_{k+1}=(1-\alpha_k)\Delta_k+\alpha_k\eta_k. $$

This recursion uses the actual magnitude of the target difference to determine contraction. A normalized direction would change that algebra and require a different step-size design.

The coefficient $\alpha_k$ therefore has three related meanings:

  1. the fraction of the proposed movement taken;
  2. the degree of trust placed in the latest observation;
  3. the weight assigned to the latest inferred location relative to the current position.

Keeping the target difference unnormalized lets the update use the information it already has instead of throwing distance away.

A Concrete Movement

Suppose you are at 0 meters and the latest sound suggests that the riverbank is at 120 meters. The full proposed vector has length 120 meters. With $\alpha_k=0.25$, you move:

$$ 0.25\times120=30\text{ meters}. $$

Later, if you are at 90 meters and a new judgment places the river at 110 meters, the same $\alpha_k=0.25$ moves you only 5 meters. The coefficient fixes a fraction, not a physical distance.