https://dudusoar.github.io/tags/iterative-methods/Recent content in Iterative Methods on Yuchen DuHugo -- 0.164.0en-usFri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/01-learning-from-a-noisy-target/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/01-learning-from-a-noisy-target/Stochastic approximation begins when the true target is hidden and each update can use only a noisy estimate of it.https://dudusoar.github.io/posts/stochastic-approximation/02-model-error-as-distance-to-target/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/02-model-error-as-distance-to-target/Why convergence should track distance to the true target rather than the size of the latest update.https://dudusoar.github.io/posts/stochastic-approximation/04-what-the-step-size-means/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/04-what-the-step-size-means/The step size is simultaneously a movement ratio, a trust coefficient, and a weight on new information.https://dudusoar.github.io/posts/stochastic-approximation/07-noiseless-versus-noisy-updates/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/07-noiseless-versus-noisy-updates/Alpha equal to one is optimal for an exact target but accepts the full disturbance when the target is noisy.