https://dudusoar.github.io/tags/noise/Recent content in Noise on Yuchen DuHugo -- 0.164.0en-usFri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/01-learning-from-a-noisy-target/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/01-learning-from-a-noisy-target/Stochastic approximation begins when the true target is hidden and each update can use only a noisy estimate of it.https://dudusoar.github.io/posts/stochastic-approximation/03-what-convergence-requires/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/03-what-convergence-requires/A compact map of the step-size and noise assumptions behind stochastic-approximation convergence.https://dudusoar.github.io/posts/stochastic-approximation/05-why-the-step-size-stays-below-one/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/05-why-the-step-size-stays-below-one/Step sizes between zero and one are the only range that combines monotone error contraction with attenuation of new noise.https://dudusoar.github.io/posts/stochastic-approximation/06-why-alpha-cannot-equal-one/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/06-why-alpha-cannot-equal-one/With alpha equal to one, every update discards history and inherits the full error of the latest noisy target.https://dudusoar.github.io/posts/stochastic-approximation/07-noiseless-versus-noisy-updates/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/07-noiseless-versus-noisy-updates/Alpha equal to one is optimal for an exact target but accepts the full disturbance when the target is noisy.https://dudusoar.github.io/posts/stochastic-approximation/08-memoryless-updates-and-markov-sufficiency/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/08-memoryless-updates-and-markov-sufficiency/An update can erase the previous iterate without becoming Markov; Markov sufficiency is a property of the chosen state.https://dudusoar.github.io/posts/stochastic-approximation/09-expectation-and-variance-of-two-updates/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/09-expectation-and-variance-of-two-updates/A direct derivation compares memoryless and history-preserving updates under zero-mean finite-variance noise.https://dudusoar.github.io/posts/stochastic-approximation/10-what-noise-assumptions-buy-us/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/10-what-noise-assumptions-buy-us/Zero mean, finite variance, independence, and identical distribution each support a different part of noise averaging.https://dudusoar.github.io/posts/stochastic-approximation/11-alpha-one-requires-vanishing-noise/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/11-alpha-one-requires-vanishing-noise/If every update fully replaces the current position, convergence requires the river-location judgment error itself to disappear.https://dudusoar.github.io/posts/stochastic-approximation/18-expanding-the-error-recursion/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/18-expanding-the-error-recursion/The explicit error formula separates the residual initial condition from the weighted sum of all past disturbances.https://dudusoar.github.io/posts/stochastic-approximation/20-how-noise-variance-decays/Fri, 17 Jul 2026 00:00:00 +0000https://dudusoar.github.io/posts/stochastic-approximation/20-how-noise-variance-decays/Each disturbance is controlled twice: by its entry coefficient and by every contraction that follows it.