Series: Building Auditable LLM Systems · Part 1 of 11 A small LLM prototype can often be built with one prompt. Put the current state, a few examples, the output format, and an instruction into a single block of text, then see whether the model returns a plausible answer. That is enough for a smoke test. It is not enough for a system that must be reproduced, audited, or trusted to affect an external process. ...
Series: Stochastic Approximation from First Principles · Part 1 of 22 Stochastic approximation studies a basic question: can an iterative update converge when every target observation contains noise? Robbins and Monro introduced the framework in 1951, and Dvoretzky gave a general convergence theorem in 1956. Its basic form is: $$ \text{New}=\text{Old}+\alpha(\text{Target}-\text{Old}). $$Imagine searching for a river at night. Let $w_k$ denote your actual position after step $k$. The riverbank has a fixed true location $w^*$, but you cannot see or use it directly. You can only listen to the water and form a location judgment, $\text{target}_k$. ...
Series: Stochastic Approximation from First Principles · Part 2 of 22 After writing the update $$ w_{k+1}=w_k+\alpha_k(\text{target}_k-w_k), $$we need a quantity that tells us whether the iteration is approaching the true target. In the forest analogy, the question is: Am I moving toward the river, or merely moving around the forest? Two Ways to Define Error The first option is the change between consecutive iterates: $$ w_{k+1}-w_k = \alpha_k(\text{target}_k-w_k) = \alpha_k(w^*+\eta_k-w_k). $$The second option is the displacement from the current position to the true target: ...
Series: Building Auditable LLM Systems · Part 2 of 11 An LLM system that learns from historical experience and acts online has two different jobs: produce reusable experience from past evidence; use authorized experience to make a current decision. These jobs may reuse the same engineering layers, but they require different information permissions and prompt responsibilities. The clean boundary is simple: offline processing produces memory; online control consumes it read-only. Why Mixing Offline and Online Stages Is Dangerous When extraction and execution are mixed, experimental boundaries become difficult to defend. ...
Series: Building Auditable LLM Systems · Part 3 of 11 An LLM schema does more than list fields. It defines the boundary between facts, model judgment, and feedback. Two principles are especially important: Label whether each value comes from deterministic computation or model inference. Store every model judgment with the explicit rationale produced for that decision. The first principle protects the factual boundary. The second preserves the model’s decision-time account of its judgment. ...
Series: Stochastic Approximation from First Principles · Part 3 of 22 To prove that a noisy update converges, we must constrain both the mechanism we control and the noise we do not. The original update is: $$ w_{k+1}=w_k+\alpha_k(w^*+\eta_k-w_k), $$and its error recursion is: $$ \Delta_{k+1}=(1-\alpha_k)\Delta_k+\alpha_k\eta_k. $$Assumptions on the Step Size The standard conditions are: $$ 0\le\alpha_k<1, $$$$ \sum_k\alpha_k=\infty, $$and: $$ \sum_k\alpha_k^2<\infty. $$Each controls a different failure mode: $0\le\alpha_k<1$ keeps each new position between the current position and the noisy target. $\sum_k\alpha_k=\infty$ prevents the process from losing the ability to correct its initial error. $\sum_k\alpha_k^2<\infty$ limits the cumulative strength with which noise enters the system. Assumptions on the Noise At the basic level, the disturbance sequence $\{\eta_k\}$ is assumed to satisfy: ...
Series: Building Auditable LLM Systems · Part 4 of 11 Separate scenario design for an LLM decision system into two layers. The first layer generates worlds that satisfy the physical, operational, and lifecycle constraints of the domain. The second layer organizes those worlds into named task families that define what the model should learn, where it should be tested, and which claims the evaluation can support. These layers answer different questions: ...
Series: Stochastic Approximation from First Principles · Part 4 of 22 In the update $$ w_{k+1}=w_k+\alpha_k(\text{target}_k-w_k), $$what does $\alpha_k$ actually represent? Separate Direction from Distance Treat the update as a vector movement: $$ w_{k+1} =w_k+ \underbrace{\alpha_k\|\text{target}_k-w_k\|}_{\text{actual distance moved}} \cdot \underbrace{ \frac{\text{target}_k-w_k} {\|\text{target}_k-w_k\|} }_{\text{unit direction}}. $$The vector $\text{target}_k-w_k$ contains both a direction and a distance: $\|\text{target}_k-w_k\|$ is the distance suggested by the latest observation. $\alpha_k\|\text{target}_k-w_k\|$ is the distance the update actually moves. $\alpha_k$ is the fraction of the suggested movement that the update accepts. In the river analogy, you walk an $\alpha_k$ fraction of the estimated distance toward the location implied by the current sound. The step size is therefore both a movement ratio and a degree of trust in the latest estimate. ...
Series: Building Auditable LLM Systems · Part 5 of 11 The prompt is the model’s decision interface, not a place to concatenate everything the system knows. A well-designed prompt follows the order of the decision. It explains the task, the action boundary, the current facts, relevant history or external experience, the requested justification, and the output contract. Keep content with different sources and responsibilities in distinct blocks. This structure makes it possible to know what the model actually saw, why it made a judgment, and whether two methods were compared under the same prompt contract. ...
Series: Stochastic Approximation from First Principles · Part 5 of 22 Why should $\alpha_k$ lie between zero and one? The answer has two parts: geometry explains overshooting, and the error recursion explains noise amplification. Geometry Without Noise If $\eta_k=0$, then: $$ w_{k+1}=(1-\alpha_k)w_k+\alpha_k w^*. $$The result depends on the value of $\alpha_k$: Step size Position of $w_{k+1}$ Behavior $\alpha_k\in(0,1)$ Between $w_k$ and $w^*$ Moves steadily toward the target $\alpha_k=1$ Exactly at $w^*$ Reaches the target in one step $\alpha_k>1$ Beyond $w^*$ Overshoots and may oscillate In the river analogy, $\alpha_k>1$ means walking past the estimated river location. With no noise, that merely overshoots the river. With noise, it overshoots a location that may already be wrong. ...