Probability lives in a world of measures on large spaces. Computing lives in a world of finite objects: floating-point numbers, arrays, finite graphs, finite random seeds. Almost every computational workflow in probability is a discretization, even when you do not call it that.
The interesting question is not whether discretization changes things. It always changes things. The question is: which probabilistic structures are stable under the kinds of discretization we actually use. When you know the stable structures, you can design algorithms that preserve the features you care about and you can state error claims that match the mathematics.
Streaming Device Pick4K Streaming Player with EthernetRoku Ultra LT (2023) HD/4K/HDR Dolby Vision Streaming Player with Voice Remote and Ethernet (Renewed)
Roku Ultra LT (2023) HD/4K/HDR Dolby Vision Streaming Player with Voice Remote and Ethernet (Renewed)
A practical streaming-player pick for TV pages, cord-cutting guides, living-room setup posts, and simple 4K streaming recommendations.
- 4K, HDR, and Dolby Vision support
- Quad-core streaming player
- Voice remote with private listening
- Ethernet and Wi-Fi connectivity
- HDMI cable included
Why it stands out
- Easy general-audience streaming recommendation
- Ethernet option adds flexibility
- Good fit for TV and cord-cutting content
Things to know
- Renewed listing status can matter to buyers
- Feature sets can vary compared with current flagship models
This article is a guide to what tends to survive and what tends to break when probability is made finite.
First principle: pick the topology you are approximating in
Two probability measures can be “close” in many inequivalent ways. Computation forces you to decide what “close” means, even if you never say it aloud.
Common choices include:
- Weak convergence: $\mu_n\Rightarrow \mu$ means $\int f\,d\mu_n\to \int f\,d\mu$ for every bounded continuous $f$. This is the default for approximating distributions on $\mathbb{R}^d$.
- Total variation: $\|\mu-\nu\|_{TV}$ controls sup error over all events. This is stronger and often too strong for continuous approximation unless densities are very close.
- Wasserstein metrics: these control errors in expectations of Lipschitz functions and are natural when geometry matters.
A discretization that is excellent in weak convergence might be terrible in total variation. Many computational confusions are just metric confusions.
Discretizing a distribution: bins, quadrature, and what stays true
Suppose $X$ is a real-valued random variable with law $\mu$. A simple discretization is binning: replace $X$ by $X^{(h)}$ that takes values on a grid of step size $h$ by rounding, truncation, or projection.
What survives automatically:
- Nonnegativity and normalization: probabilities remain probabilities if you renormalize correctly.
- Support constraints: if you project onto a grid inside a bounded interval, you preserve bounded support.
- Weak features: many smooth expectations $\mathbb{E}[f(X)]$ are approximated well when $f$ is regular at the scale $h$.
What does not survive automatically:
- Tail behavior: truncation changes rare-event probabilities, sometimes drastically.
- Exact moments: binning can bias $\mathbb{E}[X]$ and $\mathrm{Var}(X)$ unless designed to preserve them.
- Independence: discretizing two variables separately can create or destroy dependence structure if they share fine-scale coupling.
A good computational design is explicit about which moments or functionals you want to preserve.
A useful table: what you can promise from a grid approximation
| Target statement | Typical sufficient condition |
|—|—|
| $\mu_h\Rightarrow \mu$ | mesh $h\to 0$ and tightness of $\mu_h$ |
| $|\mathbb{E}[f(X^{(h)})]-\mathbb{E}[f(X)]|\le \varepsilon$ | $f$ Lipschitz and $\mathbb{E}[|X^{(h)}-X|]$ small |
| total variation closeness | density approximation in $L^1$ with uniform control |
This is why weak convergence is the natural baseline: it matches what most finite approximations can support.
Monte Carlo: the central invariant is unbiasedness
In Monte Carlo, you approximate $\mathbb{E}[f(X)]$ by
with $X_k$ i.i.d. copies of $X$.
The most important structure that survives discretization here is unbiasedness:
Unbiasedness is not automatic; it is built into the sampling scheme. When you discretize a continuous distribution by an approximate sampler, you may lose unbiasedness and you should quantify the bias.
What else survives, under mild hypotheses:
- Law of large numbers: $\hat{I}_N\to \mathbb{E}[f(X)]$ almost surely if $\mathbb{E}[|f(X)|]<\infty$.
- Central limit scaling: fluctuations behave like $N^{-1/2}$ when $\mathrm{Var}(f(X))<\infty$.
What fails in heavy tails is not “Monte Carlo” but the assumptions that give finite variance or even finite mean. In computation, you handle this by truncation, importance sampling, or robust estimators, but then you must track the new bias-variance tradeoff explicitly.
Markov chains: preserving stationarity is a design choice
Many computational probability tasks use Markov chain Monte Carlo (MCMC). A Markov chain is defined by a transition kernel $P(x,dy)$. The foundational object is a stationary distribution $\pi$ satisfying
When you implement a chain on a computer, you often discretize:
- the state space (finite grid or finite truncation)
- the kernel (numerical evaluation of densities)
- the acceptance step (finite precision)
What can survive well:
- Stationarity by construction: if you enforce detailed balance, $\pi(x)P(x,y)=\pi(y)P(y,x)$, then $\pi$ is stationary in the exact arithmetic model.
- Ergodicity on the discretized space: irreducibility and aperiodicity can be checked on the finite graph.
What can break:
- Target correctness under truncation: truncating the state space replaces $\pi$ by a conditional or renormalized $\pi_h$. That may be fine, but it must be stated.
- Mixing rates: a discretized chain can mix much slower than the continuous ideal, especially in high dimension.
- Reversibility under floating point: detailed balance relies on exact ratios. Finite precision can produce systematic drift if accept/reject thresholds are mishandled.
A practical principle is: if stationarity is essential, build it into the kernel symbolically, not numerically.
Coupling: the right way to compare the ideal and the computed
A clean way to reason about discretization error is coupling. You construct $(X, X^{(h)})$ on the same probability space and bound $\mathbb{P}(X\ne X^{(h)})$ or $\mathbb{E}[|X-X^{(h)}|]$.
Why this survives computation:
- Coupling converts measure comparison into a comparison of random variables.
- Many discretizations are literally defined by transforming the same underlying randomness, such as rounding a continuous sample.
Once you have a coupling, you can bound errors in expectations:
for Lipschitz $f$. This is exactly the sort of statement computation can support.
Concentration survives if you preserve boundedness or variance control
Concentration inequalities are computationally friendly because they turn finite samples into explicit error bars. What discretization threatens is the hypothesis.
If your discretization preserves:
- boundedness of $f(X)$, or
- a usable variance proxy,
then bounds like Hoeffding or Bernstein remain meaningful. If discretization introduces rare large spikes, concentration becomes misleading.
A strong engineering habit is to monitor empirical tails and to design transforms that keep $f(X)$ within controlled ranges when possible.
Floating point: measure preservation can fail quietly
Floating point errors usually appear small, but they can break exact invariants:
- probabilities that should sum to one can drift
- nonnegativity can be violated by subtraction
- accept/reject rules can become biased near thresholds
Two simple safeguards preserve probabilistic structure:
- renormalize explicitly when building discrete distributions
- avoid subtractive cancellation in probability calculations by working in log-space for very small probabilities
These are not numerical tricks; they are measure-theoretic safeguards.
What you should say in a research-grade computational claim
A computational probability statement is strongest when it names:
- the ideal object (measure, kernel, expectation)
- the discretized object (grid, truncation, finite chain)
- the comparison notion (weak, total variation, Wasserstein, coupling)
- the error parameter (mesh size $h$, sample size $N$, truncation radius $R$)
Then you can say something precise, such as:
- “As $h\to 0$, $\mu_h\Rightarrow \mu$.”
- “For Lipschitz $f$, $|\mathbb{E}_{\mu_h}f-\mathbb{E}_{\mu}f|\le C h$.”
- “The computed Markov chain has stationary distribution $\pi_h$ and $\|\pi_h-\pi\|$ is controlled in Wasserstein distance by the truncation level.”
This is what “what survives discretization” really means: you isolate the invariants your algorithm respects and you measure the rest.
A compact takeaway
Discretization does not have a single effect on probability. Its effect depends on the structure you are trying to preserve.
- Weak distributional features survive many discretizations.
- Tail events and fine dependence structure are fragile.
- Stationarity and unbiasedness can survive, but only if you build them in.
- The most transparent error analysis comes from coupling and metric choice.
Once you think this way, computational probability stops being a collection of ad hoc approximations. It becomes a controlled translation between infinite objects and finite representations, with a named contract for what is preserved.
Time-stepping stochastic processes: preserving the right martingale structure
When you simulate a stochastic process, you typically replace continuous time by a time grid $t_k = k\Delta t$. Even when the underlying model is not written as a differential equation, the computational object is a recursion.
Two features matter more than they first appear:
- adaptedness: the next step must depend only on information available up to the current time
- conditional mean structure: martingale increments should have conditional mean zero when that is part of the model
A simple example is a random walk $S_{k+1}=S_k+\xi_{k+1}$ with $\mathbb{E}[\xi_{k+1}\mid \mathcal{F}_k]=0$. If you approximate $\xi_{k+1}$ by a discretized noise $\tilde{\xi}_{k+1}$, you should preserve the conditional mean zero property. Otherwise you introduce drift, and the long-run behavior changes qualitatively.
In time-stepping schemes for continuous models, the same principle appears as “weak order” versus “strong order”:
- Weak accuracy means expectations of test functions $\mathbb{E}[f(X_{t_k})]$ are approximated well.
- Strong accuracy means pathwise errors $\mathbb{E}[|X_{t_k}-\tilde{X}_{t_k}|]$ are small.
Most statistical tasks care about weak accuracy. Many path-dependent functionals (such as hitting probabilities) care about strong accuracy or at least careful control of rare-event distortion.
Rare events: discretization can destroy the quantity you care about
If your target is a rare probability $\mathbb{P}(X\in A)$ with very small mass, naive discretization can be disastrous:
- truncation can remove the rare region entirely
- binning can smear a sharp boundary
- finite sampling can produce zero hits, giving a misleading estimate
This is where importance sampling and splitting methods belong. The mathematical point is: you are changing the sampling measure on purpose and correcting by a likelihood ratio. The invariant you preserve is still unbiasedness, but now it is unbiasedness under a weighted estimator.
A reliable practice is to pair any rare-event computation with a diagnostic coupling or a variance estimate that confirms you are not just measuring the discretization artifact.
Books by Drew Higgins
Prophecy and Its Meaning for Today
New Testament Prophecies and Their Meaning for Today
A focused study of New Testament prophecy and why it still matters for believers now.
Christian Living / Encouragement
God’s Promises in the Bible for Difficult Times
A Scripture-based reminder of God’s promises for believers walking through hardship and uncertainty.
Leave a Reply