Numerical analysis rewards careful thinking, but it also punishes small confusions brutally. Many of the most painful errors are not deep technical failures. They are category mistakes: mixing norms, mixing error layers, misreading diagnostics, or applying the right theorem to the wrong regime.

This article lists common mistakes that repeatedly appear in coursework, research prototypes, and production codes. Each mistake comes with a replacement habit that keeps you out of trouble.

Mistake: believing smaller steps always improve accuracy

Refining a mesh or timestep reduces discretization error, but it can increase arithmetic error. A useful mental model is a two-term error budget:

where $C h^p$ represents truncation error of order $p$, $\varepsilon$ is machine precision, and the $h^{-1}$ factor represents accumulation of rounding over roughly $1/h$ steps or operations.

The curve has a minimum. Past that point, smaller $h$ makes the total error worse.

A replacement habit:

• predict the order $p$ from theory,
• estimate the scale of $C$ from a coarse run,
• look for a plateau or a minimum in an error-versus-$h$ experiment,
• stop refining when the plot stops improving.

Mistake: treating a small residual as a small error

For linear systems $Ax=b$, the residual is $r = b – A\tilde x$. A small residual does not guarantee a small forward error $x-\tilde x$ when the system is ill-conditioned. The inequality to remember is:

where $\kappa(A)$ is the condition number in the relevant norm.

If $\kappa(A)$ is large, residual-based stopping criteria can claim success while the answer is still inaccurate.

A replacement habit:

• report both residual norm and an estimate of $\kappa(A)$ when feasible,
• use backward error language to interpret results,
• for iterative solvers, monitor error proxies that incorporate conditioning, such as preconditioned residuals.

Mistake: switching norms mid-proof or mid-plot

A stability bound proved in $L^2$ does not automatically control an $L^\infty$ error. A convergence plot made in one norm does not justify a claim stated in another.

This mistake shows up in PDE discretizations, where energy methods naturally live in $L^2$ or $H^1$, but applications might care about pointwise behavior.

A replacement habit:

• state the norm in every theorem statement and every error table,
• when you need a different norm, insert the inequality that connects them, such as inverse estimates or Sobolev embeddings, and make clear what constants depend on mesh resolution.

Mistake: assuming a method is stable because it converges on one example

Stability is a uniform statement over a class of inputs and refinement levels. A method can converge for a benign problem and still be unstable for nearby data or slightly different parameters.

This often happens in time stepping: an explicit scheme can look accurate on a coarse grid because the unstable modes are not resolved, then fail abruptly once the grid resolves them.

A replacement habit:

• test stability on a family of examples that refine $h$ and vary key parameters,
• include at least one example designed to excite the worst-case mode the scheme is known to struggle with,
• if the method has a known step restriction, treat it as part of the method, not as optional advice.

Mistake: ignoring scaling and non-dimensionalization

Poor scaling turns a well-posed problem into a numerically hostile one. Units and magnitudes matter because floating point has finite dynamic range and because conditioning depends on relative scales.

A replacement habit:

• scale variables so typical values are near one when possible,
• scale equations so coefficients are comparable in magnitude,
• interpret condition numbers and stability constants in the scaled variables, not in the raw units.

Mistake: reading a log-log plot without checking the asymptotic regime

Convergence rates are asymptotic statements. If you are not in the asymptotic regime, the slope you measure is not the order you think you are measuring.

Common symptoms:

• the slope changes dramatically across refinement levels,
• the finest points flatten due to rounding,
• the coarsest points reflect pre-asymptotic behavior.

A replacement habit:

• fit slopes only on the portion of the curve that is visibly linear on the log-log scale,
• report the refinement range used for the fit,
• include a reference slope line or a table of ratios so the reader can verify the order directly.

Mistake: confusing local error with global error

Many methods have a local defect of order $h^{p+1}$ but a global error of order $h^p$. The difference is accumulation over many steps.

A replacement habit:

• explicitly write the error recurrence,
• apply a discrete Grönwall inequality or a stability estimate to convert local to global,
• keep track of where factors like $1/h$ come from.

Mistake: treating roundoff as random noise you can ignore

Roundoff can behave like noise, but it can also be structured, especially when cancellation occurs. Subtracting two nearly equal numbers destroys significant digits. Summing a long list in a naive order can lose accuracy.

A replacement habit:

• rewrite expressions to avoid catastrophic cancellation when possible,
• use compensated summation for long reductions,
• check whether your problem involves subtracting close quantities and adjust the formulation.

Mistake: using default tolerances as if they were mathematical truth

Many scientific computing libraries choose defaults that are reasonable for generic tasks, not for your specific accuracy claim. A tolerance that is safe for one scale can be useless for another.

A replacement habit:

• relate tolerances \to a target error budget in physical or geometric units,
• scale stopping criteria by a meaningful norm of the data,
• report the tolerance and the norm used in the stopping condition so results are interpretable.

Mistake: assuming discretization preserves qualitative structure automatically

Many continuous problems have structure that matters: positivity, monotonicity, conservation, maximum principles, or invariants. Discretizations do not preserve these automatically.

A replacement habit:

• identify the structure that matters before you discretize,
• choose a method known to respect it, or add stabilization that enforces it,
• test the structure directly in computations as part of validation, not as an afterthought.

Mistake: thinking verification is the same as validation

Verification asks: did you solve the discrete equations correctly? Validation asks: do the equations represent the phenomenon you care about?

A numerical method can be verified and still be irrelevant.

A replacement habit:

• use manufactured solutions to verify convergence and implementation,
• use problem-specific invariants or experimental comparisons to validate the model,
• keep these two goals distinct in both writing and code structure.

Mistake: mixing discretization error with solver error

Many modern workflows have two approximation layers at once.

• a discretization produces a finite-dimensional problem, such as a linear system or a nonlinear residual
• an iterative solver produces an approximate solution of that finite problem

If you refine the mesh but keep the solver tolerance fixed, the solver error can dominate and you will mistakenly conclude the discretization is not converging. If you tighten the solver tolerance aggressively on a coarse mesh, you can waste work solving an imprecise model to unnecessary accuracy.

A replacement habit:

• decide an error budget and allocate part to discretization and part to the solver
• scale solver tolerances with refinement so the solver error stays below the discretization error
• when possible, measure both by monitoring a discretization indicator and a solver residual in the same norm

Mistake: using unstable differencing or interpolation formulas near the boundary

Boundary handling is a frequent hidden source of failure. A scheme can be stable in the interior and become unstable or low-order because boundary stencils are inconsistent or because ghost values are filled incorrectly.

A replacement habit:

• derive boundary stencils from the same accuracy principle used in the interior
• test boundary-only manufactured solutions where the interior is simple but boundary behavior is sharp
• measure error not only globally but also near boundaries so the diagnostic cannot hide local defects

Mistake: forgetting that floating point arithmetic is not associative

The identity $(a+b)+c = a+(b+c)$ is true in exact arithmetic but false in floating point. Reordering operations can change results, sometimes dramatically, especially when magnitudes vary widely.

A replacement habit:

• when summing many terms, order by magnitude or use compensated summation
• avoid formulas that subtract nearly equal quantities
• treat “fast vectorized reductions” as numerical objects that may need a stability argument, not as transparent replacements for exact sums

Mistake: treating Monte Carlo output as a deterministic approximation

Randomized methods and Monte Carlo integration come with statistical error. Reporting a single number without uncertainty is not an error bound.

A replacement habit:

• report an empirical variance estimate and a confidence interval
• ensure independent sampling assumptions are met or explain how dependence is handled
• check convergence by plotting error against sample size on a scale that reflects the $N^{-1/2}$ rate typical for plain Monte Carlo

Mistake: assuming a single refinement path is enough evidence

A method can look good on one mesh family and fail on another, especially on irregular grids or anisotropic refinement. Similarly, a method can pass a smooth manufactured test and fail on a solution with limited regularity.

A replacement habit:

• test at least one smooth case and one limited-regularity case
• use more than one refinement family when geometry allows it
• vary parameters that control difficulty, such as conditioning or frequency, not only the mesh size

Mistake: losing track of what was actually computed

The most avoidable failures are bookkeeping failures: a parameter is changed, a tolerance is altered, a normalization is forgotten, and the resulting plot is interpreted as if the setup were unchanged.

A replacement habit:

• log parameters, norms, and tolerances with every run
• store the discretization scale and solver settings alongside the output
• when presenting results, include the exact method variant and key settings so the conclusion is attached to the correct computation

Quick self-checks before you trust a result

When you are about to claim an algorithm works, the following checks catch a large fraction of failures.

• error decreases at the expected rate over a visible asymptotic regime
• residual is small and conditioning is not ignored
• refinements do not trigger instability
• norms are stated and consistent
• tolerances are tied to an error budget
• qualitative structure is monitored if it matters
• reference solutions are described and credible

Numerical analysis is rigorous when it refuses to confuse “it runs” with “it is accurate.” Avoiding these mistakes is less about memorizing warnings and more about building the habit of asking: what exactly is being bounded, in which norm, and under which stability guarantee.

Numerical analysis is full of theorems that promise what happens “as $h\to 0$” or “as $N\to\infty$.” That asymptotic language is valuable, but it is easy to lose the practical and conceptual center of the subject: you understand a numerical method when you can build examples that reveal its strengths and its failure modes.

This article gives a practical recipe for constructing examples on purpose. The point is not to produce a zoo of special cases. The point is to create test problems that isolate a phenomenon so clearly that both the computation and the analysis have nowhere to hide.

Why constructing examples is a skill, not an afterthought

A good example does at least one of the following.

• It tests a hypothesis under the sharpest regime where it might fail.
• It separates two effects that are usually entangled, such as discretization error and conditioning.
• It produces a ground truth so error can be measured without guessing.
• It forces the method to confront the exact structure it claims to respect: symmetry, conservation, monotonicity, positivity, or smoothness.

A weak example does the opposite: it is so generic that any method looks fine, or so chaotic that you cannot explain why a method fails.

The recipe: build from the phenomenon backward

A reliable workflow is to start with the phenomenon you want to see, then engineer the data so that the phenomenon is unavoidable.

Step one: name the phenomenon precisely

Examples become useful when you can describe the target stress in a single sentence.

• strong sensitivity to small perturbations of the data
• slow convergence of an iterative method due to spectrum clustering
• loss of accuracy near a boundary because the solution has a sharp transition
• cancellation in floating point because two large terms nearly subtract
• loss of order because the target function is not smooth enough for the method’s assumptions

If you cannot state the phenomenon, you cannot design the example.

Step two: choose whether you want a truth model or a stress model

A truth model is built to support measurement: you can evaluate the exact answer or a reference solution to high precision.

A stress model is built to push a method into a regime where it might break. You might not have a closed-form truth, but you can still diagnose behavior with invariants and consistency checks.

Both are legitimate. A mature numerical study usually includes both.

Step three: control the scale parameter explicitly

Most numerical behavior is revealed by scaling. Decide what you refine and what you hold fixed.

• mesh spacing $h$
• timestep $\Delta t$
• polynomial degree $p$
• iteration count $k$
• noise amplitude $\sigma$
• condition number target $\kappa$

Then decide how the other quantities should scale with that parameter. Many confusing experiments are confusing because they refine $h$ while silently changing something else that matters just as much.

Manufactured solutions: the cleanest way to get ground truth

For differential equations, the “method of manufactured solutions” is the most reliable way to build a truth model.

You choose a smooth function $u(x,t)$ that you want to be the solution, then compute the forcing term that makes it a solution.

For example, pick

Then compute

If you want a heat equation with diffusivity $\nu$,

then the forcing must be

Boundary and initial data are taken from $u$. Now you have a test problem where the exact solution is known and smooth, so you can verify the predicted convergence rate of your spatial and temporal discretization.

You can also manufacture non-smooth solutions on purpose to test order reduction. Choose a function that is continuous but has a kink, then build the forcing accordingly. This is one of the cleanest ways to check whether your error analysis is truly using the regularity assumptions it claims to use.

Building linear algebra examples with a chosen spectrum

Many algorithms, especially iterative solvers, are controlled by spectral properties. A practical way to build examples is to pick eigenvalues first, then build a matrix that has them.

If you want a symmetric positive definite matrix with eigenvalues $\lambda_1,\dots,\lambda_n$, choose an orthogonal matrix $Q$ and set

Then $A$ has the chosen spectrum. You can design examples that:

• have a specified condition number $\kappa = \lambda_{\max}/\lambda_{\min}$,
• have clustered eigenvalues that favor conjugate gradient,
• have a few outliers that slow down basic iterations,
• have eigenvalues spanning multiple scales to stress preconditioning.

A related construction controls singular values for non-symmetric or rectangular problems by using an SVD-like build:

with orthogonal $U$, $V$.

These constructions let you test a solver’s dependence on conditioning without mixing in unrelated complications such as geometry or boundary effects.

Examples for time stepping: separate stability from accuracy

For time stepping methods, it is easy to confuse stability limitations with truncation error. Build examples that separate them.

A simple approach is to use scalar test equations.

• For non-stiff behavior: $y’ = -y$ has a benign decay.
• For stiff behavior: $y’ = -\alpha y$ with $\alpha\gg 1$ forces explicit methods to take tiny steps for stability.

You can then check two regimes explicitly.

• Fix a stable $\Delta t$ and refine it to observe the convergence rate.
• Fix a target accuracy and increase $\alpha$ \to see how the stability restriction interacts with cost.

If you only refine $\Delta t$ in a regime where the scheme is unstable, you will not learn anything about accuracy because the method is not producing meaningful approximations in that regime.

Examples for quadrature: smooth, oscillatory, nearly singular

Quadrature error is driven by structure.

• Smooth integrands with bounded derivatives tend to follow classical rate predictions.
• Highly oscillatory integrands can defeat naive sampling.
• Nearly singular integrands reveal whether a method is handling endpoint behavior correctly.

A practical trio of examples uses integrals on $[0,1]$.

• Smooth: $\int_0^1 e^{-x^2}\,dx$.
• Oscillatory: $\int_0^1 \cos(\omega x)\,dx$ with $\omega$ large.
• Nearly singular: $\int_0^1 x^{-1/2}\,dx$, which is integrable but not smooth at the endpoint.

Each example has a known exact value or a high-precision reference. Together they reveal whether a quadrature scheme is relying on smoothness, whether it needs special handling for oscillations, and whether it is robust to endpoint singularity.

Examples for interpolation and approximation: force the pathology

Interpolation and approximation are notorious for producing good pictures that hide bad mathematics. To test methods honestly, use examples that are designed to display worst-case behavior.

A classic smooth but challenging target on $[-1,1]$ is

With equispaced nodes and high-degree polynomial interpolation, one often sees large oscillations near endpoints. With nodes clustered near the endpoints, the behavior improves dramatically. This is not a matter of taste. It is a structural fact about how polynomial interpolation distributes error.

For piecewise approximations, build examples with limited regularity.

• A function with a kink tests whether you can get high order without smoothness.
• A function with a narrow spike tests resolution and adaptive refinement.

Iterative methods: examples that reveal contraction constants

For fixed point iterations and nonlinear solvers, the contraction constant matters. Build examples where you can compute or bound it.

A clean fixed point example is

The iteration $x_{k+1} = \cos(x_k)$ converges because $|\sin(x)|\le 1$ and, near the fixed point, the derivative is less than one in magnitude. You can then build a variant with a derivative close to one to force slow convergence, making the role of the contraction constant visible rather than abstract.

For Newton’s method, multiple roots provide a classic slow-down example. If

then the root at $x=1$ has multiplicity $m$, and Newton’s method loses its usual quadratic convergence. This kind of example is essential when evaluating stopping criteria, because residual-based stopping can be misleading near multiple roots.

How to document an example so it teaches

A good numerical example is reproducible in concept even if the reader never runs your code. That means your writeup should include information that allows the reader to reason about the result.

• the exact definition of the problem, including boundary and initial data
• the refinement parameter and how it is varied
• the norm or metric used to measure error
• the reference solution strategy if an exact truth is not available
• the stopping criteria for iterative components
• a brief explanation of what phenomenon the example is designed to isolate

When you include these elements, your examples become part of your argument rather than decoration.

A final principle: examples should be legible to theory

The best examples in numerical analysis have a dual personality.

• They are simple enough that you can explain the behavior with a short theoretical narrative.
• They are sharp enough that the method’s limitations become visible.

If you can do both, your examples will not just confirm a method. They will reveal the structure that the method is actually using. That is the point of constructing examples on purpose.

Numerical analysis is often described as “the study of algorithms for approximate computation.” That is accurate, but it hides the organizing idea that makes the subject coherent: a numerical method is only as good as the error statement you can support. If you can explain why your output is close to the desired object, how that closeness scales as you refine the discretization, and how the inevitable floating point noise influences the answer, then you are doing numerical analysis. If you cannot, you are doing computation that might work.

This guide is a set of proof moves you can reuse across problems. It starts with error bounds because they force you to identify the right quantities, norms, and stability notions before you can claim anything meaningful.

The three-layer model of error

Almost every serious error bound separates three effects.

• Modeling error: the mathematical model is already an approximation of the physical or geometric situation.
• Discretization error: even if the model is exact, you approximate an infinite or continuous object by a finite one: a mesh, a timestep, a truncated basis, a finite sample.
• Arithmetic error: the computations are done in floating point and contain rounding.

A clean analysis is not required to treat these layers equally, but it must make clear which layer is being bounded. Many arguments collapse because they silently switch layers. The most common accidental switch is to prove a discretization error bound while ignoring arithmetic, and then interpret the result as a bound for a computed answer.

Start by choosing the right notion of “small”

An error bound is meaningless until you decide how to measure error. This is not a cosmetic decision. Stability, conditioning, and convergence can change when you change norms.

A useful habit is to write down three objects explicitly.

• The exact object you want, call it $u$.
• The ideal discrete object produced by your method in exact arithmetic, call it $u_h$ where $h$ is the discretization scale.
• The computed object produced on a real machine, call it $\tilde u_h$.

Then decide which distance you actually care about.

• $\lVert u – u_h\rVert$ is the discretization error in the chosen norm.
• $\lVert u_h – \tilde u_h\rVert$ is the arithmetic or algorithmic error.
• $\lVert u – \tilde u_h\rVert$ is the total error, typically bounded by the triangle inequality.

If you keep these symbols separate on paper, you catch many mistakes early.

The core template: consistency plus stability implies convergence

A large fraction of numerical analysis follows one pattern: you show that the method is locally accurate and that it does not amplify small errors too much.

In the most abstract form you have an operator equation

and a discrete operator $\mathcal L_h$ such that the method produces $u_h$ with

You then build the argument from two ingredients.

• Consistency: if you insert the exact solution into the discrete equation, the defect is small:

is controlled by a bound like $\lVert \text{defect}_h\rVert \le C h^p$.

• Stability: the inverse of $\mathcal L_h$, or the discrete solution operator, does not blow up:

where $S$ may depend on the problem but is not allowed to explode as $h\to 0$ in the regime where you claim convergence.

Then the global error bound is usually one line:

so

This is the cleanest version of the idea. Many settings require work because $\mathcal L_h$ is not explicitly invertible, or because the stability bound is more subtle, but the logic stays the same.

A warning about what “stability” means

In numerical analysis, “stability” is not a single property. It is a family of properties that compare input perturbations to output perturbations. A stability statement is always of the form:

> If the data is perturbed by $\delta$, the output changes by at most $K\delta$ in the chosen norm.

You do not need a perfect stability constant $K$. You need a stability statement that is honest about how $K$ depends on the problem parameters and on $h$.

Worked example: explicit Euler and discrete Grönwall

Consider an initial value problem

with $f$ globally Lipschitz with constant $L$. The explicit Euler method is

The local truncation error is defined by inserting the exact solution into the method:

A Taylor expansion gives $\tau_{n+1} = O(h)$, and more precisely if $y$ is twice differentiable then

so $\lvert \tau_{n+1}\rvert \le C h$ for an explicit constant $C$.

Now define the global error $e_n := y(t_n) – y_n$. Subtract the numerical update from the exact update:

Subtracting yields

Using the Lipschitz property:

Since $\lvert \tau_{n+1}\rvert \le C h$, we get

A discrete Grönwall inequality gives

Since $e_0=0$, we obtain the familiar global error order $\lvert e_n\rvert = O(h)$ over a fixed time interval. The key steps were:

• define the right error variable $e_n$,
• express the next error in terms of the current error plus a defect,
• use a stability bound on the propagation factor $1+hL$,
• sum the defect contributions in a controlled way.

That is the method in miniature. The same reasoning appears in multistep methods, Runge–Kutta schemes, and time stepping for PDEs, with more elaborate stability estimates.

Worked example: quadrature and how smoothness enters

Take an integral $I = \int_a^b g(x)\,dx$. The composite trapezoidal rule with step $h=(b-a)/N$ is

A standard derivation gives the error representation

for some $\xi\in(a,b)$ if $g$ has a continuous second derivative. The structure of the bound matters more than the constant.

• The error depends on a higher derivative of the integrand.
• The error scales like $h^2$ when that derivative is bounded.

This is a recurring theme: rates come from regularity. When the function is not smooth, the order deteriorates and you must redesign the method or change the analysis. For oscillatory integrals or nearly singular functions, you do not fix the problem by hoping a smaller $h$ will behave well. You fix it by using the structure of the integrand in the scheme and in the bound.

Conditioning and backward error: linear algebra as numerical analysis

For linear systems $Ax=b$, an “error bound” is often more meaningful when phrased as backward error:

> The computed $\tilde x$ is the exact solution of a nearby problem $(A+\Delta A)\tilde x = b+\Delta b$, with $\Delta A$, $\Delta b$ small.

Backward error separates algorithmic quality from problem difficulty. If the algorithm is backward stable, then the forward error $\lVert x-\tilde x\rVert$ is controlled by the conditioning of the map $b\mapsto A^{-1}b$. In a norm $\lVert\cdot\rVert$ one basic statement is:

where $\kappa(A) = \lVert A\rVert\,\lVert A^{-1}\rVert$ is the condition number.

The proof move you should recognize is the same as before: identify the operator inverse that amplifies perturbations, then bound it.

A posteriori error bounds: using the residual

Not all error statements are a priori rates like $C h^p$. Many practical methods are guided by a posteriori bounds, where a computable quantity controls the error.

A simple example appears in fixed point iteration. Suppose $x$ solves $x = T(x)$, with $T$ a contraction with constant $q<1$. If $x_k$ is an iterate, the residual is $r_k := x_k - T(x_k)$. A standard bound is

This is powerful because $r_k$ is directly computable. Similar ideas occur in finite element methods, where residuals and flux jumps give local indicators that guide mesh refinement.

The proof pattern is again stability: the inverse of $I-T$ exists on the contraction regime, and $\lVert (I-T)^{-1}\rVert\le (1-q)^{-1}$.

Proof moves worth memorizing

When you read or write numerical analysis, you will see the same moves with different symbols.

• Error decomposition: write the quantity you want as “ideal discrete error plus algorithmic error,” then bound each separately.
• Defect equation: subtract the discrete equation from the exact equation inserted into the discrete framework.
• Stability inequality: show that the discrete solution operator does not blow up in the norm you care about.
• Perturbation lemma: bound how the solution changes when data changes.
• Energy method: multiply by the error, integrate or sum, and bound a discrete energy in time-dependent settings.
• Discrete Grönwall: control accumulation of local defects.
• Backward error viewpoint: rewrite the computed answer as an exact answer \to a nearby problem and then apply conditioning.

How to use this guide on a new problem

When you meet a new method, resist the urge to chase the final rate immediately. Instead, ask questions in an order that forces the correct architecture.

• What is the exact mathematical target, and what is the discrete object you actually compute?
• In which norm is the claim intended?
• What is the defect when the exact object is inserted into the discrete scheme?
• What statement prevents the scheme from amplifying small errors?
• Does the stability constant depend on a hidden parameter that can explode?
• Is the meaningful error a priori, a posteriori, or backward?

If you answer those questions cleanly, the proof is usually close. If you cannot answer them, the method may still work in practice, but the analysis is not yet truthful.

The deeper point is that numerical analysis is not a collection of isolated tricks. It is a discipline of making approximation claims precise. Error bounds are the language that turns computation into mathematics.