Input uncertainties - angle of attack, Mach number, random shape geometry. Output uncertainty - lift, drag, lift and drag- coefficients, the whole pressure, density, velocity, turbulence kinetic energy fields.

1. Numerical Methods for Uncertainty
Quantification in Aerodynamic
Alexander Litvinenko and Hermann G. Matthies
Abstract I
n this work we research the influence of uncertainties in parameters and geome-
try on the solution in numerical aerodynamic. Typical examples of parameters with
uncertainties are the angle of attack, the Mach number and airfoil geometry. We
quantify presented uncertainties and define how they propagate out. The RANS
solver is TAU code with k-w turbulence model. Discretisation techniques which
we used here are Karhunen-Lo`eve and polynomial chaos expansions. To integrate
high-dimensional integrals in probabilistic space we used Monte Carlo simulations
and collocation methods. Probability density as well as cumulative distribution func-
tions are computed by the usage of the response surface, constructed via polynomial
chaos expansion.
1 Introduction
Very often mathematical models contain parameters, right-hand sides, initial and
boundary conditions which are uncertain. Possible reasons for uncertainties are,
e.g., lack of data, random external influences or a random or uncertain environment.
Typical examples of uncertain values are the angle of attack, the Mach number,
the Reynolds number, airfoil geometry, parameters in turbulence modelling and vis-
cosity. All these uncertainties can affect the solution dramatically. In this paper we
concentrate on the angle of attack, the Mach number and the airfoil geometry. One
of the most promising techniques to model such uncertainties is the usage of random
variables and random fields. To solve the given system numerically, in our case, the
stochastic Navier Stokes equation with a k-w turbulence model, one has to discretise
the deterministic operator as well as the stochastic part. The total dimension of the
Institute of Scientific Computing, Technische Universit¨at Braunschweig, Hans-Sommer str. 65,
38106, Braunschweig, Germany, e-mail: wire@tu-bs.de,
WWW home page: http://www.wire.tu-bs.de
1

2. 2 Alexander Litvinenko and Hermann G. Matthies
resulting discrete system is the product of dimensions of the deterministic part and
the stochastic part. The dimension of the stochastic part may be large.
As soon as the coefficients of the model are uncertain the solution (velocity, pres-
sure, CL, CD, CM etc) will also be uncertain. The information of the interest usually
is not the whole set of the solutions (too much data), but some other stochastic in-
formation — cumulative distribution function, density function, the mean value,
variance, exceedance probability etc.
A recent, efficient and most easy to implement technique for solving non-linear
systems with stochastic coefficients is the stochastic collocation method. The great
advantage in comparison with, e.g., the stochastic Galerkin method is that no modi-
fications of the deterministic code are necessary. The advantage in comparison with
Monte Carlo (MC) methods is a much smaller computational cost. But we also
note that if the stochastic Galerkin method is implemented (especially for non-linear
problems) it can beat the stochastic collocation methods.
Additional aim is to speed up the stochastic collocation method by applying dif-
ferent sparse data techniques for approximation of stochastic input data as well as
for computing sparse representation of the solution for further post-processing, that,
in its turn, can also be very expensive numerically.
We have implemented two different strategies to quantify the influence of un-
certainties in the angle of attack α and in the Mach number Ma. In the first one
(Section 5.1) we assumed that the cumulative distribution functions, mean values
and standard deviations of the random variables α and Ma are given. Then for each
pair αi and Mai we compute the solution with the help of the TAU code. Since the
exact stochastic solution is unknown, we compare the obtained results with the re-
sults obtained via Monte Carlo simulations. In the second strategy (Section 5.2) we
assume that the turbulence in the atmosphere randomly changes the velocity vec-
tor and the angle of attack (see Fig. 2). We model turbulence in the atmosphere
by two additionally axes-parallel velocity vectors v1 and v2, which have Gaussian
distributions.
Numerical examples demonstrate the influence of the uncertainties in the Mach
number (Ma) and angle of attack (α) on the stochastic solution (CL and CD).
The structure of the paper is the following. Section 2 presents the short theory
about direct computation of high dimensional integrals. We remind the theory of
Monte Carlo simulations as well as the theory of sparse grid methods. In Section
5 we describe two ways of modelling of uncertainties in parameters. Section 6 is
devoted to low-rank data formats and to the data compression. Numerical results
are presented in Section 7.
2 Direct integration
In this section we remind Monte Carlo methods and sparse integration grids (see
more in [15, 23]). In problems with uncertainties the values of interest are the mean
value, variance, density function of the solution etc. The mean value of a functional

3. Numerical Methods for Uncertainty Quantification 3
Ψ of the solution u(x) can be computed as an integral over multidimensional domain
Θ:
Ψu(x) = E(Ψ(x,θ,u(x,θ))) =
Θ
Ψ(x,θ,u(x,θ))P(dθ). (1)
To compute integral numerically, one approximates the “infinite dimensional” mea-
sure space Θ by a “finite dimensional” one, ΘM, i.e. one uses only finitely many (M)
random variables:
Ψu(x) ≈ ΨN =
N
∑
z=1
wzΨ(x,θz,u(θz)) =
N
∑
z=1
wzΨN(θz), (2)
where u(θz) is the approximate solution produced by the deterministic solver for
the realisation θz (e.g. from space RM). An example of the functional Ψ can be the
lift coefficient CL depending on the angle of attack α and on the Mach number Ma
CL = CL(α,Ma) = CL(α(θ),Ma(θ)). (3)
The evaluation points are θz ∈ ΘM, and wz are the weights. Particularly, in MC
methods the weights are wz = 1/N. It is also this approximate solution u(θz) for
each realisation θz which makes the direct integration methods so costly.
To compute integral (2), proceed in the following way:
Algorithm: (Computation of integral)
1. Select points {θz|z = 1,...,N} ⊂ ΘM according to the integration rule.
2. For each θz —a realisation of the stochastic system—solve the deterministic
problem with that fixed realisation, yielding u(θz).
3. For each θz compute the integrand Ψu(x,θz,u(θz)) in (2).
4. Compute the sum in (2).
2.1 Monte Carlo simulations
Monte Carlo methods (MC methods) obtain the integration points as N independent
random realisations of θz ∈ ΘM distributed according to the probability-measure Γ
on ΘM, and use constant weights wz = 1/N. MC methods are probabilistic as the
integration points are chosen randomly, and therefore the approximation ΨN and
the error are random variables.
Monte Carlo is very robust, and almost anything can be tackled by it, provided
it has finite variance. Its other main advantage is that its asymptotic behaviour for
N → ∞ is not affected by the dimension of the integration domain as for most other
methods.
Due to the O( ΨN L2 N−1/2) behaviour of the error, MC methods converge
slowly as N → ∞ —for instance, the error is reduced by one order of magnitude
if the number of evaluations is increased by two orders. The MC methods are well
suited for integrands with small variance and low accuracy requirements. Monte

4. 4 Alexander Litvinenko and Hermann G. Matthies
Carlo methods are well-parallelable and the easiest in implementation. These meth-
ods do not require modification of the deterministic code.
2.2 Quadrature rules and sparse integration grids
The textbook approach to an integral like (2) would be to take a good one-
dimensional quadrature rule, and to iterate it in every dimension; this is the full
tensor product approach.
Assume that we use one-dimensional Gauss-Hermite-formulas Qk with k ∈ N
integration points θ j,k and weights wj,k, j = 1,...,k. As it is well-known, they
integrate polynomials of degree less than 2k exactly, and yield an error of order
O(k−(2r−1)) for r-times continuously differentiable integrands, hence takes smooth-
ness into full account.
If we take a tensor product of these rules by iterating them M times, we have
ΨN = QM
k (Ψ) := (Qk ⊗···⊗Qk)(Ψ) =
M
j=1
Qk(Ψ)
=
k
∑
j1=1
···
k
∑
jM=1
wj1,k ···wjM ,kΨN(θ j1,k,...,θ jM,k).
This “full” tensor quadrature evaluates the integrand on a regular mesh of N = kM
points, and the approximation-error has order O(N−(2r−1)/M). Due to the exponen-
tial growth of the number of evaluation points and hence the effort with increasing
dimension, the application of full tensor quadrature is impractical for high stochastic
dimensions. This has been termed the “curse of dimensions” [25].
Sparse grid, hyperbolic cross, or Smolyak quadrature [30, 4, 8] can be applied in
much higher dimensions—for some recent work see e.g. [18, 25, 5, 26, 28] and the
references therein. A software package is available in [27].
Like full tensor quadrature, a Smolyak quadrature formula is constructed from
tensor products of one-dimensional quadrature formulas, but it combines quadrature
formulas of high order in only some dimensions with formulas of lower order in the
other dimensions. For a multi-index η ∈ NM the Smolyak quadrature formula is
ΨN = SM
k (Ψ) := ∑
k≤|η|≤k+M−1
(−1)k+M−1−|η| k −1
|η|−k
M
j=1
Qηj (Ψ).
For a fixed k the number of evaluations grows significantly slower in the number
of dimensions than for full quadrature. The price is a larger error: full quadrature
integrates monomials θη
= θη1
1 ···θηM
M exactly if their partial degree maxjηj does
not exceed 2k−1. Smolyak formulas SM
k integrate multivariate polynomials exactly
only if their total polynomial degree |η| is at most 2k−1. However the error is only

5. Numerical Methods for Uncertainty Quantification 5
O(N−r(logN)(M−1)(r+1)) with N = O 2k/(k!) Mk evaluation points for a r-times
differentiable function. This has been used up to several hundred dimensions.
It may be seen that smoothness of the function is taken into account very dif-
ferently by the various integration methods. The “roughest” is Monte Carlo, it only
feels the variance, quasi Monte Carlo feels the function’s variation, and the Smolyak
rules actually take smoothness into account.
A particular kind of Smolyak quadrature formulas is sparse Gauss-Hermite grids.
The algorithms for construction of sparse Gauss-Hermite grids are well known (e.g.,
[18]). An example of a sparse Gauss-Hermite grid (αi, Mai), i = 1..137, is shown in
Fig. 1). This figure demonstrates a 2D sparse Gauss-Hermite grid for the perturbed
angle of attack α
′
and the Mach number Ma
′
. Points of a sparse grid are collocation
Fig. 1 Sparse Gauss-Hermite grids for the perturbed angle of attack α
′
and the Mach number Ma
′
,
n = {13, 29, 137}.
points and in these points we compute deterministic code.
3 Karhunen-Lo`eve expansion
By definition, the Karhunen-Lo`eve expansion (KLE) of κ(x,ω) is the following
series [21]
κ(x,ω) = µκ(x)+
∞
∑
ℓ=1
λℓφℓ(x)ξℓ(ω), (4)
where ξℓ(ω) are uncorrelated random variables and µκ(x) = Eκ(x) is the mean value
of κ(x,ω), λℓ and φℓ are the eigenvalues and the eigenvectors of problem
Tφℓ = λℓφℓ, φℓ ∈ L2
(G ), ℓ ∈ N, (5)
and operator T is defined like follows
T : L2
(G ) → L2
(G ), (Tφ)(x) :=
G
covκ(x,y)φ(y)dy.

6. 6 Alexander Litvinenko and Hermann G. Matthies
For numerical purposes one truncates the KLE (4) to a finite number m of terms. In
the case of a Gaussian random field, the ξℓ are independent standard normal random
variables. In the case of a non-Gaussian random field, the ξℓ are uncorrelated but
not necessary independent. Dependent random variables can be approximated in a
set of new independent Gaussian random variables [11, 31], e.g.
ξℓ(ω) = ∑
α∈J
ξ
(α)
ℓ Hα(θ(ω)),
where θ(ω) = (θ1(ω),θ2(ω),...), ξ
(α)
ℓ are coefficients, Hα, α ∈ J , is a Hermitian
basis (see Appendix) and J := {α|α = (α1,...,αj,...), αj ∈ N0} a multi-index set.
For the purpose of actual computation, truncate the polynomial chaos expansion
(PCE) [11, 31] after finitely many terms.
After a finite element discretisation [17] the discrete eigenvalue problem (5)
looks like
MCMφℓ = λh
ℓ Mφℓ, Cij = covκ(xi,yj). (6)
Here the mass matrix M is stored in a usual data sparse format and the dense ma-
trix C ∈ Rn×n (requires O(n2) units of memory) is approximated in the sparse
H -matrix format [17] (requires only O(nlogn) units of memory) or in the Kro-
necker low-rank tensor format [16]. If not the complete spectrum is of interest, but
only a part of it then the needed computational resources can be drastically reduced
[1]. To compute m eigenvalues (m ≪ n) and corresponding eigenvectors we apply
an iterative Krylov subspace (Lanczos) eigenvalue solver for symmetric matrices
[20, 1, 19, 29]. This eigensolver requires only matrix-vector multiplications. All
matrix-vector multiplications are performed in the H -matrix format which will cost
O(nlogn), where n is number degrees of freedom.
For memory requirements and computing times of H -matrix approximations
see [17].
4 Polynomial Chaos expansion and response surface
As suggested by Wiener [31], any random variable (e.g. the lift coefficient CL) may
be represented as a series of polynomials in uncorrelated and independent Gaussian
variables θ = (θ1,...,θm), this is the polynomial chaos expansion (PCE) (see Ap-
pendix). For this representation we take Hermitian basis Hβ , β = (β1,...,βj,...) ∈
J a multiindex, J a multiindex set (see Appendix):
CL(θ) = ∑
β∈J
Hβ (θ)CLβ . (7)
Decomposition (7) can be understood as a response surface for CL. As soon as the
response surface is built, one can obtain the value CL(θ) for any θ almost for free
(only by evaluating the polynomial (7)). It can be very practical if one needs, e.g.

7. Numerical Methods for Uncertainty Quantification 7
106 evaluations of CL(·). Since Hermite polynomials are orthogonal, the coefficients
CLβ can be computed by projection:
CLβ =
1
β! Θ
Hβ (θ)CL(θ)dP(dθ), (8)
where Θ the Gaussian probability space. This multidimensional integral can be
computed approximately, for example, on a sparse Gauss-Hermite grid
CLβ =
1
β!
n
∑
i=1
Hβ (θi)CL(θi)wi, (9)
where weights wi and points θi are defined from sparse Gauss-Hermite integration
rule.
5 Statistical modelling of uncertainties
We have implemented two different strategies to research simultaneous propagation
of uncertainties in the angle of attack α and in the Mach number Ma. In the first
strategy (Section 5.1) we assumed that the distributions, the mean values and stan-
dard deviations for the random variables α and Ma are given. Then for each pair
αi and Mai of a sparse Gauss-Hermite grid we compute the solution with help of
the deterministic code (TAU code) as well as different statistical functionals of in-
terest. Since sparse Gauss-Hermite grid methods may be unstable, we compare the
obtained results with the results of Monte Carlo simulations. In the second strategy
(Section 5.2), we assume that the turbulence in the atmosphere randomly and simul-
taneously changes the velocity vector and the angle of attack (see Fig. 2). We model
turbulence in the atmosphere by two additionally axes-parallel velocity vectors v1
and v2, which have Gaussian distribution.
5.1 Distribution functions of α and Ma are given
In real-life applications distribution functions of the Mach number Ma and the angle
of attack α are unknown. As a start point we consider the uniform and the Gaussian
distributions. For our further numerical experiments we choose mean values and
standard deviations as in Table 5. The Reynolds number is Re = 6.5e + 6 and the
computational geometry is RAE-2822 airfoil.

8. 8 Alexander Litvinenko and Hermann G. Matthies
5.2 Modelling of turbulence in the atmosphere
In this section we model influence of uncertainties in the turbulence in the atmo-
sphere on the angle of attack α and on the Mach number (see Fig. 2). One should
not mix this kind of turbulence with the turbulence in the boundary layer reasoned
by friction. We assume that turbulence vortices in the atmosphere are comparable
with the size of the airplane.
We model the turbulence in the atmosphere via two vectors (in 2D)
α
v
v
u
u’
α’
v1
2
Fig. 2 Two random vectors v1 and v2 model turbulence in the atmosphere. Airfoil, the old and new
freestream velocities u and u
′
, the old and new angles of attack α and α
′
.
v1 =
σθ1
√
2
and v2 =
σθ2
√
2
. (10)
where θ1 and θ2 two Gaussian random variables with zero mean and unit variance,
σ = Iu∞ the standard deviation of the turbulent velocity fluctuations, I the mean
turbulence intensity and u∞ is the freestream velocity.
Denoting θ := θ2
1 +θ2
2 , then v := v2
1 +v2
2 = Iu∞√
2
θ and β := arctgv2
v1
. After
easy computations the new angle of attack will be as follows
α
′
= arctg
sinα +zsinβ
cosα −zcosβ
, where z :=
Iθ
√
2
(11)
and the new Mach number
Ma
′
= Ma 1 +
I2θ2
2
−
√
2Iθ cos(β +α). (12)
By default, in the TAU code, the mean turbulence intensity is I = 0.001.
Thus, alternatively to the way of modelling introduced in Sec. 5.1, uncertainties
in the angle of attack α
′
= α
′
(θ1,θ2) and in the Mach number Ma
′
= Ma
′
(θ1,θ2)
are described via two standard normal variables θ1 and θ2. This is the second way
of modelling.

9. Numerical Methods for Uncertainty Quantification 9
5.3 Uncertainties in geometry
We model uncertainties in the airfoil geometry G by the usage of a random field
κ(x,ω):
∂Gε(ω) = {x+εκ(x,ω)n(x) : x ∈ ∂G }, (13)
where ∂G is the surface of airfoil, n(x) a normal vector in point x and ε a small
parameter.
To generate Z realisations of airfoils with uncertainties (e.g., for MC or collocation
methods) we follow to the Algorithm below:
1. Assume the covariance function cov(·,·) for the random field κ(x,ω) is given
2. Compute (in a sparse data format!) Cij := cov(pi, pj) for all grid points
3. Solve eigenproblem (6)
4. Each random vector ξ = (ξ1(ω),...,ξm(ω)) in KLE
κ(x,ω) ≈ ∑m
i=1
√
λiφiξi(ω) results new airfoil.
Sparse approximation of the dense matrix C is done in [17, 16].
In Fig. 3 one can see 69 realisations of RAE-2822 airfoil with uncertainties in ge-
ometry. The largest uncertainties in the airfoil geometry are in the position x ≈ 0.58.
The used covariance function is of Gaussian type:
cov(p1, p2) = σ2
·exp(−ρ2
), (14)
where σ is a parameter, p1 = (x1,x2), p2 = (y1,y2) ∈ ∂G two points,
ρ(p1, p2) =
2
∑
i=1
|xi −yi|2/l2
i , and li are correlation length scales. (15)
The influence of uncertainties in the airfoil geometry on the solution will be re-
searched as follows:
Algorithm: (Computation and usage of the response surface)
1. Compute m eigenpairs of the discrete eigenvalue problem Eq. (6).
2. Generate a sparse Gauss-Hermite grid in m-dimensional space. Denote the num-
ber of grid points by N.
3. For each grid point θ = (θ1,...,θm) from item (2) compute KLE Eq. (4). Each
KLE is a new airfoil geometry γ(x,θ).
4. For each new airfoil solve the problem (call the TAU code).
5. Using the solution from item (4) and Hermite polynomials, build response sur-
face (Sec. 4).
6. Generate 106 points in m-dimensional space and evaluate response surface in
these points.
7. Using the values form item (6) compute statistical functionals of interest.
The response surface from item (5) will be as follows

10. 10 Alexander Litvinenko and Hermann G. Matthies
Fig. 3 69 RAE-2822 airfoils with uncertainties.
CL(θ1,...,θm) = ∑
β∈J
Hβ (θ1,...,θm)CLβ .
6 Data compression
A large number of stochastic realisations requires a large amount of memory and
computational resources. To decrease memory requirements and the computing time
we offer to use a low-rank approximation for all realisations of input and output ran-
dom fields. For each new realisation only corresponding low-rank update is com-
puted (see, e.g. [3]). It can be practical when, e.g. many thousands Monte Carlo
simulations are computed and stored.
Let vi ∈ Rn be the solution vector (already centred), where i = 1..Z a number
of stochastic realisations of the solution. Build from all these vectors the matrix
W = (v1,...,vZ) ∈ Rn×Z. Consider the factorization
W = ABT
where A ∈ Rn×k
and B ∈ RZ×k
. (16)
Definition 1. We say that matrix W is a rank-k matrix if the representation (16) is
given. We denote the class of all rank-k matrices for which factors A and BT in (16)
exist by R(k,n,Z). If W ∈ R(k,n,Z) we say that W has a low-rank representation.

11. Numerical Methods for Uncertainty Quantification 11
The first aim is to compute a rank-k approximation ˜W of W, such that
W − ˜W < ε, k ≪ min{n,Z}.
The second aim is to compute an update for the approximation ˜W with a linear
complexity for every new coming vector vZ+1. Below we present the algorithm
which does this.
To get the reduced singular value decomposition we omit all singular values,
which are smaller than some level ε or, alternative variant, we leave a fixed number
of largest singular values. After truncation we speak about reduced singular value
decomposition (denoted by rSVD) ˜W = ˜U ˜Σ ˜VT , where ˜U ∈ Rn×k contains the first k
columns of U, ˜V ∈ RZ×k contains the first k columns of V and ˜Σ ∈ Rk×k contains
the k-biggest singular values of Σ. There is theorem (see more in [24] or [6]) which
tells that matrix ˜W is the best approximation of W in the class of all rank-k matrices.
The computation of such basic statistics as the mean value, the variance, the ex-
ceedance probability can be done with a linear complexity. The following examples
illustrate computation of the mean value and the variance.
Let W = (v1,...,vZ) ∈ Rn×Z and its rank-k representation W = ABT , A ∈ Rn×k,
BT ∈ Rk×Z be given. Denote the j-th row of matrix A by aj ∈ Rk and the i-th col-
umn of matrix BT by bi ∈ Rk.
1. One can compute the mean solution v ∈ Rn as follows
v =
1
Z
Z
∑
i=1
vi =
1
Z
Z
∑
i=1
A·bi = Ab, (17)
The computational complexity is O(k(Z +n)), besides O(nZ)) for usual dense
data format.
2. One can compute the mean value of the solution in a grid point xj as follows
v(xj) =
1
Z
Z
∑
i=1
vi(xj) =
1
Z
Z
∑
i=1
aj ·bT
i = ajb. (18)
The computational complexity is O(kZ).
3. One can compute the variance of the solution var(v) ∈ Rn by the computing the
covariance matrix and taking its diagonal. First, we compute the centred matrix
Wc := W −WeT , where W = W · e/Z and e = (1,...,1)T . Computing Wc costs
O(k2(n + Z)) (addition and truncation of rank-k matrices). By definition, the
covariance matrix is cov = WcWT
c . The reduced singular value decomposition
of Wc is Wc = UΣVT , Σ ∈ Rk×k, can be computed with a linear complexity via
the QR algorithm (Section 6.1). Now, the covariance matrix can be written like
cov =
1
Z −1
WcWT
c =
1
Z −1
UΣVT
VΣT
UT
=
1
Z −1
UΣΣT
UT
. (19)
The variance of the solution vector (i.e. the diagonal of the covariance matrix in
(19)) can be computed with the complexity O(k2(Z +n)).

12. 12 Alexander Litvinenko and Hermann G. Matthies
4. One can compute the variance value var(v(xj)) in a grid point xj with a linear
computational cost.
5. To compute minimum or maximum of the solution in a point xj over all realisa-
tions cost O(kZ).
For further estimations of numerical complexity we recall the following theorem
Theorem 1. Table 1 shows storage requirements and computational complexities
for rank-k matrices.
Table 1 Storage requirements and computational complexities for rank-k matrices.
Operation Description Complexity
storage(W ) W ∈ R(k,n,Z), W = ABT k(n+Z)
Wx W ∈ R(k,n,Z),W = ABT ,x ∈ RZ 2k(n+Z)−n−k
W′ +W′′ W′ ∈ R(k′,n,Z),W′′ ∈ R(k′′,n,Z) (n+Z)(k′ +k′′)
W′W′′ W′ ∈ R(k′,n,Z),W′′ ∈ R(k′′,n,Z) 2k′k′′(n+Z)−k′′(n+k′)
rSVD(W ) W = ABT ∈ R(k,n,Z) 6k2(n+Z)+22k3
Truncate the rank k → k′ of W W ∈ R(k,n,Z) 6k2(n+Z)+22k3
Proof: see [9], [7], [2], [10].
6.1 Low-rank update with linear complexity
Let W = ABT ∈ Rn×Z and matrices A and B be given. An rSVD W = UΣVT can be
computed efficiently in three steps (QR algorithm for computing the reduced SVD):
1. Compute (reduced) a QR-factorization of A = QARA and B = QBRB, where
QA ∈ Rn×k, QB ∈ RZ×k, and upper triangular matrices RA,RB ∈ Rk×k.
2. Compute an reduced SVD of RART
B = U′ΣV′T .
3. Compute U := QAU′, V := QAV′T .
QR-decomposition can be done faster if a part of matrix A (or B) is orthogonal (see,
e.g. [3]). The first and third steps need O((n+Z)k2) operations and the second step
needs O(k3). The total complexity of rSVD is O((n +Z)k2 +k3).
Suppose we have already matrix W = ABT ∈ Rn×Z containing solution vectors. Sup-
pose also that matrix W
′
∈ Rn×m contains new m solution vectors. For the small
matrix W
′
, computing the factors C and DT
, such that W
′
= CDT
, is not expen-
sive. Now our purpose is to compute with a linear complexity the new matrix
Wnew ∈ Rn×(Z+m) in the rank-k format. To do this, we build two concatenated ma-
trices Anew := [AC] ∈ Rn×2k and BT
new = blockdiag[BT DT ] ∈ R2k×(Z+m). Note that
the difficulty now is that matrices Anew and Bnew have rank 2k. To truncate the rank
from 2k to k we use the QR-algorithm above. Obtain

13. Numerical Methods for Uncertainty Quantification 13
Wnew = UΣVT
= U(VΣT
)T
= AnewBT
new,
where Anew ∈ Rn×k and BT
new ∈ Rk×(Z+m). Thus, the “update” of the matrix W is
done with a linear complexity O((n +m+Z)k2
+k3
).
7 Numerics
We demonstrate the influence of uncertainties in the angle of attack, the Mach num-
ber and the airfoil geometry on the solution (the lift, drag, lift coefficient and skin
friction coefficient). As an example we consider two-dimensional RAE-2822 airfoil.
The deterministic solver is the TAU code with k-w turbulence model. We assume
that α and Ma are Gaussian with means α = 2.79, Ma = 0.734 and the standard
deviations σ(α) = 0.1 and σ(Ma) = 0.005.
In Fig.4 we compare the cumulative distribution and density functions for the lift
and drag, obtained via the response surface (PCE of order 1) and via 6360 Monte
Carlo simulations. To get a large sample we use sparse Gauss-Hermite grids (with 13
and 29 nodes) to build corresponding response surfaces and then perform 106 MC
evaluations on each response surface. Thus, one can see that very cheap collocation
method (13 or 29 deterministic evaluations) produces similar to MC method with
6360 simulations. But, at the same time we can not say which result is more precise.
The exact solution is unknown and 6360 MC simulations are too few.
The graphics in Fig. 5 demonstrate error bars [mean−σ,mean+σ], σ the stan-
dard deviation, for the pressure coefficient cp and absolute skin friction cf in each
surface point of the RAE2822 airfoil. The data are obtained from 645 realisation
of the solution. One can see that the largest error occur at the shock (x ≈ 0.6). A
possible explanation is that the shock position is expected to slightly change with
varying parameters α and Ma.

14. 14 Alexander Litvinenko and Hermann G. Matthies
Fig. 4 Density functions (first row), cumulative distribution functions (second row) of CL (left)
and CD (right). PCE is of order 1 with two random variables. Three graphics computed with 6360
MC simulations, 13 and 29 collocation points.
Fig. 5 Error bars [mean − σ, mean + σ], σ standard deviation, in each point of RAE2822 airfoil
for the cp and cf.

15. Numerical Methods for Uncertainty Quantification 15
To decrease memory requirements we write all Z = 645 realisations of the so-
lution as matrices ∈ R512×645 and compute their rank-k approximations. In Table
2 one can see dependence of the accuracy of the rank-k approximations (in the
spectral norm) on the rank k. Additionally, one can also see much smaller memory
requirement (dense matrix format costs 2.6MB). In the two last rows we compare
computing time needed for SVD-update Algorithm described in Section 6.1 with
the standard SVD of the global matrix ∈ R512×645. One can see that SVD-update
Algorithm performs faster.
Table 2 Accuracy, computing time and memory requirements of the rank-k approximation of the
solution matrices D = [density], P = [pressure], CP = [cp]; CF = [cf] ∈ R512×645.
rank k 2 5 10 20
D− ˜Dk 2/ D 2 6.6e-1 4.1e-2 3.5e-3 3.5e-4
P − ˜Pk 2/ P 2 6.9e-1 8.4e-2 8.2e-3 7.2e-4
CP − ˜CPk 2/ CP 2 6.0e-3 5.3e-4 3.2e-5 2.4e-6
CF − ˜CFk 2/ CF 2 9.0e-3 7.7e-4 4.6e-5 3.5e-6
memory, kB 18 46 92 185
SVD-update time, sec 0.58 0.60 0.62 0.68
usual SVD time, sec 0.55 0.63 2.6 3.8

16. 16 Alexander Litvinenko and Hermann G. Matthies
Fig. 6 demonstrates decay of 100 largest eigenvalues of four matrices, corre-
sponding to the pressure, density, pressure coefficient cp and absolute skin friction
cf. Each matrix belongs to the space R512×645
.
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
−20
−15
−10
−5
0
5
log, #eigenvalues
log,values
pressure
density
cp
cf
Fig. 6 Decay (in log-scales) of 100 largest eigenvalues of the matrices constructed from 645 solu-
tions (pressure, density, cf, cp) on the surface of RAE-2822 airfoil.
In Table 3 one can see dependence of the relative error (in the Frobenious norm)
on the rank k. Seven solution matrices contain pressure, density, turbulence kinetic
energy (tke), turbulence omega (to), eddy viscosity (ev), x-velocity (xv), z-velocity
(zv) in the whole computational domain with 260000 dofs. Additionally, one can
also see much smaller memory requirement (dense matrix format costs 1.25GB).
Table 3 Relative errors and memory requirements of rank-k approximations of the solution matri-
ces ∈ R260000×600. Memory required for the storage of each matrix in the dense matrix format is
1.25 GB.
rank k pressure density tke to ev xv zv memory,
MB
10 1.9e-2 1.9e-2 4.0e-3 1.4e-3 1.4e-3 1.1e-2 1.3e-2 21
20 1.4e-2 1.3e-2 5.9e-3 3.3e-4 4.1e-4 9.7e-3 1.1e-2 42
50 5.3e-3 5.1e-3 1.5e-4 9.1e-5 7.7e-5 3.4e-3 4.8e-3 104

17. Numerical Methods for Uncertainty Quantification 17
In Table 4 one can see corresponding computing times: time required for the
SVD-update Algorithm described in Section 6.1 and the time required for the stan-
dard SVD of the global matrix ∈ R260000×600
. A possible explanation for the large
computing time for the standard SVD is the lack of memory and expensive swap-
ping of data.
Table 4 Computing times (for Table 3) of rank-k approximations of the solution matrices ∈
R260000×600.
rank k SVD-update time, sec. usual SVD time, sec.
10 107 1537
20 150 2084
50 228 8236
7.1 α and Ma have Gaussian distribution
For further numerical experiments we choose mean values and standard deviations
as in Table 5.
Table 5 Mean values and standard deviations
mean st. deviation, σ σ/mean
Angle of attack, α 2.790 0.1 0.036
Mach number, Ma 0.734 0.005 0.007
Table 6 demonstrates application of sparse Gauss-Hermite two-dimensional grids
with n = {5, 13, 29} grid points. The Hermite polynomials (see Section 4) are of
order 1 with two random variables. In the last column of each table we compute
the measure of uncertainty σ/mean. For instance, for n = 5 it shows that 3.6% and
0.7% (Table 5) of uncertainties in α and in Ma correspondingly result in 2.1% and
15.1% of uncertainties in the lift and drag coefficients (Table 6). These three grids
show very similar results in the the mean value and in the standard deviation. At
the same time the results obtained via 1500 MC simulations are very similar to the
results computed on sparse Gauss-Hermite grids above.
Thus, one can make conclusion that the sparse Gauss-Hermite grid with a small
number of points, e.g. n = 13, produces similar to MC results.
Table 7 demonstrates statistics obtained for the case when random variables α
and Ma have uniform distribution. Comparing Table 7 with Table 6 one can see
that, in the case of uniform distribution of uncertain parameters, the uncertainties in

18. 18 Alexander Litvinenko and Hermann G. Matthies
Table 6 Uncertainties obtained on sparse Gauss-Hermite grids with 5 ,13, 29 points and with 1500
MC simulations.
n mean st. dev. σ σ/mean
5 CL 0.8530 0.0180 0.021
CD 0.0206 0.0031 0.151
13 CL 0.8530 0.0174 0.020
CD 0.0206 0.0030 0.146
29 CL 0.8520 0.0180 0.021
CD 0.0206 0.0031 0.151
MC 1500 CL 0.8525 0.0172 0.020
CD 0.0206 0.0030 0.146
the lift and drag coefficients are smaller. Namely, 1.2% and 8.8% (for CL and CD)
against 2% and 14.6% in the case of the Gaussian distribution. But, uncertainties
in the input parameters α and Ma, in the case of the uniform distribution, are also
smaller: 2.1% and 0.4% against 3.5% and 0.7%.
Table 7 Statistic obtained from 3800 MC simulations, α and Ma have uniform distribution.
mean st. dev. σ σ/mean
α 2.787 0.058 0.021
Ma 0.734 0.003 0.004
CL 0.853 0.0104 0.012
CD 0.0205 0.0018 0.088
7.2 α(θ1,θ2), Ma(θ1,θ2), where θ1, θ2 have Gaussian distributions
In this section we illustrate numerical results for the model described in Section 5.2.
Table 8 shows statistics (the mean value, variance and standard deviation), com-
puted on sparse Gauss-Hermite grids with n = 137 grid points.
Table 9 compares uncertainties computed on sparse Gauss-Hermite grids with
n = {137, 381, 645} nodes with the uncertainties computed by the MC method
(17000 simulations). All three grids and MC forecast very similar uncertainties
σ/mean in the drag coefficient CD and in the lift coefficient CL.
Table 10 compares the mean values computed on sparse Gauss-Hermite grids (n
nodes) with 17000 MC simulations. One can see that the errors are very small. But
this table tells only that sparse Gauss-Hermite grid with n points can be successfully
used to compute the mean values CL and CD.

19. Numerical Methods for Uncertainty Quantification 19
Table 8 Statistics obtained on sparse Gauss-Hermite grid with 137 points.
mean st. dev. σ σ/mean
α 2.8 0.2 0.071
Ma 0.73 0.0026 0.0036
CL 0.85 0.0373 0.044
CD 0.01871 0.00305 0.163
Table 9 Comparison of results obtained by a sparse Gauss-Hermite grid (n grid points) with 17000
MC simulations.
n 137 381 645 MC, 17000
σCL
CL
0.044 0.042 0.042 0.045
σCD
CD
0.163 0.159 0.16 0.1589
|CL−CL0|
CL
7.6e-4 1.3e-3 1.6e-3 4.2e-4
|CD−CD0|
CD
1.66e-2 1.46e-2 1.4e-2 2.1e-2
Table 10 Comparison of mean values obtained by MC simulations and by sparse Gauss-Hermite
grid with n grid points.
n 137 381 645
|CLn−CLMC|
CLMC
·100% 0.1% 0.1% 0.1%
|CDn−CDMC|
CDMC
·100% 0.4% 0.6% 0.7%
7.3 Uncertainties in the geometry
We follow the Algorithm described in Section 5.3. The number of KLE terms (4) is
m = 3. We assume that the covariance function is of Gaussian type
cov(p1, p2) = σ2
·exp(−d2
), d = |x1 −x2|2/l2
1 +|z1 −z2|2/l2
2,
where σ = 10−3
, p1 = (x1,0,z1), p2 = (x2,0,z2), the covariance lengths l1 =
|maxi(x)−mini(x)|/10 and l2 = |maxi(z)−mini(z)|/10. Stochastic dimension is 3
and number of sparse Gauss-Hermite points is 25. After the response surfaces for CL
and CD are built (see Section 4), we generated 106 MC points and evaluate values
of both response surfaces in these points. Table 11 demonstrate the computed statis-
tics. Surprisingly small are uncertainties in the CL and CD — 0.58% and 0.65%
correspondingly. A possible explanation can be a small uncertain perturbations in
the airfoil geometry.

20. 20 Alexander Litvinenko and Hermann G. Matthies
Table 11 Statistics obtained for uncertainties in the geometry. Gaussian covariance function was
used. PCE of order 1 with 3 random variables. Sparse Gauss-Hermite grid contains 25 points.
mean st. dev. σ σ/mean
CL 0.8552 0.0049 0.0058
CD 0.0183 0.00012 0.0065
8 Conclusion
In this work we research how uncertainties in the input parameters (the angle of
attack α and the Mach number Ma) and in the airfoil geometry influence the solution
(lift, drag, pressure coefficient and absolute skin friction). Uncertainties in the Mach
number and in the angle of attack weakly affect the lift coefficient (1% − 3%) and
strongly affect the drag coefficient (around 14%). Uncertainties in the geometry
influence both the lift and drag coefficients weakly (less that 1%), but changes in
the geometry were also very small. Results obtained via collocation method on a
sparse Gauss-Hermite grid are comparable with Monte Carlo results, but require
much less deterministic evaluations (and as a sequence - smaller computing time).
To compare the results computed on a sparse Gauss-Hermite grids we used MC
simulations. We note that to get reliable results with Monte Carlo methods one
should perform 105
-107
simulations, but it is impossible to do in a reasonable time
(1 simulation with TAU code requires at least 10000 iterations and takes between 20
and 80 minutes). We performed 17000 simulations and this allows us to make only
approximate comparison.
To make statistical computational more efficient (linear complexity and linear
storage besides quadratic or even cubic) an additional research in this work was de-
voted to the low-rank approximation of the results. We found out that all realisations
of the solution can be approximated and stored in the low-rank format. This format
allows us to compute all important statistics with linear complexity and drastically
reduces memory requirements.
Acknowledgements It is acknowledged that this research has been conducted within the project
MUNA under the framework of the German Luftfahrtforschungsprogramm funded by the Ministry
of Economics (BMWA). The authors would like also to thank Elmar Zandler for his matlab package
“Stochastic Galerkin library” [32].
9 Appendix A — Multi-Indices
In the above formulation, the need for multi-indices of arbitrary length arises. For-
mally they may be defined by [23]
β = (β1,...,βj,...) ∈ J := N
(N)
0 , (20)

21. Numerical Methods for Uncertainty Quantification 21
which are sequences of non-negative integers, only finitely many of which are non-
zero. As by definition 0! := 1, the following expressions are well defined:
|β| :=
∞
∑
j=1
βj, (21)
β! :=
∞
∏
j=1
βj!, (22)
ℓ(β) := max{ j ∈ N|βj > 0}. (23)
9.1 Appendix B — Hermite Polynomials
As there are different ways to define—and to normalise—the Hermite polynomials,
a specific way has to be chosen. In applications with probability theory it seems
most advantageous to use the following definition [14, 11, 12, 13, 22]:
hk(t) := (−1)k
et2/2 d
dt
k
e−t2/2
; ∀t ∈ R, k ∈ N0,
where the coefficient of the highest power of t —which is tk for hk —is equal to
unity.
The first five polynomials are:
h0(t) = 1, h1(t) = t, h2(t) = t2
−1,
h3(t) = t3
−3t, h4(t) = t4
−6t2
+3.
The recursion relation for these polynomials is
hk+1(t) = t hk(t)−khk−1(t); k ∈ N.
These are orthogonal polynomials w.r.t standard Gaussian probability measure Γ ,
where Γ (dt) = (2π)−1/2e−t2/2 dt —the set {hk(t)/
√
k!|k ∈ N0} forms a complete
orthonormal system (CONS) in L2(R,Γ ) —as the Hermite polynomials satisfy
∞
−∞
hm(t)hn(t)Γ (dt) = n!δnm.
Multi-variate Hermite polynomials will be defined for an infinite number of vari-
ables, i.e. for t = (t1,t2,...,tj,...) ∈ RN, the space of all sequences. This uses the
multi-indices defined in Appendix A. For α = (α1,...,αj,...) ∈ J remember that
except for a finite number all other αj are zero; hence in the definition of the multi-
variate Hermite polynomial

22. 22 Alexander Litvinenko and Hermann G. Matthies
Hα(t) :=
∞
∏
j=1
hαj (tj); ∀t ∈ RN
, α ∈ J ,
except for finitely many factors all others are h0, which equals unity, and the infinite
product is really a finite one and well defined.
The space RN
can be equipped with a Gaussian (product) measure, again de-
noted by Γ . Then the set {Hα(t)/
√
α!| α ∈ J } is a CONS in L2(RN,Γ ) as the
multivariate Hermite polynomials satisfy
RN
Hα(t)Hβ (t)Γ (dt) = α!δαβ ,
where the Kronecker symbol is extended to δαβ = 1 in case α = β and zero other-
wise.
References
1. Z. Bai, J. Demmel, J. Dongarra, A. Ruhe, and H. van der Vorst, editors. Templates for the
solution of algebraic eigenvalue problems - A practical guide, volume 11 of Software, Envi-
ronments, and Tools. Society for Industrial and Applied Mathematics (SIAM), Philadelphia,
PA, 2000.
2. S. B¨orm, L. Grasedyck, and W. Hackbusch. Hierarchical Matrices, volume 21 of Lecture
Note. Max-Planck Institute for Mathematics, Leipzig, 2003. www.mis.mpg.de.
3. Matthew Brand. Fast low-rank modifications of the thin singular value decomposition. Linear
Algebra Appl., 415(1):20–30, 2006.
4. H.-J. Bungartz and M. Griebel. Sparse grids. Acta Numer., 13:147–269, 2004.
5. Th. Gerstner and M. Griebel. Numerical integration using sparse grids. Numer. Algorithms,
18(3-4):209–232, 1998.
6. G. H. Golub and Ch. F. Van Loan. Matrix computations. Johns Hopkins Studies in the
Mathematical Sciences. Johns Hopkins University Press, Baltimore, MD, third edition, 1996.
7. L. Grasedyck. Theorie und anwendungen hierarchischer matrizen. Ph.D. Thesis, University
of Kiel, Germany, 2001.
8. M. Griebel. Sparse grids and related approximation schemes for higher dimensional problems.
In Foundations of computational mathematics, Santander 2005, volume 331 of London Math.
Soc. Lecture Note Ser., pages 106–161. Cambridge Univ. Press, Cambridge, 2006.
9. W. Hackbusch. A sparse matrix arithmetic based on H -matrices. I. Introduction to H -
matrices. Computing, 62(2):89–108, 1999.
10. W. Hackbusch. Hierarchische Matrizen - Algorithmen und Analysis. Springer, 2009.
11. T. Hida, Hui-Hsiung Kuo, J. Potthoff, and L. Streit. White noise - An infinite-dimensional
calculus, volume 253 of Mathematics and its Applications. Kluwer Academic Publishers
Group, Dordrecht, 1993.
12. H. Holden, B. Øksendal, J. Ubøe, and T. Zhang. Stochastic partial differential equations.
Probability and its Applications. Birkh¨auser Boston Inc., Boston, MA, 1996. A modeling,
white noise functional approach.
13. S. Janson. Gaussian Hilbert spaces, volume 129 of Cambridge Tracts in Mathematics. Cam-
bridge University Press, Cambridge, 1997.
14. G. Kallianpur. Stochastic filtering theory, volume 13 of Applications of Mathematics.
Springer-Verlag, New York, 1980.

23. Numerical Methods for Uncertainty Quantification 23
15. A. Keese. Numerical solution of systems with stochastic uncertainties. A general purpose
framework for stochastic finite elements. Ph.D. Thesis, TU Braunschweig, Germany, 2004.
16. B. N. Khoromskij and A. Litvinenko. Data sparse computation of the Karhunen-Lo`eve expan-
sion. Numerical Analysis and Applied Mathematics: International Conference on Numerical
Analysis and Applied Mathematics, AIP Conf. Proc., 1048(1):311–314, 2008.
17. B. N. Khoromskij, A. Litvinenko, and H. G. Matthies. Application of hierarchical matrices
for computing the Karhunen-Lo`eve expansion. Computing, 84(1-2):49–67, 2009.
18. A. Klimke. Sparse grid interpolation toolbox: www.ians.uni-stuttgart.de/spinterp/. 2008.
19. C. Lanczos. An iteration method for the solution of the eigenvalue problem of linear differen-
tial and integral operators. J. Research Nat. Bur. Standards, 45:255–282, 1950.
20. R. B. Lehoucq, D. C. Sorensen, and C. Yang. ARPACK users’ guide. Solution of large-scale
eigenvalue problems with implicitly restarted Arnoldi methods., volume 6 of Software, Envi-
ronments, and Tools. Society for Industrial and Applied Mathematics (SIAM), Philadelphia,
PA, 1998.
21. M. Lo`eve. Probability theory I. Graduate Texts in Mathematics, Vol. 45, 46. Springer-Verlag,
New York, fourth edition, 1977.
22. Paul Malliavin. Stochastic analysis, volume 313 of Grundlehren der Mathematischen Wis-
senschaften [Fundamental Principles of Mathematical Sciences]. Springer-Verlag, Berlin,
1997.
23. H. G. Matthies. Uncertainty quantification with stochastic finite elements. 2007. Part 1.
Fundamentals. Encyclopedia of Computational Mechanics.
24. L. Mirsky. Symmetric gauge functions and unitarily invariant norms. Quart. J. Math. Oxford
Ser. (2), 11:50–59, 1960.
25. E. Novak and K. Ritter. The curse of dimension and a universal method for numerical integra-
tion. In Multivariate approximation and splines (Mannheim, 1996), volume 125 of Internat.
Ser. Numer. Math., pages 177–187. Birkh¨auser, Basel, 1997.
26. E. Novak and K. Ritter. Simple cubature formulas with high polynomial exactness. Constr.
Approx., 15(4):499–522, 1999.
27. K. Petras. Smolpack—a software for Smolyak quadrature with delayed Clenshaw-Curtis
basis-sequence. http://www-public.tu-bs.de:8080/ petras/software.html.
28. K. Petras. Fast calculation of coefficients in the Smolyak algorithm. Numer. Algorithms,
26:93–109, 2001.
29. Y. Saad. Numerical methods for large eigenvalue problems. Algorithms and Architectures for
Advanced Scientific Computing. Manchester University Press, Manchester, 1992.
30. S. A. Smolyak. Quadrature and interpolation formulas for tensor products of certain classes
of functions. Sov. Math. Dokl., 4:240–243, 1963.
31. N. Wiener. The homogeneous chaos. American Journal of Mathematics, 60:897–936, 1938.
32. E. Zander. A malab/octave toolbox for stochastic galerkin methods:
http://ezander.github.com/sglib/. 2008.