1. Attractors of Distribution
Generalized Central limit theorem and Stable
distribution
Xiong Wang 王雄
Centre for Chaos and Complex Networks
City University of Hong Kong
1
2. Outline
Normal distribution
most prominent probability distribution
in simple system.
Central limit theorem
Why normal distribution is so normal
Power Law
most prominent probability
distribution in complex system.
Generalized central limit theorem
Stable distribution: Attractor family of 2
distributions
6. Normal distribution
where
parameter μ
is the mean
or
expectation
(location of
the peak)
and σ 2 is the
variance, the
mean of the
squared
deviation,
6
7. 3-sigma rule
about 99.7% are within three standard
deviations
7
10. Central limit theorem
The central limit
theorem states that the
sum of a number of
independent and
identically distributed
(i.i.d.) random
variables with finite
variances will tend to a
normal distribution as
the number of
variables grows.
C:chaosTalklevyIllustratingTheCentralLimitThe
http://demonstrations.wolfram.com/IllustratingTheCentralLimitTheoremWith
10
SumsOfUniformAndExpone/
11. Other distributions can be
approximated by the normal
The binomial distribution B(n, p) is
approximately normal N(np, np(1 − p)) for
large n and for p not too close to zero or one.
The Poisson(λ) distribution is approximately
normal N(λ, λ) for large values of λ.
The chi-squared distribution χ2(k) is
approximately normal N(k, 2k) for large ks.
The Student's t-distribution t(ν) is
approximately normal N(0, 1) when ν is large.
11
12. Galton Board
If the probability of bouncing right on a pin is p (which
equals 0.5 on an unbiased machine) the probability that
the ball ends up in the kth bin equals
According to the central limit theorem the binomial
distribution approximates the normal distribution
provided that n, the number of rows of pins in the
machine, is large.
http://www.youtube.com/watch?v=xDIyAOBa_yU
C:chaosTalklevyIllustratingTheCentralLimitTheoremWithSumsOf
BernoulliRandomV.cdf
12
13. Principle of maximum entropy
According to the principle of maximum
entropy, if nothing is known about a
distribution except that it belongs to a certain
class, then the distribution with the largest
entropy should be chosen as the default.
13
14. Another viewpoint
For a given mean and variance, the
corresponding normal distribution is the
continuous distribution with the maximum
entropy.
Therefore, the assumption of normality
imposes the minimal prior structural
constraint beyond these moments
14
15. Summary of normal
distribution
First, the normal distribution is very tractable
analytically, that is, a large number of results
involving this distribution can be derived in
explicit form.
15
16. Summary of normal
distribution
Second, the normal distribution arises as the
outcome of the central limit theorem, which
states that under mild conditions the sum of a
large number of random variables is
distributed approximately normally.
Finally, the "bell" shape of the normal
distribution makes it a convenient choice for
modelling a large variety of random variables
encountered in practice.
16
21. Normal vs Power law
You can hardly find a person twice as tall as
you
Fair enough…
This is normal distribution
But you can easily find a person 10000 times
richer than you…
Extremely unfair…
This is power law distribution
21
22. Examples of power laws
a. Word frequency: Estoup.
b. Citations of scientific papers: Price.
c. Web hits: Adamic and Huberman
d. Copies of books sold.
e. Diameter of moon craters: Neukum & Ivanov.
f. Intensity of solar flares: Lu and Hamilton.
g. Intensity of wars: Small and Singer.
h. Wealth of the richest people.
i. Frequencies of family names: e.g. US & Japan not
Korea.
j. Populations of cities.
24. The Power Law Phenomenon
Power Law
Bell Curve Distribution
Many nodes with
Most nodes few links
have the same
No. of nodes
No. of nodes
with k links
with k links
number of
links
# of links (k) # of links
No highly (k) A few nodes
connected nodes with many links
26. Attraction basin of Gaussian
The central limit theorem states that the sum
of a number of independent and identically
distributed (i.i.d.) random variables with finite
variances will tend to a normal distribution as
the number of variables grows.
All distribution with finite variance form the
attraction basin of Gaunssian.
what about the distribution having infinite variance?
26
31. Cauchy–Lorentz distribution
PDF
Characteristic function
Observe that the characteristic function is not
differentiable at the origin: So the Cauchy
distribution does not have an expected value 31
or Variance.
32. Generalized central limit
theorem
A generalization due to Gnedenko and
Kolmogorov states that the sum of a number
of random variables with power-law tail
distributions decreasing as 1 / | x | α + 1 where
0 < α < 2 (and therefore having infinite
variance) will tend to a stable distribution
f(x;α,0,c,0) as the number of variables grows.
32
33. Stable distribution
In probability theory, a random variable is
said to be stable (or to have a stable
distribution) if it has the property that a linear
combination of two independent copies of the
variable has the same distribution, up to
location and scale parameters.
The stable distribution family is also
sometimes referred to as the Lévy alpha-
stable distribution.
33
34. Such distributions form a four-parameter
family of continuous probability distributions
parametrized by location and scale
parameters μ and c, respectively, and two
shape parameters β and α, roughly
corresponding to measures of asymmetry
and concentration, respectively (see the
figures).
C:chaosTalklevyStableDensityFunction.cdf
38. Unified normal and power law
For α = 2 the distribution reduces to a Gaussian
distribution with variance σ2 = 2c2 and mean μ; the
skewness parameter β has no effect
The asymptotic behavior is described, for α < 2
38
39. Log-log plot of skewed centered stable distribution PDF's showing the
power law behavior for large x. Again the slope of the linear portions
is equal to -(α+1)
40. Concluding Remarks
The importance of stable probability
distributions is that they are "attractors" for
properly normed sums of independent and
identically-distributed (iid) random variables.
The normal distribution is one family of stable
distributions.
Without the finite variance assumption the limit
may be a stable distribution, which has the
power law behavior for large x.
40
41. Analogy
Chen’s attractor family Stable distribution family
At first, Lorenz attractor At first, normal
was found distribution was found
For a long time, this was thought as the only story…
Then a question raised naturally…
Could there be any extension?
Then a family of Then a family of attractor
attractors were found of distribution were found
which unified Lorenz and which unified normal
Chen attractor distribution and power
law
42. Xiong Wang 王雄
Centre for Chaos and Complex Networks
City University of Hong Kong
Email: wangxiong8686@gmail.com
42