1. The Journal of Systems and Software 77 (2005) 103–112
www.elsevier.com/locate/jss
Bayesian network based software reliability prediction with
an operational profile q
Cheng-Gang Bai *
Department of Automatic Control, Beihang University (Beijing University of Aeronautics and Astronautics), Beijing 100083, China
Received 29 June 2004; received in revised form 11 November 2004; accepted 26 November 2004
Available online 24 December 2004
Abstract
This paper uses a Bayesian network to model software reliability prediction with an operational profile. Due to the complexity of
software products and development processes, software reliability models need to possess the ability to deal with multiple param-
eters. A Bayesian network exhibits a strong ability to adapt to problems involving complex variant factors. A special kind of Bayes-
ian network named a Markov Bayesian network has been applied successfully into modeling software reliability prediction.
However, the existing research did not pay enough attention to the fact that the failure characteristics of many software systems
often depend on the specific operation performed. In this paper, an extended Markov Bayesian network is developed to model soft-
ware reliability prediction with an operational profile. The extended Markov Bayesian network proposed in the paper is focused on
discrete-time failure data. Methods to solve the network are proposed, and an example is used to illustrate the utilization of the
model.
Ó 2004 Elsevier Inc. All rights reserved.
Keywords: Software reliability; Software failure rate; Operational profile; Bayesian network; Markov Bayesian network; Discrete-time failure data;
Software run
1. Introduction such as the Jelinski–Moranda model (Jelinski and Mor-
anda, 1972) and the Goel–Okumoto NHPP model (Goel
Research on how to predict software reliability accu- and Okumoto, 1979). This kind of model suggests that a
rately is of great practical importance. Many software stochastic process, especially a non-homogeneous Pois-
reliability prediction models have been developed and son process (NHPP) (Boland and Singh, 2002, 2003),
described in the literature (Goel, 1985; Xie, 1991, can describe the failure process of software.
1993), among which many classic probability models However, many of these models make unrealistic
are widely discussed. For some recent references, see, restrictive assumptions to ensure tractability and solv-
e.g. (Cai, 1998b; Sahinoglu et al., 2001; Malaiya et al., ability (Bishop and Pullen, 1988; Lyu, 1996), such as
2002; Zheng, 2002; Ozekici and Soyer, 2003; Reussner removing defects completely (no new defects introduced
et al., 2003). The traditional software reliability predic- when removing the one detected), or data completeness
tion models mainly belong to stochastic process models, (no missing data). These restrictive assumptions have
limited the development and applications of these
probability models in recent years, and as a result
q
researchers have begun to explore other approaches.
Supported by the National Natural Science Foundation of China
(Grant No. 60473067 and Grant No. 60233020).
The assumptions have been relaxed by making no distri-
*
Tel.: +861082315087; fax: +861082317328. bution assumptions on the failure process (Pfefferman
E-mail address: bcg@buaa.edu.cn and Cemuschi-Frias, 2002), but this advantage is
0164-1212/$ - see front matter Ó 2004 Elsevier Inc. All rights reserved.
doi:10.1016/j.jss.2004.11.034

2. 104 C.-G. Bai / The Journal of Systems and Software 77 (2005) 103–112
Nomenclature
ð1Þ
{D, S, P} denote Bayesian network Hj the failure rate of Oj at the beginning of test-
D the set of variables (or nodes) of Bayesian ing process j = 1, 2, . . ., m
ðiÞ
network Hj the failure rate of Oj after the (i À 1)th run,
S the set of conditional probability distribu- i = 2, . . . , n + 1, j = 1, 2, . . ., m
tion(CPD) of Bayesian network Oj the j th operation, j = 1,2, . . ., m
P the set of marginal probability distributions Ai the ith run, Ai = j if the jth operation Oj is
of Bayesian network carried out at ith run, i = 1,2, . . ., n
p(X) if X is a continuous variable, it denotes the Qi the operational profile for the ith run,
probability density function of X; if X is a i = 1,2, . . ., n
discrete variable, it denotes the probability Qn+1 the operational profile for the possible next
distribution of X run after n runs
MBN Markov Bayesian network Q operational profile if it is invariant
Xi(1 6 i 6 n) the remaining defects number after the Zi the result of the i th run, Zi = 1 if a failure
ith failure occurs at ith run, and otherwise Zi = 0, i =
Yi(1 6 i 6 n) the time between the (i À 1)th and ith 1, 2, . . ., n
failure d denotes the decrement of failure rate after a
EMBN extended Markov Bayesian network failure occurs and the failure-causing defects
n number of software testing runs are removed
m number of operations Beta(Æ ja, b) denotes Beta distribution with para-
f failure order meters a,b
gf the number of runs from the fth failure to Dir(Æ jq1, . . . , qm) denotes Dirichlet distribution with
(f + 1)th failure parameters q1, . . ., qm
R the software reliability after n runs
compensated by the added noise in the software reliabil- atively new technique, the Bayesian network, can be ap-
ity prediction results. Non-probability models, e.g. neu- plied to model and solve such problems. Bayesian
ral networks, have been applied (Karunanithi and networks can describe many factors of the system simul-
Whitley, 1992; Khoshgoftaar and Szabo, 1996; Sitte, taneously and also can express the interrelations be-
1999; Cai et al., 2001). Recent research has shown some tween them efficiently. It combines precise probability
advantage of neural networks in predictive capabilities distributions and prior expertise information in a way
over the traditional NHPP model (Ho et al., 2003). that is more suitable to solving problems with many
However, there are still some problems with this ap- uncertainties. Due to these desirable properties, Bayes-
proach. First, due to the randomness of the network ian networks have been increasingly applied in many
structure, it is difficult to express the interrelationships fields (Bickmore, 1994; Madigan et al., 1996; Lewis
between the input variables. Second, it is necessary to and Ransing, 1997; Castillo et al., 1999; Zhu, 2003),
ensure the integrity of the neural network, which means although this approach has found applications in reli-
that if the data is absent in some input nodes, it would ability only in the last seven years (Sigurdsson et al.,
be difficult for the neural network to process effectively. 2001). A Bayesian framework combining diverse sources
Amongst many unreasonable assumptions in most of evidence was developed to assess system dependabil-
traditional models, one of the most serious limitations ity (Neil et al., 1996; Bouissou et al., 1999). It has been
is the reliance on past failure data, alone, to predict soft- used to predict software reliability in the early phases of
ware reliability. The future failure time not only relies on the development by incorporating information ahead of
the past failure data, but also relies on other variables testing (Smidts et al., 1998) and to develop a causal
such as how the software is used. model for software defect rates prediction (Fenton and
Models of such complex problems as software reli- Neil, 1999). It has also been combined with fault trees
ability prediction in large software systems should incor- for reliability analysis (Pai, 2001), and used for the cer-
porate prior information and available expert tification of COTS (Commercial Off-The-Shelf) software
experience. The Bayesian method provides a proper tool reliability (Yu and Johnson, 2003). In our previous work
to deal with this problem (Kuo and Yang, 1995,1996; (Bai et al., 2005), a software failure prediction model
Kuo and Lee, 1997; Basu and Ebrahimi, 2003). How- based on Markov Bayesian networks was developed.
ever, it is difficult for traditional Bayesian models to deal However, an undesirable restriction in almost all of
with problems involving complex variant factors. A rel- the software reliability models in the literature is that

3. C.-G. Bai / The Journal of Systems and Software 77 (2005) 103–112 105
the parameters of the software failure process are inde- (3) P is a set of marginal probability distributions.
pendent of the operation that the software performs P = {p(D)jD 2 D} stands for the probability distri-
(Ozekici and Soyer, 2003). Musa (1993, 1996) describes bution of variable D.
how operational profiles can be built and states that
there is substantial benefit to be gained by applying In the Bayesian network, variables are used to ex-
them. An operational profile describes how users em- press the events or objects. The problem could be mod-
ploy a system (Musa, 1993). There has been some work eled with the behavior of these variables. In general, we
on the topic of software reliability with an operational first calculate (or determine from expert experience) the
profile. Cai (1998a) developed various methods to model probability distribution of each variable and the condi-
operational profiles. Ozekici and Soyer (2003) provide a tional probability distribution between them. Then from
stochastic and statistical framework to model software these distributions we can obtain the joint distributions
reliability in the presence of an operational profile. Their of these variables. Finally, some deductions can be
method is based on the assumption that software reli- developed for some variables of interest using some
ability behavior can be measured in terms of continuous other known variables.
time. However, many systems depart from this assump- A Bayesian network is generally based on Bayesian
tion (Cai, 2000). For example, the reliability behavior of statistical theory. Compared with other traditional data
a bank transaction processing software system should be analysis methods, a Bayesian network has the following
measured in terms of how many transactions are suc- properties:
cessful, rather than how long the software system oper-
ates without failure. Cai (2003) discussed how to assess (1) A Bayesian network can be connected to all the
software reliability with an operational profile during variables of interest, so it is expected to increase
the software validation phase. In the validation phase, the precision of the analysis. For example, in a
it is assumed that no detected defects are removed. problem with two input and one output variables,
Different from these earlier works, this paper de- the two input variables might be correlated. If both
scribes an extended Markov Bayesian network that of the input variables can be observed, most of the
can deal with software reliability prediction with an models will be able to predict the output. However,
operational profile. This paper is organized as follows. if one of the inputs cannot be observed, then few
Section 2 presents the background of the paper. In Sec- models can provide a precise prediction, and the
tion 3, we develop an extended Markov Bayesian Net- correlation between these two variables is not used.
work model for software reliability prediction. In On the contrary, it is natural for a Bayesian network
Section 4, we discuss the techniques for solving the to measure and use this kind of correlation.
model under various assumptions. In Section 5, a case (2) A Bayesian network can model the causal relation-
study is given with real software failure data, and ship between variables. Thus, the interdependence
satisfactory results are obtained. of the variables could be studied in order to improve
prediction. For example, one market analyst may
want to know whether it is worthwhile to increase
2. Background the intensity of some advertising campaign in order
to gain more sales of a certain product. To solve this
2.1. Bayesian network problem, the analyst needs to know whether the
advertisement is an influencing factor, and the
The definition of a Bayesian network can be found in extent of its influence. Bayesian networks can help
many versions, but the basic form (Pearl, 1986) is stated to solve problems like this.
as follows: a Bayesian network is a directed probability
graph, connecting the relative variables with arcs, and A Bayesian network with Bayesian statistical meth-
this kind of connection expresses the conditional depen- ods can combine the relative knowledge in the problem
dence between the variables. The formal definition domain and the available data effectively. No one with
follows. practical experiences can deny the importance of the
A Bayesian network is defined as the set of {D, S, P}, prior information or the problem domain knowledge,
where especially in circumstances where little or no data are
available or when data collection is costly.
(1) D is a set of variables (or nodes).
(2) S is a set of conditional probability distributions 2.2. Bayesian network based software reliability
(CPD). prediction model
S = {p(DjParents(D))jD 2 D}, Parents(D) & D stands
for all the parent nodes for D, p(DjParents(D)) is the Let Xi (1 6 i 6 n) denote the number of remaining
conditional distribution of variable D. defects after the ith failure, and Yi (1 6 i 6 n) the time

4. 106 C.-G. Bai / The Journal of Systems and Software 77 (2005) 103–112
X1 X2 X3 Xn 3. Bayesian network model with an operational profile
… 3.1. Extended Markov Bayesian network
Assume that there are m operations and n runs. Let
… Oj = j denote the jth operation of them operations. Let
Ai (i = 1, . . . , n) denote the ith run. Ai = j (i = 1, . . . , n)
if the jth operation Oj is carried out at ith run. Let Qi
Y1 Y2 Y2 Yn
(i = 1, . . . , n) denote operational profile at ith run
Fig. 1. A Markov Bayesian network (MBN). n o
ðiÞ ðiÞ
Qi ¼ qj j qj ¼ Pr½Ai ¼ jŠ ð1 6 j 6 mÞ
ði ¼ 1; . . . ; nÞ:
between the (i À 1)th and ith failure. During the defects Let
removal process, it is possible to both remove and intro-
duce defects, and hence Xi+1 is dependent on Xi. The 1 if a failure occurs at the i th run
Zi ¼ ði ¼ 1; . . . ; nÞ
time interval between software failures is influenced 0 if no failure occurs at the ith run
by the number of remaining defects. Assuming that
p(xi+1jxi) and p(yijxi) are independent of time, then a ð1Þ
Let H j ðj ¼ 1; . . . ; mÞ denote the failure rate of Oj at the
Markov Bayesian network can be used to model this ðiÞ
beginning of the testing. Let H j ði ¼ 2; . . . ; n þ 1;
problem (Jordan et al., 1999). The network graph is j ¼ 1; . . . ; mÞ denote the failure rate of Oj after the
given in Fig. 1 (Bai et al., 2005). (i À 1)th run. Comparing with our previous work (Bai
et al., 2005), we use failure rate in this study rather than
2.3. Operational profile the number of remaining defects. The main reasons are
(1) it is easier to get the relationship between failure rate
Musa (1993) said Ôa profile is simply a set of disjoint and testing results than that between the number of
(only one can occur at a time) alternatives with the prob- remaining defects and testing results; (2) it is easier to
ability that each will occur.Õ An operational profile sim- predict software reliability based on the failure rate
ply consists of the set of all operations that a system is rather than on remaining defects. So, failure rate is
designed to perform and their probabilities of occur- much more interesting.
rence. It provides a quantitative characterization of Let R denote the software reliability after the total n
how the system will be used in the field, making it an runs. Qn+1 denotes the operational profile after the total
essential ingredient of software reliability engineering. n runs.
Technically speaking, one can think of an operational Fig. 2 shows the extended Markov Bayesian network
profile as a generic random variable that indicates the (EMBN).
operations that will be performed by the system (Ozekici
and Soyer, 2003). 3.2. Determining the distributions
2.4. Software run In order to use the EMBN model to predict software
reliability, the following distributions are essential:
The Poisson model (Bai et al., 2005) is based on the ð1Þ ð1Þ
assumption that software reliability behavior can be (1) The initial distributions of H j . pðH j Þ ðj ¼ 1;
measured in terms of calendar time, clock time or . . . ; mÞ. These could be estimated using software
CPU execution time. Although this assumption is metrics from the development of the software prod-
appropriate for a wide scope of systems, there are many uct, or be given by experts.
systems, which depart from this assumption. For exam- (2) The initial distributions of Qi (i = 1, . . . , n + 1). Qi is
ple, the reliability behavior of a bank transaction pro- the operational profile at the ith run. It can be
cessing software system should be measured in terms assumed that the initial distributions of Qi can be
of how many transactions are successful, rather than gotten from historical data.
of how long the software system operates without failure (3) The conditional distribution of Zi (i = 1, . . . , n)
(Cai, 2000). h i
ðiÞ ðiÞ ðjÞ
Cai (2000) proposed the conception of ‘‘a run’’ to de- Pr Z i ¼ 1 j Ai ; H 1 ; . . . ; H ðiÞ ¼ H j Á qi
m
scribe discrete time. A run is minimum execution unit of
h i
software. Any software execution process can be divided ðiÞ ðiÞ ðjÞ
Pr Z i ¼ 0 j Ai ; H 1 ; . . . ; H ðiÞ ¼ 1 À H j Á qi
m
into a series of runs.

5. C.-G. Bai / The Journal of Systems and Software 77 (2005) 103–112 107
H1 (1) H1
(2)
H1
(n)
H1(n+1) 3.3. Software reliability prediction
… Although Eq. (1) is useful in evaluating the prediction
probabilities, the densities in Eq. (1) can be difficult to
compute directly. Here, we propose to use the Gibbs
sampling approach to approximate Eq. (1). According
…
…
…
…
…
to the Gibbs sampling theory (Gelfand and Smith,
(2) 1985), the key point is to sample from the conditional
Hm(1) Hm Hm(n) Hm
(n+1)
distributions of every variable under condition that all
… other variables are known.
ðkÞ
(1) H l
( ðkÞ )
… ðiÞ
H l j R; H j ði ¼ 1;...;n þ 1; j ¼ 1;...;m; ði;jÞ 6¼ ðk;lÞÞ
p
Z1 Z2 Zn Ai ði ¼ 1;...;nÞ; Z i ði ¼ 1;...;nÞ; Qi ði ¼ 1;...;n þ 1Þ
R
( ðiÞ )
R; H j ði ¼ 1;...;n þ 1; j ¼ 1;...;mÞ; Ai ði ¼ 1;...;nÞ
/p
A1 A2 An Z i ði ¼ 1;...;nÞ; Qi ði ¼ 1;...;n þ 1Þ
…
Y
nþ1 Y
n
¼ pfR j Qnþ1 ; H nþ1 ðj ¼ 1;...;mÞg Á
j pfQi g Á pfAi j Qi g
i¼1 i¼1
Y
m
ð1Þ
YY
nþ1 m
ðiÞ ðiÀ1Þ
Á pfH j g Á pfH j j H j ; AiÀ1 ; Z iÀ1 g
… j¼1 iÀ2 j¼1
Q1 Q2 Qn Qn+1 Y
n
ðiÞ
Á pfZ i j H j ðj ¼ 1;...;mÞ; Ai g
Fig. 2. Extended Markov Bayesian network.
i¼1
if 1 k n + 1
(4) The conditional distribution of Ai (i = 1, . . . , n) ( ðkÞ ðiÞ )
ðiÞ ðiÞ
H l j R;H j ði ¼ 1;...;n þ 1; j ¼ 1;...;m; ði;jÞ 6¼ ðk;lÞÞ
pfAi ¼ j j Qi ¼ fqj ; j ¼ 1; . . . ; mgg ¼ qj p
ðiÞ Ai ði ¼ 1;...;nÞ; Z i ði ¼ 1;...;nÞ; Qi ði ¼ 1;...;n þ 1Þ
(5) The conditional distribution of H j ði ¼ 2; . . . ; n þ 1Þ.
ðkÞ ðkÀ1Þ ðkÞ
Cai (2000) proposed five models to describe the / pfH l j H l ;AkÀ1 ;Z kÀ1 g Á pfZ k j H l ;Ak g
changing trend about hazard rate function (the prob-
ðkþ1Þ ðkÞ
ability that the software fails). Therefore, the 5 mod- Á pfH l j H l ;Ak ;Z k g
els can be applied into this step. For example, when
ðiþ1Þ ðiÞ if k = 1
Ai = j 0 , it is feasible to let H j0 ¼ H j0 À dZ i ;
ðiþ1Þ ðiÞ 0 ( ðkÞ ðiÞ
)
Hj ¼ H j ; j 6¼ j . H l j H j ði ¼ 1; . . . ; n; j ¼ 1; . . . ; m; ði; jÞ 6¼ ðk; lÞÞ
(6) The conditional distribution of R p
X ðnþ1Þ ðnþ1Þ
m Ai ði ¼ 1; . . . ; nÞ; Z i ði ¼ 1; . . . ; nÞ; Qi ði ¼ 1; . . . ; nÞ
R¼1À H j qj ð1Þ ð1Þ ð2Þ ð1Þ
j¼1 / pfH l g Á pfZ 1 j H l ; A1 g Á pfH l j H l ; A1 ; Z 1 g
As in standard Bayesian network, the joint probabil- if k = n + 1
ðiÞ
ity density of R, H j ði ¼ 1; . . . ; n þ 1; j ¼ 1; . . . ; mÞ, Ai ( ðkÞ ðiÞ )
H l j R;H j ði ¼ 1; . . . ; n þ 1; j ¼ 1; . . . ; m; ði; jÞ 6¼ ðk; lÞÞ
(i = 1, . . . , n), Zi (i = 1, . . . , n), Qi (i = 1, . . . , n + 1) is p
( ) Ai ði ¼ 1; .. . ; nÞ; Z i ði ¼ 1;. . . ; nÞ; Qi ði ¼ 1;. . . ; n þ 1Þ
ðiÞ
R; H j ði ¼ 1; . . . ; n þ 1; j ¼ 1; . . . ; mÞ; Ai ði ¼ 1; . . . ; nÞ;
p
Z i ði ¼ 1; . . . ; nÞ; Qi ði ¼ 1; . . . ; n þ 1Þ / pfH l
ðnþ1Þ ðnÞ
j H l ; An ; Z n g
Y
nþ1 Y
n
¼ pfR j Qnþ1 ; H nþ1 ðj ¼ 1; . . . ; mÞg Á
j pfQi g Á pfAi j Qi g So, the distribution
i¼1 i¼1
( ðkÞ ðiÞ
)
Y
m
ð1Þ
YY
nþ1 m
ðiÞ ðiÀ1Þ H l j R; H j ði ¼ 1;...;n þ 1; j ¼ 1;...;m; ði;jÞ 6¼ ðk;lÞÞ
Á pfH j g Á pfH j j H j ; AiÀ1 ; Z iÀ1 g p
j¼1 iÀ2 j¼1
Ai ði ¼ 1;...;nÞ; Z i ði ¼ 1;...;nÞ; Qi ði ¼ 1;...;n þ 1Þ
Y
n
ðiÞ
Á pfZ i j H j ðj ¼ 1; . . . ; mÞ; Ai g ð1Þ
i¼1 can be sampled.

6. 108 C.-G. Bai / The Journal of Systems and Software 77 (2005) 103–112
(2) Zk. If k n (4) Qk
( )
8 9 ðiÞ
Z k j R; H ðiÞ ði ¼ 1;. ..; n þ 1; j ¼ 1;.. .;mÞ = Qk j R;H j ði ¼ 1; .. .; n þ 1; j ¼ 1; .. .; mÞ
j p
p Ai ði ¼ 1; .. .; nÞ; Z i ði ¼ 1; .. .; nÞ; Qi ði ¼ 1;. .. ;n þ 1; i 6¼ kÞ
: ; ( )
Ai ði ¼ 1;.. .;nÞ; Z i ði ¼ 1; ... ;n;i 6¼ kÞ; Qi ði ¼ 1;.. .;n þ 1Þ ðiÞ
R; H j ði ¼ 1;. .. ;n þ 1; j ¼ 1;. .. ;mÞ; Ai ði ¼ 1; .. .; nÞ
8 9 /p
R; H ðiÞ ði ¼ 1;. ..; n þ 1; j ¼ 1;.. .;mÞ; Ai ði ¼ 1;.. .;nÞ = Z i ði ¼ 1;. .. ;nÞ; Qi ði ¼ 1; .. .; n þ 1Þ
j
/p Y
nþ1 Y
n
: ; ¼ pfR j Qnþ1 ; H nþ1 ðj ¼ 1; .. .; mÞg Á pfQi g Á pfAi j Qi g
Z i ði ¼ 1;. ..; nÞ; Qi ði ¼ 1;.. .;n þ 1Þ j
i¼1 i¼1
Y
nþ1 Y
n Y
m
ð1Þ
YY
nþ1 m
ðiÞ ðiÀ1Þ
¼ pfR j Qnþ1 ; H nþ1 ðj ¼ 1; ... ;mÞg Á pfQi g Á pfAi j Qi g Á pfH j g Á pfH j j H j ; AiÀ1 ; Z iÀ1 g
j
i¼1 i¼1 j¼1 iÀ2 j¼1
Y
n
ðiÞ
Y
m YY
nþ1 m Á pfZ i j H j ðj ¼ 1;. .. ;mÞ; Ai g
ð1Þ ðiÞ ðiÀ1Þ
Á pfH j g Á pfH j j Hj ; AiÀ1 ; Z iÀ1 g i¼1
j¼1 iÀ2 j¼1
if k n + 1
Y
n ( ðiÞ
)
ðiÞ Qk j R; H j ði ¼ 1; . . . ; n þ 1; j ¼ 1; .. . ; mÞ
Á pfZ i j H j ðj ¼ 1;.. .;mÞ; Ai g p
i¼1 Ai ði ¼ 1; . . . ;nÞ; Z i ði ¼ 1; . .. ; nÞ; Qi ði ¼ 1; . . . ;n þ 1; i 6¼ kÞ
ðiÞ
/ pfZ i j H j ðj ¼ 1;. ..; mÞ; Ai g Á pfH Ak
ðkþ1Þ ðkÞ
j H Ak ; A k ; Z k g / pfQk g Á pfAk j Qk g
if k = n + 1
So, the distribution ( )
ðiÞ
Qk j R; H j ði ¼ 1; .. . ; n þ 1; j ¼ 1; . . . ; mÞ
8 9 p
Z k j R; H ðiÞ ði ¼ 1;. ..;n þ 1; j ¼ 1;...; mÞ
j
= Ai ði ¼ 1; . . .; nÞ; Z i ði ¼ 1; . . . ; nÞ; Qi ði ¼ 1; . . . ; n þ 1; i 6¼ kÞ
p
: ; nþ1
Ai ði ¼ 1;...; nÞ; Z i ði ¼ 1; ...;n;i 6¼ kÞ; Qi ði ¼ 1; ...;n þ 1Þ / pfR j Qnþ1 ; H j ðj ¼ 1; . . . ; mÞg Á pfQnþ1 g
So, the distribution
can be sampled. ( )
ðiÞ
Qk j R; H j ði ¼ 1; . . . ; n þ 1; j ¼ 1; . . . ; mÞ
(3) Ak p
8 9 Ai ði ¼ 1; . . . ; nÞ; Z i ði ¼ 1; . . . ; nÞ; Qi ði ¼ 1; . . . ; n þ 1; i 6¼ kÞ
Ak j R; H ðiÞ ði ¼ 1; . . . ; n þ 1; j ¼ 1; . . . ; mÞ
j
=
p can be sampled.
: ;
Ai ði ¼ 1; . . . ; n; i 6¼ kÞ; Z i ði ¼ 1; . .. ; nÞ; Qi ði ¼ 1; . . . ; n þ 1Þ (5) R
8 9 ( ðiÞ
)
R; H ðiÞ ði ¼ 1; . . . ; n þ 1; j ¼ 1; .. . ; mÞ; Ai ði ¼ 1; . . .; nÞ =
j
R j H j ði ¼ 1;...;n þ 1; j ¼ 1;...;mÞ
/p p
: ; Ai ði ¼ 1;...;nÞ; Z i ði ¼ 1;...;nÞ; Qi ði ¼ 1;...;n þ 1Þ
Z i ði ¼ 1; .. . ; nÞ; Qi ði ¼ 1; . . . ; n þ 1Þ ( )
ðiÞ
R;H j ði ¼ 1;...;n þ 1; j ¼ 1;...;mÞ; Ai ði ¼ 1;...;nÞ
Y
nþ1 Y
n /p
¼ pfR j Qnþ1 ; H nþ1 ðj ¼ 1; . . . ;mÞg Á
j pfQi g Á pfAi j Qi g Z i ði ¼ 1;...;nÞ; Qi ði ¼ 1;...;n þ 1Þ
i¼1 i¼1
Y
nþ1 Y
n
Y
m YY
nþ1 m ¼ pfR j Qnþ1 ;H nþ1 ðj ¼ 1;...;mÞg Á
j pfQi g Á pfAi j Qi g
ð1Þ ðiÞ ðiÀ1Þ
Á pfH j g Á pfH j j H j ; AiÀ1 ; Z iÀ1 g i¼1 i¼1
j¼1 iÀ2 j¼1
Y
m
ð1Þ
YY
nþ1 m
ðiÞ ðiÀ1Þ
Y
n Á pfH j g Á pfH j j H j ; AiÀ1 ; Z iÀ1 g
ðiÞ
Á pfZ i j H j ðj ¼ 1; .. . ; mÞ; Ai g / pfAk j Qk g j¼1 iÀ2 j¼1
i¼1
Y
n
ðiÞ
ðkþ1Þ ðkÞ ðkÞ
Á pfZ i j H j ðj ¼ 1;...;mÞ; Ai g
Á pfH j j H j ; Ak ; Z k g Á pfZ k j H Ak ; Ak g i¼1
¼ pfR j Qnþ1 ;H nþ1 ðj ¼ 1;...;mÞg
j
So, the distribution
8 9 So, the distribution
Ak j R; H ðiÞ ði ¼ 1; . . . ; n þ 1; j ¼ 1; . . . ; mÞ
j
= ( )
ðiÞ
p R j H j ði ¼ 1; . . . ; n þ 1; j ¼ 1; . . . ; mÞ
: ; p
Ai ði ¼ 1; . . . ; n; i 6¼ kÞ; Z i ði ¼ 1; . . . ; nÞ; Qi ði ¼ 1; . . . ; n þ 1Þ Ai ði ¼ 1; . . . ; nÞ; Z i ði ¼ 1; . . . ; nÞ; Qi ði ¼ 1; . . . ; n þ 1Þ
can be sampled. can be sampled.

7. C.-G. Bai / The Journal of Systems and Software 77 (2005) 103–112 109
From the above discussion it is derived that the (15) Predict the software reliability
conditional distributions of every variable can be Xm
ðnþ1Þ
sampled. Therefore, if we know the values of any R¼1À qj pðH j Þ
subset of all variables in the model, the others can j¼1
be inferred by the Gibbs sampling approach. We (16) End.
call this algorithm the Gibbs sampling-based
Algorithm.
Although the time which the algorithm consumes is 4. A case study
manageable, if all Ai (i = 1, . . . , n) , Zi (i = 1, . . . , n)
are known, a simpler algorithm to predict software We apply this model to software known as the Space
reliability can be derived by a message-passing program. Rothermel et al. (2001) describe the program
scheme (Pearl, 1986). We call this the Simplified as follows: ‘‘Space consists of 9,564 lines of C code
Algorithm. (6,218 executable), and functions as an interpreter for
Assume that operational profile is invariable, that is an array definition language (ADL). The program reads
Q1 = Q2 = Á Á Á a file that contains several ADL statements, and checks
So, let Q = {qjjqj = Pr[Ai = j], (1 6 j 6 m)}. the contents of the file for adherence to the ADL gram-
Given Ai = ai (i = 1, . . . , n), Zi = zi (i = 1, . . . , n), the mar and to specific consistency rules. If the ADL file is
simplified algorithm is as follows: correct, space outputs an array data file containing a list
of array elements, positions, and excitations; otherwise,
(1) Give Ai = ai(i = 1, . . . , n), Zi = zi(i = 1, . . . , n) the program outputs an error message’’. We apply
ð1Þ ð1Þ
(2) Give the prior distributions of H j ) pðH j Þ, EMBN to this data set to predict the software reliability
(j = 1, . . . , m) behavior in this program.
(3) Give the prior distribution of Q = {qj,j = 1, . . . , With the assumption that the initial failure rates of
m})p(Q) each operation follow a Beta distribution and the oper-
(4) 1 ) i ational profile follows a Dirichlet distribution, we apply
ðiÞ ðiÞ ðiÞ
(5) pðH j Þ ) p ðH j Þ, p ðH j Þ (j = 1, . . . , m) denote the model for software reliability prediction.
ðiÞ
the prior distribution of H j . Assume that
ðiÞ
(6) Calculate the posterior distribution of H j ðj ¼ ð1Þ ðiÞ
ðiÞ H j $ Betaðaj ; bðiÞ Þ;
j Q $ Dirððq1 ; . . . ; qm Þ
1; . . . ; mÞ by Bayesian rule. Let hðH j j Z i ; Ai Þ
ðiÞ j q1 ; . . . ; qm Þ
denote the posterior distribution of H j ,
respectively.
where Beta(a, b) is Beta distribution with parameters a,
ðiÞ ðiÞ ðiÞ b.
hðH j j Z i ¼ zi ; Ai ¼ ai Þ / pfZ i ¼ zi j H j gpfH j g j ¼ ai
ðiÞ ðiÞ Dir((q1, . . . , qm)jq1, . . . , qm) is a Dirichlet distribution,
hðH j j Z i ¼ zi ; Ai ¼ ai Þ ¼ pfH j g j 6¼ ai
if
ðiÞ ðiÞ
(7) hðH j j Z i ¼ zi ; Ai ¼ ai Þ ) pðH j Þ pðq1 ;q2 ; .. . ;qm j q1 ; q2 ; . .. ;qm Þ
ðiþ1Þ
(8) Calculate the distribution of H j (j = 1, . . . , m) by 8 m
C Pq
!
ðiþ1Þ ðiÞ
j¼1 j Q qj À1 m Pm
Hj ¼ H j À zi d; j ¼ ai
Qm qi ; 0 6 pi 6 1; i ¼ 1;. . .; m; pj ¼ 1
¼
and j¼1 Cðqj Þ j¼1
j¼1
ðiþ1Þ ðiÞ
:
Hj ¼ Hj ; j 6¼ ai 0 otherwise
ðiþ1Þ
Let pðH j Þ (j = 1, . . . , m) denote the distribution The simplified algorithm is adopted in this case study.
of
ðiþ1Þ
Hj : For step (6) of the algorithm, we have
(9) Calculate the posterior distribution of Q by ð1Þ ð1Þ ð1Þ
H j j Z 1 ¼ z1 ; A1 ¼ a1 $ Betaðaj þ z1 ; bj þ 1 À z1 Þ;
Bayesian rule.
j ¼ ai
hðQ j Ai ¼ ai Þ / pðAi ¼ ai j QÞpðQÞ
ðiþ1Þ
(10) h(QjAi = ai) ) p(Q) In order to keep H j still a Beta distribution, we use
ðiþ1Þ ðiÞ ðiþ1Þ ðiÞ
(11) If i = n, then go (12); else i + 1 ) i and go (5) EH j ¼ EH j À d other than H j ¼ H j À d in the
ðnþ1Þ step (8). From this we have
(12) Output pðH j Þ (j = 1, . . . , m)
!
(13) Output p(Q) ð1Þ ð1Þ ð1Þ
aj þ z1 À dz1 ðaj þ bj þ 1Þ
ð2Þ
(14) Estimate the operational profile: Q = {qjjj = 1, . . . , H j $ Beta ð1Þ
m} from p(Q) bð1Þ þ 1 À z1 þ dz1 ðaj þ bj þ 1Þ
j
ð1Þ

8. 110 C.-G. Bai / The Journal of Systems and Software 77 (2005) 103–112
So, Table 1
( ð2Þ ð1Þ ð1Þ ð1Þ
Software reliability prediction by EMBN model
aj ¼ aj þ z1 À dz1 ðaj þ bj þ 1Þ Failure Operational profile Software reliability
j ¼ a1
ð2Þ ð1Þ order f
bj ¼ bð1Þ
j þ 1 À z1 þ dz1 ðaj þ ð1Þ
bj þ 1Þ For Set A For Set B
( 1 0.1995, 0.3112, 0.4894 0.5001 0.6967
ð2Þ ð1Þ 2 0.1991, 0.3115, 0.4894 0.8571 0.8889
aj ¼ aj
j 6¼ a1 3 0.1991, 0.3112, 0.4897 0.9231 0.9286
bj ¼ bð1Þ
ð2Þ
j 4 0.1995, 0.3112, 0.4894 0.9444 0.9347
5 0.1991, 0.3112, 0.4897 0.9600 0.9583
Furthermore 6 0.1991, 0.3115, 0.4894 0.9643 0.9630
( ðiþ1Þ ðiÞ ðiÞ 7 0.1991, 0.3112, 0.4897 0.9697 0.9688
aj ¼ aj þ zi À dzi ðaj þ bðiÞ þ 1Þ
j 8 0.1991, 0.3112, 0.4897 0.9744 0.9730
ðiÞ
j ¼ ai 9 0.1991, 0.3112, 0.4897 0.9773 0.9762
bj ¼ bðiÞ þ 1 À zi þ dzi ðaj þ bðiÞ þ 1Þ
ð2Þ
j j 10 0.1995, 0.3112, 0.4894 0.9792 0.9787
( 11 0.1995, 0.3112, 0.4894 0.9815 0.9808
ðiþ1Þ ðiÞ
aj ¼ aj 12 0.1991, 0.3115, 0.4894 0.9833 0.9828
ðiþ1Þ
j 6¼ ai 13 0.1991, 0.3112, 0.4897 0.9848 0.9844
bj ¼ bðiÞ
j 14 0.1991, 0.3112, 0.4897 0.9861 0.9855
15 0.1991, 0.3112, 0.4897 0.9882 0.9878
In the step (9), let 16 0.1991, 0.3112, 0.4897 0.9891 0.9888
17 0.1991, 0.3115, 0.4894 0.9901 0.9897
ð1Þ ðjÞ ðmÞ
hðQ j Ai ¼ ai Þ $ Dir qi ; . . . ; qi þ 1; . . . ; qi j ¼ aj 18 0.1991, 0.3115, 0.4894 0.9907 0.9903
19 0.1991, 0.3115, 0.4894 0.9915 0.9912
20 0.1995, 0.3112, 0.4894 0.9922 0.9919
and assume that q1 = 2, q2 = 2, q3 = 2, d = 0.0002.
21 0.1991, 0.3112, 0.4897 0.9927 0.9925
In this study, it is interesting to see to what extent the 22 0.1991, 0.3112, 0.4897 0.9932 0.9930
prediction is dependent on different initial values of the 23 0.1991, 0.3112, 0.4894 0.9937 0.9935
parameters. Two kind of initial values of parameters are 24 0.1991, 0.3112, 0.4897 0.9941 0.9939
as follows: 25 0.1991, 0.3112, 0.4897 0.9945 0.9944
26 0.1991, 0.3112, 0.4897 0.9949 0.9948
Set A: a11 ¼ 1; b11 ¼ 1; a12 ¼ 1; b12 ¼ 1; a13 ¼ 1; b13 ¼ 1:
Set B: a11 ¼ 1; b11 ¼ 3; a12 ¼ 2; b12 ¼ 2; a13 ¼ 1; b13 ¼ 4:
In order to evaluate the performance of the proposed
The software is tested for 3000 runs in total, and there model, we compare our results with those of the Kaani-
are three operations (m = 3). Twenty-six failures occur che–Kanoun model (KK model) (Kaaniche and
in total during testing. Because we are interested in the Kanoun, 1992). The KK model is one of the few dis-
trend of the software reliability prediction along with crete software reliability models. Most discrete software
the testing process, all the software reliability predic- reliability models assume that all design faults can be
tions for n = 1, . . . , 3000 are calculated and studied. removed from the software. However, this is not typical
However, for the sake of space limitation, only those of most large software systems. The KK model, as well
data with failures occurring are listed in Table 1. as ours, does not need this assumption. Due to space
Table 1 shows the results of software reliability by limitation, only the parameter Set A is used for our
EMBN model. The serial numbers of failure order are model. Table 2 shows the failure interval (the number
listed in the first column with the title ‘‘Failure Order’’. of runs from one failure to the next failure) predictions
The estimations of operational profile after f failures oc- for each of the models, and Fig. 3 illustrates the results.
curred are listed in the second column. Table 1 shows From the results, we see that the EMBN model has good
the estimations of operational profile basically keep con- estimation and prediction capabilities. From the results,
stant, especially when f is large. The software reliability we can see that when the failure order is small, the KK
prediction for parameters values Set A and Set B are model is unable to predict the failure intervals. In
listed in the third and the fourth column, respectively. contrast, the EMBN model is always able to make pre-
Table 1 shows that at the beginning of the test the third dictions. It seems that better predictions can be achieved
column is quite different from the fourth column. That by combining the prior experience and the failure data
means that at the beginning of the test the prior infor- from software testing. However, in the very late stage
mation play a key role in predicting software reliability. of software testing the behaviors of software are sud-
It tells us that if the number of test runs is not very big denly changed. This means that it is difficult to find
the prior distribution should be selected carefully. How- more defects by the current software testing methods.
ever, the trend of software reliability becomes stable In this situation, both of the two methods fail to give
regardless of the initial values of the parameters. That proper predictions. If we want to find more defects,
means that if the number of test runs is large, assuming the software testing methods should be changed (Bai
different prior distributions donÕt make much difference. et al., 2003). This means that software testing profiles

9. C.-G. Bai / The Journal of Systems and Software 77 (2005) 103–112 111
Table 2 obtain, so prior information should be incorporated.
The SPACE failure data (* means no value) For this, the Bayesian approach seems appropriate
Failure Failure interval Predictions by Predictions by (Pham and Pham, 2001). From this starting point, a
order f data gf EMBN model KK Model Bayesian network model is proposed to predict software
1 5 2 * reliability. This is desirable for complex software sys-
2 14 7 * tems where prior information may be available from ex-
3 16 13 *
4 27 18 *
perts or former development processes and experience.
5 17 25 20 However, Bayesian network models require the user
6 31 28 19 to develop prior distributions. There exist several ways
7 34 33 20 to do this, such as relying on expert opinion. The expert
8 35 39 25 opinion includes empirical experience with other similar
9 37 44 28
10 41 48 31
software, knowledge of software testing, and other
11 54 54 38 background information. Although specific assumptions
12 58 60 44 were made in the development of our model, the initial
13 59 66 50 distributions can also be obtained by combining infor-
14 122 72 54 mation gained in the former development process.
15 79 85 67
16 99 92 73
Moreover, other distributions can be assumed to reflect
17 79 101 83 different testing configurations.
18 135 107 89 Bayesian models often involve high complexity and
19 136 117 99 computational effort. As a result, the suggested algo-
20 137 128 110 rithm, the Gibbs sampling-based Algorithm, would be
21 142 137 116
22 163 147 126
time consuming. However, it is unreasonable to expect
23 185 158 138 that a simple model would be able to model a complex
24 193 170 150 system. A very simple algorithm for a special case, which
25 224 183 164 often happens in practice, is proposed in the paper.
26 865 197 178 Beta distributions are adopted as priors for the case
study. The Beta distribution has many interesting prop-
erties. The most important property of the Beta distribu-
900 tion is that it is conjugated with the binomial
distribution. Thus the Beta distribution fits very well
800
for the variables that are defined on (0, 1). However,
700 studies with other distributions present avenues for fur-
Failure Data ther research.
600
ηf
500
400 Acknowledgment
ENBN Model
300 The author would like to thank the two anonymous
K K Model
200 referees and the editor for providing helpful comments.
100
0 References
0 5 10 15 20 25 30
f
Bai, C.G., Hu, Q.P., Xie, M., 2005. Software failure prediction based
Fig. 3. Comparison between EMBN model and KK model. on a Markov Bayesian network model. Journal of Systems and
Software 74 (3), 275–282.
Bai, C.G., Cai, K.Y., Chen, T.Y., 2003. An efficient defect estimation
method for software defect curves. In: Proceedings of the 27th
will be changed. Due to the complexity of this problem,
Annual International Computer Software and Applications Con-
it is difficult to solve in the current paper. However, this ference, Dallas, USA, pp. 534–539.
problem will be studied further in the future. Basu, S., Ebrahimi, N., 2003. Bayesian software reliability models
based on Martingale processes. Technometrics 45 (2), 150–158.
Bickmore, T.W., 1994. Real-time sensor data validation. NASA
Contractor Report 195295, National Aeronautics and Space
5. Conclusion
Administration.
Bishop, P., Pullen, F.D., 1988. Probabilistic modeling of software
For most safety-critical software with high reliability failure characteristics. in: Proceedings of the IFAC Workshop
requirements, an adequate sample of failures is hard to SAFECOMPÕ88, pp. 87–93.

10. 112 C.-G. Bai / The Journal of Systems and Software 77 (2005) 103–112
Boland, P.J., Singh, H., 2002. Determining the optimal release time for Lewis, R.W., Ransing, R.S., 1997. A semantically constrained Bayes-
software in the geometric Poisson reliability model. International ian network for manufacturing diagnosis. International Journal of
Journal of Reliability, Quality and Safety Engineering 9 (3), 201– Production Research 35, 2171–2187.
213. Lyu, M.R. (Ed.), 1996. Handbook of Software Reliability Engineering.
Boland, P.J., Singh, H., 2003. A birth-process approach to MorandaÕs McGraw-Hill, New York.
geometric software-reliability model. IEEE Transactions on Reli- Madigan, D., Mosurski, K., Almond, R.G., 1996. Explanation in
ability 52 (2), 168–174. belief networks. Journal of Computational and Graphical Statistics
Bouissou, M., Martin, F., Ourghanlian, A., 1999. Assessment of a 6, 160–187.
safety-critical system including software: A Bayesian belief network Malaiya, Y.K., Li, M.N., Bieman, J.M, 2002. Software reliability
for evidence sources. in: Proceedings of the Annual Reliability and growth with test coverage. IEEE Transactions on Reliability 51 (4),
Maintainability Symposium, pp. 142–150. 420–426.
Cai, K.Y., 1998a. Software defect and operational profile modeling. Musa, J.D., 1993. Operational profiles in software reliability engineer-
Kluwer Academic Publishers, Dordrecht. ing. IEEE Software 10 (1993), 14–32.
Cai, K.Y., 1998b. On estimating the number of defects remaining in Musa, J.D., 1996. The operational profile. In: Ozekici (Ed.), Reliability
software. Journal of Systems and Software 40, 93–114. and Maintenance of Complex Systems, NATO ASI Series, vol.
Cai, K.Y., 2000. Towards a conceptual framework of software run F154. Springer, Berlin, pp. 332–343.
reliability modeling. Information Sciences 126 (1–4), 137–163. Neil, M., Littlewood, B., Fenton, N., 1996. Applying Bayesian belief
Cai,, K.Y., 2003. In: Proceedings of the 3rd International Conference networks to system dependability assessment. in: Proceedings
on Quality Software. IEEE Computer Society Press, Silver Spring, of Safety Critical Systems Club Symposium, Springer-Verlag,
MD, pp. 32–39. Berlin.
Cai, K.Y., Cai, L., Wang, W.D., 2001. On the neural network Ozekici, S., Soyer, R., 2003. Reliability of software with an operational
approach in software reliability modeling. Journal of Systems and profile. European Journal of Operational Research 149 (2), 459–
Software 58, 47–62. 474.
Castillo, E., Sarabia, J.M., Solares, C., 1999. Uncertainty analyses in Pai, G.J., 2001. Combining Bayesian belief networks with fault tree to
fault trees and Bayesian networks using FORM SORM methods. enhance software reliability analysis. in: Proceedings of the IEEE
Reliability Engineering and Systems Safety 65, 29–40. International Symposium on Software Reliability Engineering.
Fenton, N., Neil, M., 1999. Software metrics: successes, failures and Pearl, J., 1986. Fusion, propagation and structuring in belief network.
new directions. Journal of Systems and Software 47, 149–157. Artificial Intelligence 29, 241–288.
Gelfand, A., Smith, A., 1985. Sampling based approaches to calculat- Pfefferman, J.D., Cemuschi-Frias, B., 2002. A non-parametric non-
ing marginal densities. Journal of the American Statistical Asso- stationary procedure for failure prediction. IEEE Transactions on
ciation 85, 385–409. Reliability 51 (4), 434–442.
Goel, A.L., 1985. Software reliability models: assumptions, limitations, Pham, L., Pham, H., 2001. A Bayesian predictive software reliability
and applicability. IEEE Transactions on Software Engineering SE- model with pseudo-failures. IEEE Transactions on Systems, Man
11, 1411–1423. and Cybernetics 31 (3), 233–238.
Goel, A.L., Okumoto, K., 1979. Time dependent error detection rate Reussner, R.H., Schmidt, H.W., Poernomo, I.H., 2003. Reliability
model for software reliability and other performance measure. prediction for component-based software architectures. Journal of
IEEE Transactions on Reliability 28 (3), 206–211. Systems and Software 66 (3), 241–252.
Ho, S.L., Xie, M., Goh, T.N., 2003. A study of the connectionist Rothermel, G., Untch, R., Chu, C., Harrold, M.J., 2001. Prioritizing
models for software reliability prediction. Computers and Math- test cases for regression testing. IEEE Transactions on Software
ematics with Application 46 (7), 1037–1045. Engineering 27, 929–948.
Jelinski, Z., Moranda, P.B., 1972. Software reliability research. In: Sahinoglu, M., Deely, J.J., Capar, S., 2001. Stochastic Bayes measures
Proceedings of the Statistical Methods for the Evaluation of to compare forecast accuracy of software-reliability models. IEEE
Computer System Performance. Academic Press, New York, pp. Transactions on Reliability 50 (1), 92–97.
465–484. Sigurdsson, J.H., Walls, L.A., Quigley, J.L., 2001. Bayesian belief nets
Jordan, M.I., Ghahramani, Z., Jaakkola, T.S., Saul, L.K., 1999. An for managing expert judgment and modeling reliability. Quality
introduction to variational methods for graphical models. In: and Reliability Engineering International 17, 181–190.
Jordan, M. (Ed.), Learning in Graphical Models. MIT Press, Sitte, R., 1999. Comparison of software-reliability-growth predictions:
Cambridge, MA. neural networks vs parametric-recalibration. IEEE Transactions
Kaaniche, M., Kanoun, K., 1992. The discrete-time hyperexponential on Reliability 48, 285–291.
model for software reliability growth evaluation. in: Proceedings of Smidts, C., Stoddard, R.W., Stutzke, M., 1998. Software reliability
the Third International Symposium on Software Reliability Engi- models: an approach to early reliability prediction. IEEE Trans-
neering, pp. 64–75. actions on Reliability 47, 268–278.
Karunanithi, N., Whitley, D., 1992. Prediction of software reliability Xie, M., 1991. Software reliability modeling. World Scientific Pub-
using feedforward and recurrent neural nets. in: International Joint lisher, Singapore.
Conference on Neural Networks, vol. 1, pp. 800–805. Xie, M., 1993. Software reliability models: a selected bibliography.
Khoshgoftaar, T.M., Szabo, R.M., 1996. Using neural networks to Journal of Software Testing, Verification and Reliability 3, 32–
predict software faults during testing. IEEE Transactions on 38.
Reliability 45 (4), 56–462. Yu, Y., Johnson, B.W., 2003, A BBN approach to certifying the
Kuo, L., Lee, J.C., 1997. Bayes inference for S-shaped software- reliability of COTS software system reliability. in: Proceedings
reliability growth models. IEEE Transactions on Reliability 46 (1), Annual Reliability and Maintainability Symposium, pp. 19–24.
76–80. Zheng, S.H., 2002. Dynamic release policies for software systems with
Kuo, L., Yang, T.Y., 1996. Bayesian computation for nonhomogene- a reliability constraint. IIE Transactions 34 (3), 253–262.
ous Poisson processes in software reliability. Journal of the Zhu, W.P., 2003. Using Bayesian network on network tomography.
American Statistical Association 91 (434), 763–773. Computer Communication 26, 155–163.