This document discusses Bayesian logic programs, which extend Bayesian networks to allow for objects and relations. Bayesian logic programs use logic programming concepts like predicates, objects, and relations combined with probability theory. They define a probability distribution over infinitely many random variables. The document provides examples of how Bayesian networks can be represented as Bayesian logic programs and discusses semantics including consistency conditions and support networks.

1. Bayesian Logic Programs Kristian Kersting, Luc De Raedt Albert-Ludwigs University Freiburg, Germany Summer School on Relational Data Mining 17 and 18 August 2002, Helsinki, Finland

2. Context Real-world applications uncertainty complex, structured domains logic objects, relations,functors probability theory discrete, continuous Bayesian networks Logic Programming (Prolog) + Bayesian logic programs

7. Bayesian networks [Pearl ´88] bt_ann bt_brian bt_cecily bt_dorothy bt_eric bt_gwenn bt_unknown2 bt_unknown1 bt_fred bt_henry bt_irene bt_john Based on the stud farm example [Jensen ´96] 

8. Bayesian networks [Pearl ´88] bt_ann bt_brian bt_cecily bt_dorothy bt_eric bt_gwenn bt_unknown2 bt_unknown1 bt_fred bt_henry bt_irene bt_john Based on the stud farm example [Jensen ´96] (Conditional) Probability distribution  P(bt_cecily=aA|bt_john=aA)=0.1499 P(bt_john=AA|bt_ann=aA)=0.6906 P(bt_john=AA)=0.9909 (0.33,0.33,0.33) ... (0.0,1.0,0.0) (0.5,0.5,0.0) bt_irene bt_henry P (bt_john) AA AA AA aa aA aa aa aa (1.0,0.0,0.0)

10. From Bayesian Networks to Bayesian Logic Programs bt_ann bt_brian bt_cecily bt_dorothy bt_eric bt_gwenn bt_unknown2 bt_unknown1 bt_fred bt_henry bt_irene bt_john 

11. From Bayesian Networks to Bayesian Logic Programs bt_ann. bt_brian. bt_cecily. bt_dorothy bt_eric bt_gwenn bt_unknown2. bt_unknown1. bt_fred bt_henry bt_irene bt_john 

12. From Bayesian Networks to Bayesian Logic Programs bt_ann. bt_brian. bt_cecily. bt_dorothy | bt_ann, bt_brian. bt_eric| bt_brian, bt_cecily. bt_gwenn | bt_ann, bt_unknown2. bt_unknown2. bt_unknown1. bt_fred | bt_unknown1, bt_ann. bt_henry bt_irene bt_john 

13. From Bayesian Networks to Bayesian Logic Programs bt_ann. bt_brian. bt_cecily. bt_dorothy | bt_ann, bt_brian. bt_eric| bt_brian, bt_cecily. bt_gwenn | bt_ann, bt_unknown2. bt_unknown2. bt_unknown1. bt_fred | bt_unknown1, bt_ann. bt_henry | bt_fred, bt_dorothy. bt_irene | bt_eric, bt_gwenn. bt_john 

14. From Bayesian Networks to Bayesian Logic Programs bt_ann. bt_brian. bt_cecily. bt_dorothy | bt_ann, bt_brian. bt_eric| bt_brian, bt_cecily. bt_gwenn | bt_ann, bt_unknown2. bt_unknown2. bt_unknown1. bt_fred | bt_unknown1, bt_ann. bt_henry | bt_fred, bt_dorothy. bt_irene | bt_eric, bt_gwenn. bt_john | bt_henry ,bt_irene.  (0.33,0.33,0.33) ... (0.0,1.0,0.0) (0.5,0.5,0.0) bt_irene bt_henry P (bt_john) AA AA AA aa aA aa aa aa (1.0,0.0,0.0)

15. From Bayesian Networks to Bayesian Logic Programs % apriori nodes bt_ann. bt_brian. bt_cecily. bt_unknown1. bt_unknown1. % aposteriori nodes bt_henry | bt_fred, bt_dorothy. bt_irene | bt_eric, bt_gwenn. bt_fred | bt_unknown1, bt_ann. bt_dorothy| bt_brian, bt_ann. bt_eric | bt_brian, bt_cecily. bt_gwenn | bt_unknown2, bt_ann. bt_john | bt_henry, bt_irene.  D omain e.g. finite, discrete, continuous (conditional) probability distribution (0.33,0.33,0.33) ... (0.0,1.0,0.0) (0.5,0.5,0.0) bt_irene bt_henry P (bt_john) AA AA AA aa aA aa aa aa (1.0,0.0,0.0)

16. From Bayesian Networks to Bayesian Logic Programs % apriori nodes bt(ann). bt(brian). bt(cecily). bt(unknown1). bt(unknown1). % aposteriori nodes bt(henry) | bt(fred), bt(dorothy). bt(irene) | bt(eric), bt(gwenn). bt(fred) | bt(unknown1), bt(ann). bt(dorothy)| bt(brian), bt(ann). bt(eric) | bt(brian), bt(cecily). bt(gwenn) | bt(unknown2), bt(ann). bt(john) | bt(henry), bt(irene).  (conditional) probability distribution (0.33,0.33,0.33) ... (0.0,1.0,0.0) (0.5,0.5,0.0) bt(irene) bt(henry) P (bt(john)) AA AA AA aa aA aa aa aa (1.0,0.0,0.0)

17. From Bayesian Networks to Bayesian Logic Programs  % ground facts / apriori bt(ann). bt(brian). bt(cecily). bt(unkown1). bt(unkown1). father(unkown1,fred). mother(ann,fred). father(brian,dorothy). mother(ann, dorothy). father(brian,eric). mother(cecily,eric). father(unkown2,gwenn). mother(ann,gwenn). father(fred,henry). mother(dorothy,henry). father(eric,irene). mother(gwenn,irene). father(henry,john). mother(irene,john). % rules / aposteriori bt(X) | father(F,X), bt(F), mother(M,X), bt(M). (conditional) probability distribution AA Aa bt(F) false true father(F,X) (0.33,0.33,0.33) ... (1.0,0.0,0.0) P (bt(X)) false bt(M) mother(M,X) AA aa true

18. Dependency graph = Bayesian network bt(ann) bt(brian) bt(cecily) bt(dorothy) mother(ann,dorothy) father(brian,dorothy) bt(eric) father(brian,eric) mother(cecily,eric) bt(gwenn) mother(ann,gwenn) bt(unknown2) bt(unknown1) bt(fred) mother(ann,fred) father(unknown1,fred) bt(henry) bt(irene) mother(dorothy,henry) mother(gwenn,irene) father(eric,irene) father(fred,henry) bt(john) father(henry,john) mother(irene,john) father(unknown2,eric) 

19. bt_ann bt_brian bt_cecily bt_dorothy bt_eric bt_gwenn bt_unknown2 bt_unknown1 bt_fred bt_henry bt_irene bt_john  Dependency graph = Bayesian network

21. Bayesian Logic Programs - Examples % apriori nodes nat(0). % aposteriori nodes nat(s(X)) | nat(X). ... MC % apriori nodes state(0). % aposteriori nodes state(s(Time)) | state(Time). output(Time) | state(Time) HMM % apriori nodes n1(0). % aposteriori nodes n1(s(TimeSlice) | n2(TimeSlice). n2(TimeSlice) | n1(TimeSlice). n3(TimeSlice) | n1(TimeSlice), n2(TimeSlice). DBN  pure Prolog nat(0) nat(s(0)) nat(s(s(0)) state(0) output(0) state(s(0)) output(s(0)) ... n1(0) n2(0) n3(0) n1(s(0)) n2(s(0)) n3(s(0)) ...

30. Support network bt(ann) bt(brian) bt(cecily) bt(dorothy) mother(ann,dorothy) father(brian,dorothy) bt(eric) father(brian,eric) mother(cecily,eric) bt(gwenn) mother(ann,gwenn) bt(unknown2) bt(unknown1) bt(fred) mother(ann,fred) father(unknown1,fred) bt(henry) bt(irene) mother(dorothy,henry) mother(gwenn,irene) father(eric,irene) father(fred,henry) bt(john) father(henry,john) mother(irene,john) father(unknown2,eric) 

31. Support network bt(ann) bt(brian) bt(cecily) bt(dorothy) mother(ann,dorothy) father(brian,dorothy) bt(eric) father(brian,eric) mother(cecily,eric) bt(gwenn) mother(ann,gwenn) bt(unknown2) bt(unknown1) bt(fred) mother(ann,fred) father(unknown1,fred) bt(henry) bt(irene) mother(dorothy,henry) mother(gwenn,irene) father(eric,irene) father(fred,henry) bt(john) father(henry,john) mother(irene,john) father(unknown2,eric) 

32. Support network bt(ann) bt(brian) bt(cecily) bt(dorothy) mother(ann,dorothy) father(brian,dorothy) bt(eric) father(brian,eric) mother(cecily,eric) bt(gwenn) mother(ann,gwenn) bt(unknown2) bt(unknown1) bt(fred) mother(ann,fred) father(unknown1,fred) bt(henry) bt(irene) mother(dorothy,henry) mother(gwenn,irene) father(eric,irene) father(fred,henry) bt(john) father(henry,john) mother(irene,john) father(unknown2,eric) 

36. Relational Character % ground facts bt(ann). bt(brian). bt(cecily). bt(unknown1). bt(unknown1). father(unknown1,fred). mother(ann,fred). father(brian,dorothy). mother(ann, dorothy). father(brian,eric). mother(cecily,eric). father(unknown2,gwenn). mother(ann,gwenn). father(fred,henry). mother(dorothy,henry). father(eric,irene). mother(gwenn,irene). father(henry,john). mother(irene,john). % rules bt(X) | father(F,X), bt(F), mother(M,X), bt(M).  AA Aa bt(F) false true father(X,F) (0.33,0.33,0.33) ... (1.0,0.0,0.0) P (bt(X)) false bt(M) mother(X,M) AA aa true % ground facts bt(susanne). bt(ralf). bt(peter). bt(uta). father(ralf,luca). mother(susanne,luca). ... % ground facts bt(petra). bt(bsilvester). bt(anne). bt(wilhelm). bt(beate). father(silvester,claudien). mother(beate,claudien). father(wilhelm,marta). mother(anne, marthe). ...

41. Learning Bayesian Logic Programs Data + Background Knowledge  learning algorithm .9 .1 e b e .7 .3 .01 .99 .8 .2 b e b b e E P (A) A | E, B. B

43. What is the data about ?  A data case is a partially observed joint state of a finite, nonempty subset

47. Parameter Estimation (contd.)  is an ordinary BN marginal of marginal of

50. Gradient Ascent  Goal : Computation of

51. Gradient Ascent  Bayesian ground clause Bayesian clause

52. Gradient Ascent  Bayesian Network Inference Engine

53. Algorithm 

59. What is the data about ? ... 

62. Algorithm 

63. Example  mc(john) pc(john) bc(john) m(ann,john) f(eric,john) pc(ann) mc(ann) mc(eric) pc(eric) mc(X) | m(M,X), mc(M), pc(M). pc(X) | f(F,X), mc(F), pc(F). bt(X) | mc(X), pc(X). Original Bayesian logic program {m(ann,john)=true, pc(ann)=a, mc(ann)=?, f(eric,john)=true, pc(eric)=b, mc(eric)=a, mc(john)=ab, pc(john)=a, bt(john) = ? } ... Data cases

64. Example  mc(john) pc(john) bc(john) m(ann,john) f(eric,john) pc(ann) mc(ann) mc(eric) pc(eric) mc(X) | m(M,X), mc(M), pc(M). pc(X) | f(F,X), mc(F), pc(F). bt(X) | mc(X), pc(X). Original Bayesian logic program mc(X) | m(M,X). pc(X) | f(F,X). bt(X) | mc(X). Initial hypothesis

65. Example  mc(X) | m(M,X), mc(M), pc(M). pc(X) | f(F,X), mc(F), pc(F). bt(X) | mc(X), pc(X). Original Bayesian logic program mc(X) | m(M,X). pc(X) | f(F,X). bt(X) | mc(X). Initial hypothesis mc(john) pc(john) bc(john) m(ann,john) f(eric,john) pc(ann) mc(ann) mc(eric) pc(eric)

66. Example  mc(john) pc(john) bc(john) m(ann,john) f(eric,john) pc(ann) mc(ann) mc(eric) pc(eric) mc(X) | m(M,X), mc(M), pc(M). pc(X) | f(F,X), mc(F), pc(F). bt(X) | mc(X), pc(X). Original Bayesian logic program mc(X) | m(M,X). pc(X) | f(F,X). bt(X) | mc(X), pc(X). Refinement mc(X) | m(M,X). pc(X) | f(F,X). bt(X) | mc(X). Initial hypothesis

67. Example  mc(john) pc(john) bc(john) m(ann,john) f(eric,john) pc(ann) mc(ann) mc(eric) pc(eric) mc(X) | m(M,X), mc(M), pc(M). pc(X) | f(F,X), mc(F), pc(F). bt(X) | mc(X), pc(X). Original Bayesian logic program mc(X) | m(M,X),mc(X). pc(X) | f(F,X). bt(X) | mc(X), pc(X). Refinement mc(X) | m(M,X). pc(X) | f(F,X). bt(X) | mc(X). Initial hypothesis mc(X) | m(M,X). pc(X) | f(F,X). bt(X) | mc(X), pc(X). Refinement

68. Example  mc(john) pc(john) bc(john) m(ann,john) f(eric,john) pc(ann) mc(ann) mc(eric) pc(eric) mc(X) | m(M,X), mc(M), pc(M). pc(X) | f(F,X), mc(F), pc(F). bt(X) | mc(X), pc(X). Original Bayesian logic program mc(X) | m(M,X),pc(X). pc(X) | f(F,X). bt(X) | mc(X), pc(X). Refinement mc(X) | m(M,X). pc(X) | f(F,X). bt(X) | mc(X). Initial hypothesis mc(X) | m(M,X). pc(X) | f(F,X). bt(X) | mc(X), pc(X). Refinement

69. Example  mc(john) pc(john) bc(john) m(ann,john) f(eric,john) pc(ann) mc(ann) mc(eric) pc(eric) ... mc(X) | m(M,X), mc(M), pc(M). pc(X) | f(F,X), mc(F), pc(F). bt(X) | mc(X), pc(X). Original Bayesian logic program mc(X) | m(M,X),pc(X). pc(X) | f(F,X). bt(X) | mc(X), pc(X). Refinement mc(X) | m(M,X). pc(X) | f(F,X). bt(X) | mc(X). Initial hypothesis mc(X) | m(M,X). pc(X) | f(F,X). bt(X) | mc(X), pc(X). Refinement

71. Example Experiments  mc(X) | m(M,X), mc(M), pc(M). pc(X) | f(F,X), mc(F), pc(F). bt(X) | mc(X), pc(X). Data: sampling from 2 families, each 1000 samples Score: LogLikelihood Goal: learn the definition of bt CLAUDIEN mc(X) | m(M,X). pc(X) | f(F,X). Bloodtype mc(X) | m(M,X), mc(M), pc(M). pc(X) | f(F,X), mc(F), pc(F). bt(X) | mc(X), pc(X). highest score

73.  Thanks !