Partial Models: Towards Modeling and Reasoning with Uncertainty
1. Partial
Models:
Towards
Modeling and
Reasoning
with
Uncertainty
M.Famelis,
R.Salay, Partial Models: Towards Modeling and
M.Chechik,
Reasoning with Uncertainty
Introduction
Intuition
Motivating
Example
Modeling Michalis Famelis, Rick Salay, and Marsha Chechik
Uncertainty
Partial Models
Semantics University of Toronto
Reasoning
With
Uncertainty June 7, 2012
Property
Checking ICSE’12, Zurich, Switzerland
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
1 / 29
2. Partial
Models:
Towards
Modeling and
Intuition: Sudoku
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion Created with GNOME Sudoku 2.32.0.
2 / 29
3. Partial
Models:
Towards
Modeling and
Intuition: Sudoku
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion Created with GNOME Sudoku 2.32.0.
2 / 29
4. Partial
Models:
Towards
Modeling and
Intuition: Sudoku
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion Created with GNOME Sudoku 2.32.0.
2 / 29
5. Partial
Models:
Towards
Modeling and
Intuition: Sudoku
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion Created with GNOME Sudoku 2.32.0.
2 / 29
6. Partial
Models:
Towards
Modeling and
Intuition: Sudoku
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion Created with GNOME Sudoku 2.32.0.
2 / 29
7. Partial
Models:
Towards
Modeling and
Intuition: Sudoku
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion Created with GNOME Sudoku 2.32.0.
2 / 29
8. Partial
Models:
Towards
Modeling and
Intuition: Sudoku
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion Created with GNOME Sudoku 2.32.0.
2 / 29
9. Partial
Models:
Towards
Modeling and
Intuition: Sudoku
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion Created with GNOME Sudoku 2.32.0.
2 / 29
10. Partial
Models:
Towards
Modeling and
Intuition: Sudoku
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion Created with GNOME Sudoku 2.32.0.
2 / 29
11. Partial
Models:
Towards
Modeling and
Enough About Sudoku
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
Source: Wikimedia,
3 / 29
12. Partial
Models:
Towards
Modeling and
Goal: Uncertainty in Software
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Modeling
Explicate points of uncertainty
Introduction
Intuition
Correlate points of uncertainty
Motivating
Example
Modeling
Uncertainty
Partial Models Reasoning
Semantics
Reasoning
Check properties
With Give feedback to facilitate diagnosis
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
4 / 29
13. Partial
Models:
Towards
Modeling and
Designing a P2P Application
Reasoning
with Trying to design a P2P client application.
Uncertainty
M.Famelis,
R.Salay, What do I know?
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
5 / 29
14. Partial
Models:
Towards
Modeling and
Designing a P2P Application
Reasoning
with Trying to design a P2P client application.
Uncertainty
M.Famelis,
R.Salay, What do I not know?
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
5 / 29
15. Partial
Models:
Towards
Modeling and
Designing a P2P Application
Reasoning
with Trying to design a P2P client application.
Uncertainty
M.Famelis,
R.Salay, What do I not know?
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
5 / 29
16. Partial
Models:
Towards
Modeling and
Designing a P2P Application
Reasoning
with Trying to design a P2P client application.
Uncertainty
M.Famelis,
R.Salay, How can I explicate my uncertainty and reason in its presence?
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
5 / 29
17. Partial
Models:
Towards
Modeling and
Contribution
Reasoning
with
Uncertainty
Modeling Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
• Encode uncertainty in Partial Models.
Introduction
• Semantics: sets of conventional models.
Intuition
Motivating
Example
Modeling Reasoning in the Presence of Uncertainty
Uncertainty
Partial Models • Check properties.
Semantics
Reasoning • Give feedback to facilitate diagnosis.
With
Uncertainty
Property
Checking
Diagnosis Evaluation of Reasoning
Evaluation
Experiments • Reasoning with Partial models vs. reasoning with a set of
Case Study
conventional models
Conclusion
6 / 29
18. Partial
Models:
Towards
Modeling and
In Paper / Not In Talk
Reasoning
with
Uncertainty
M.Famelis,
Presentation of these would take too much time:
R.Salay,
M.Chechik, • Encoding conventional models in logic and back
Introduction
• Construction algorithm of Partial Models
Intuition
Motivating • Propositional Normal Form (PNF)
Example
Modeling • Graphical Normal Form (GNF)
Uncertainty
Partial Models • Diagnostic cores
Semantics
Reasoning • “Property-driven” refinement.
With
Uncertainty • Translation from PNF to GNF and vice versa
Property
Checking
Diagnosis • Evaluation of diagnostic cores and property-driven
Evaluation refinement
Experiments
Case Study • Random generation of experimental inputs
Conclusion
7 / 29
19. Partial
Models:
Towards
Modeling and
Contribution
Reasoning
with
Uncertainty
Modeling Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
• Encode uncertainty in Partial Models.
Introduction
• Semantics: sets of conventional models.
Intuition
Motivating
Example
Modeling Reasoning in the Presence of Uncertainty
Uncertainty
Partial Models • Check properties.
Semantics
Reasoning • Give feedback to facilitate diagnosis.
With
Uncertainty
Property
Checking
Diagnosis Evaluation of Reasoning
Evaluation
Experiments • Reasoning with Partial models vs. reasoning with a set of
Case Study
conventional models
Conclusion
8 / 29
20. Partial
Models:
Towards
Modeling and
Partial Models
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
9 / 29
21. Partial
Models:
Towards
Modeling and
Partial Models
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
9 / 29
22. Partial
Models:
Towards
Modeling and
Partial Models
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
9 / 29
23. Partial
Models:
Towards
Modeling and
Partial Models
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
9 / 29
24. Partial
Models:
Towards
Modeling and
Partial Models
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
9 / 29
25. Partial
Models:
Towards
Modeling and
Partial Models
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
9 / 29
26. Partial
Models:
Towards
Modeling and
Semantics of Partial Models
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
10 / 29
27. Partial
Models:
Towards
Modeling and
Semantics of Partial Models
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
10 / 29
28. Partial
Models:
Towards
Modeling and
Semantics of Partial Models
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
10 / 29
29. Partial
Models:
Towards
Modeling and
Related Ideas
Reasoning
with Behavioral modeling:
Uncertainty
• Modal Transition Systems (MTSs) [Larsen’88].
M.Famelis,
R.Salay, • Disjunctive MTSs [Larsen’91].
M.Chechik,
Introduction Software Product Lines:
Intuition
Motivating
Example
• Variability in the metamodel [Morin’09].
Modeling • Featured Transition Systems [Classen’10].
Uncertainty
Partial Models
Semantics
Reasoning Partial Models:
With
Uncertainty
• Language-independent
Property
Checking not just behavioral models!
Diagnosis
• May formula: exact encoding
Evaluation
Experiments thorough reasoning
Case Study
• Focus on systematic management of uncertainty
Conclusion
uncertainty-reducing refinement [VOLT’12]
transformations [MiSE’12]
11 / 29
30. Partial
Models:
Towards
Modeling and
Contribution
Reasoning
with
Uncertainty
Modeling Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
• Encode uncertainty in Partial Models.
Introduction
• Semantics: sets of conventional models.
Intuition
Motivating
Example
Modeling Reasoning in the Presence of Uncertainty
Uncertainty
Partial Models • Check properties.
Semantics
Reasoning • Give feedback to facilitate diagnosis.
With
Uncertainty
Property
Checking
Diagnosis Evaluation of Reasoning
Evaluation
Experiments • Reasoning with Partial models vs. reasoning with a set of
Case Study
conventional models
Conclusion
12 / 29
31. Partial
Models:
Towards
Modeling and
1) Property Checking
Reasoning
with
Uncertainty Property can be:
M.Famelis, • True: holds for all concretizations
R.Salay,
M.Chechik,
• False: holds for none
Introduction • Maybe: true for some, false for others
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
To check a property:
Semantics
- Encode model and property in propositional logic.
Reasoning
With
Uncertainty
- Use SAT solver.
Property
Checking
Diagnosis
ΦM ∧ Φp ΦM ∧ ¬Φp Property p
SAT SAT Maybe
Evaluation
Experiments
SAT UNSAT True
Case Study UNSAT SAT False
Conclusion UNSAT UNSAT (model inconsistent)
13 / 29
32. Partial
Models:
Towards
Modeling and
Property Checking: Example
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
14 / 29
33. Partial
Models:
Towards
Modeling and
Property Checking: Example
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
14 / 29
34. Partial
Models:
Towards
Modeling and
Property Checking: Example
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
14 / 29
35. Partial
Models:
Towards
Modeling and
Property Checking: Example
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
14 / 29
36. Partial
Models:
Towards
Modeling and
Property Checking: Example
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
14 / 29
37. Partial
Models:
Towards
Modeling and
Property Checking: Example
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
14 / 29
38. Partial
Models:
Towards
Modeling and
Property Checking: Example
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
14 / 29
39. Partial
Models:
Towards
Modeling and
Property Checking: Example
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
14 / 29
40. Partial
Models:
Towards
Modeling and
Property Checking: Example
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
14 / 29
41. Partial
Models:
Towards
Modeling and
Property Checking: Example
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
14 / 29
42. Partial
Models:
Towards
Modeling and
Property Checking: Example
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
14 / 29
43. Partial
Models:
Towards
Modeling and
Property Checking: Example
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
14 / 29
44. Partial
Models:
Towards
Modeling and
Property Checking: Example
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
14 / 29
45. Partial
Models:
Towards
Modeling and
Property Checking: Example
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
14 / 29
46. Partial
Models:
Towards
Modeling and
Property Checking: Example
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
14 / 29
47. Partial
Models:
Towards
Modeling and
2) Diagnosis
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Feedback:
A concretization of the Partial Model for which the
Introduction
Intuition property does not hold.
Motivating
Example
Modeling
Uncertainty Reuse the results of property checking:
Partial Models
Semantics
ΦM ∧ Φp ΦM ∧ ¬Φp Property p
Reasoning
With SAT SAT Maybe
Uncertainty SAT UNSAT True
Property UNSAT SAT False
Checking
Diagnosis UNSAT UNSAT (model inconsistent)
Evaluation
Experiments
Case Study
Conclusion
15 / 29
48. Partial
Models:
Towards
Modeling and
Diagnosis: Example
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
16 / 29
49. Partial
Models:
Towards
Modeling and
Diagnosis: Example
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
16 / 29
50. Partial
Models:
Towards
Modeling and
Contribution
Reasoning
with
Uncertainty
Modeling Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
• Encode uncertainty in Partial Models.
Introduction
• Semantics: sets of conventional models.
Intuition
Motivating
Example
Modeling Reasoning in the Presence of Uncertainty
Uncertainty
Partial Models • Check properties.
Semantics
Reasoning • Give feedback to facilitate diagnosis.
With
Uncertainty
Property
Checking
Diagnosis Evaluation of Reasoning
Evaluation
Experiments • Reasoning with Partial models vs. reasoning with a set of
Case Study
conventional models
Conclusion
17 / 29
51. Partial
Models:
Towards
Modeling and
Questions
Reasoning
with
Uncertainty
Reasoning with Partial models
M.Famelis,
R.Salay, vs
M.Chechik,
Reasoning with a set of conventional models
Introduction
Intuition
Motivating
Example
Modeling
Is there a speedup?
Uncertainty
Partial Models
Semantics How is speedup affected by changing:
Reasoning
With • model size
Uncertainty
Property
Checking
• levels of uncertainty?
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
18 / 29
52. Partial
Models:
Towards
Modeling and
Questions
Reasoning
with
Uncertainty
Reasoning with Partial models
M.Famelis,
R.Salay, vs
M.Chechik,
Reasoning with a set of conventional models
Introduction
Intuition
Motivating
Example
Modeling
Is there a speedup?
Uncertainty
Partial Models
Semantics How is speedup affected by changing:
Reasoning
With • model size
Uncertainty
Property
Checking
• levels of uncertainty?
Diagnosis
Evaluation
Experiments
Case Study
To get answers:
Conclusion 1) Experiments with random inputs.
2) Real-world case study.
18 / 29
53. Partial
Models:
Towards
Modeling and
Experiments
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
19 / 29
54. Partial
Models:
Towards
Modeling and
Experiments
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
19 / 29
55. Partial
Models:
Towards
Modeling and
Experiments
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
19 / 29
56. Partial
Models:
Towards
Modeling and
Experiments
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
19 / 29
57. Partial
Models:
Towards
Modeling and
Experiments
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
19 / 29
58. Partial
Models:
Towards
Modeling and
Experiments
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
19 / 29
59. Partial
Models:
Towards
Modeling and
Experiments
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
19 / 29
60. Partial
Models:
Towards
Modeling and
Experiments
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
19 / 29
61. Partial
Models:
Towards
Modeling and
Experiments
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
19 / 29
62. Partial
Models:
Towards
Modeling and
Case Study
Reasoning
with
Uncertainty
M.Famelis,
Why Case Study?
R.Salay,
M.Chechik, Triangulate experimental results (randomly inputs)
Introduction
with observations from a real-world scenario.
Intuition
Motivating
Example
Modeling
Case Study details:
Uncertainty
Partial Models
• Real-world software project: UMLet.
Semantics
Reasoning
• Real-world bug from UMLet bugzilla.
With
Uncertainty • Realistic bug fixes.
Property
Checking
Diagnosis
• Two properties from literature [V.D.Straeten’03].
Evaluation • 27,261 elements (XL model size)
Experiments
Case Study
• 220 concretizations (XL uncertainty size)
Conclusion
20 / 29
63. Partial
Models:
Towards
Modeling and
Case Study
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
21 / 29
64. Partial
Models:
Towards
Modeling and
Results of Evaluation
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
Reasoning with Partial models
M.Chechik,
vs
Introduction Reasoning with a set of conventional models
Intuition
Motivating
Example
Is there a speedup?
Modeling
Uncertainty
Partial Models
– Yes, it is consistently faster than reasoning with the set.
Semantics
Reasoning
With How is speedup affected by changing model size and levels of
Uncertainty
Property uncertainty?
Checking
Diagnosis
– Speedup decreases with model size.
Evaluation
Experiments
Case Study
– Speedup increases with uncertainty.
Conclusion – No slowdowns!
22 / 29
65. Partial
Models:
Towards
Modeling and
Summary
Reasoning
with
Uncertainty
Modeling Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
• Encode uncertainty in Partial Models.
Introduction
• Semantics: sets of conventional models.
Intuition
Motivating
Example
Modeling Reasoning in the Presence of Uncertainty
Uncertainty
Partial Models • Check properties.
Semantics
Reasoning • Give feedback to facilitate diagnosis.
With
Uncertainty
Property
Checking
Diagnosis Evaluation of Reasoning
Evaluation
Experiments • Reasoning with Partial models vs. reasoning with a set of
Case Study
conventional models
Conclusion
23 / 29
66. Partial
Models:
Towards
Modeling and
The Big Picture
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
24 / 29
67. Partial
Models:
Towards
Modeling and
Next Steps
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study
Conclusion
25 / 29
69. Partial
Models:
Towards
Modeling and
Language Independent!
Reasoning
with
Uncertainty
M.Famelis,
R.Salay,
M.Chechik,
Introduction
Intuition
Motivating
Example
Modeling
Uncertainty
Partial Models
Semantics
Reasoning
With
Uncertainty
Property
Checking
Diagnosis
Evaluation
Experiments
Case Study Class Diagram example from [MiSE’12].
Conclusion
27 / 29
70. Partial
Models:
Towards
Modeling and
Bibliography I
Reasoning
with
Uncertainty
P. Classen, A.and Heymans, P.Y. Schobbens, A. Legay, and J.F. Raskin.
“Model Checking Lots of Systems: Efficient Verification of Temporal
M.Famelis, Properties in Software Product Lines”.
R.Salay,
M.Chechik, In Proc. of ICSE’10, pages 335–344, 2010.
M. Famelis, Shoham Ben-David, Marsha Chechik, and Rick Salay.
Introduction
“Partial Models: A Position Paper”.
Intuition
Motivating In Proc. of MoDeVVa’11, pages 1–6, 2011.
Example
Modeling M. Famelis, R. Salay, and M. Chechik.
Uncertainty “The Semantics of Partial Model Transformations”.
Partial Models In Proc. of MiSE’12, 2012.
Semantics
Reasoning K. G. Larsen and B. Thomsen.
With “A Modal Process Logic”.
Uncertainty
In Proc. of LICS’88, pages 203–210, 1988.
Property
Checking
Diagnosis P. Larsen.
Evaluation “The Expressive Power of Implicit Specifications”.
Experiments In Proc. of ICALP’91, volume 510 of LNCS, pages 204–216, 1991.
Case Study
Conclusion
M.Famelis, R.Salay, and M. Chechik.
“The Semantics of Partial Model Transformations”.
In Proc. of MiSE’12, pages 546–560, 2012.
28 / 29
71. Partial
Models:
Towards
Modeling and
Bibliography II
Reasoning
with
Uncertainty
M.Famelis,
B. Morin, G. Perrouin, P. Lahire, O. Barais, G. Vanwormhoudt, and J. M.
R.Salay, J´z´quel.
e e
M.Chechik, “Weaving Variability into Domain Metamodels”.
J. Model Driven Engineering Languages and Systems, pages 690–705, 2009.
Introduction
Intuition R.Salay, M. Chechik, and J.Horkoff.
Motivating
Example “Managing Requirements Uncertainty with Partial Models”.
In Proc. of RE’12, pages 546–560, 2012.
Modeling
Uncertainty
R. Salay, M. Chechik, and J. Gorzny.
Partial Models
Semantics “Towards a Methodology for Verifying Partial Model Refinements”.
Reasoning
In Proc. of VOLT’12, 2012.
With
Uncertainty R. Salay, M. Famelis, and M. Chechik.
Property “Language Independent Refinement using Partial Modeling”.
Checking
Diagnosis
In Proc. of FASE’12, 2012.
Evaluation R. V. D. Straeten, T. Mens, J. Simmonds, and V. Jonckers.
Experiments “Using Description Logic to Maintain Consistency between UML Models”.
Case Study
In Proc. of UML’03, pages 326–340, 2003.
Conclusion
29 / 29