Доклад Владимира Иванова, Александра Ильина на конференции SQA Days-12, 30 ноября-1 декабря, Минск

1. Методы формальной
верификации

2. Методы формальной
верификации
… в Java 8

3. Методы формальной
верификации
… в Java 8
… да и вообще...

4. Pол
евы
игр е
ы
Владимир Иванов
Разработчик HotSpot JVM
Хардкорный девелопер
Александр Ильин
Архитектор тестирования Oracle JDK
Тролль из отдела тестирования

5. Program testing can be used to show the
presence of bugs, but never to show their
absence.
[“Structured programming”, Dahl O.J., Dijkstra E.W. and Hoare C.A.R.] (1972)
[When] you have given the proof of [a
program's] correctness, … [you] can
dispense with testing altogether.
[“Software engineering”, Naur P., Randell B.] (1969)

6. Testing
is
Running the tested software
– in different environment
– with different data
in an attempt to
– Certify conformance
– Prove program correctness
– Prove incorrectness

7. Fundamental Test Theorem
Just a few years after “Structured programming” ...
We prove … that properly structured tests are
capable of demonstrating the absence of
errors in a program.
[“Toward a Theory of Test Data Selection”, John B. Goodenough, Susan L.
Gerhart] (1975)

8. Fundamental Test Theorem
Program F(d) for domain D
Requirements: OUT(d, F(d)) = OK(d)
Data selection criteria: C
COMPLETE (T ,C )=(∀d ∈T OK (d )⇒∀d ∈ D OK (d ))∨(∀d ∈T ¬OK (d )⇒∀d ∈ D ¬OK (d ))
SUCCESSFUL(T )=∀t ∈T OK (t)
RELIABLE (C )=(∀T1 ,T2⊂ D)COMLPETE (T1 ,C )∧COMPLETE (T2 , C )⇒
(SUCCESSFUL(T1)≡SUCCESSFUL(T2))
VALID(C )=∀d ∈ D ¬OK (d )⇒(∃T ⊆ D)(COMPLETE (T ,C )∧¬SUCCESSFUL(T ))
∃T ⊆ D ,∃C
(COMPLETE (T ,C )∧RELIABLE (C )∧VALID(C )∧SUCCESSFUL(T ))⇒
∀d ∈ D OK (d )

9. But wait! It's not over yet!
I hope to have convinced you that by its very nature
responsible system design and development must
be an activity of an undeniably mathematical nature.
… programming became an industrial activity at a
moment that the American manager was extremely
fearful of relying on the education and the
intelligence of his company's employees. And
management tried to organize the industrial
programming task in such a way that each
individual programmer would need to think as little
as possible.
[“Why correctness must be a mathematical concern” E. W Dijkstra] (1979)

10. But wait! It's not over yet!
"Arrogance in computer science is measured in
nano-Dijkstras."
Alan Kay

11. But wait! It's not over yet!
"Arrogance in computer science is measured in
nano-Dijkstras."
Alan Kay
"… and micro-Kays".
Unknown source ;-)

12. Dynamic Testing
is
Running the tested software
– in different environment
– with different data
in an attempt to
– Certify conformance
– Prove program correctness (requires formal proof)
– Prove program incorrectness (practically)

13. Static testing
is
Analysis of artifacts
– source code
– binaries
– data files
in an attempt to
– discover errors
– identify suspicious patterns
– verify conformance of the artifacts

14. Static testing
includes
● Using static analyzers
– Big number of false positives
● Code reviews
– Tedious manual work
– Many errors missed
– Non formalizable

15. What defects
could by found by dynamic testing
Any defect!
You just need to invent enough test :)
only ...
It may take an indefinite amount of tests
So, the testing is, effectively, endless

16. What defects
could by found by static testing
Any defect!
You just need to look on the whole source long
enough
only ...
You can not know which ones you are detecting
and
You never know how many are left

17. What defects
are hard to find by dynamic testing
● Intermittent problems
– You may just missed it
● Platform/environment specific problem
– You just may not have the environment

18. What defects
are hard to find by static analysis
● Bugs in deep and wide class inheritance
– Virtual methods are resolved in runtime
● Bugs in modular system
– Many modules implement the same features,
modules are registered somewhere, etc.
– Same reason – modules are registered as runtime

19. Formal verification
is
Formal verification is the act of proving or
disproving the correctness of intended algorithms
underlying a system with respect to a certain
formal specification or property, using
formal methods of mathematics.

20. Formal verification vs Testing
is also
● Testing
– Upper bound for program quality
● Passed test says nothing about quality
● What matters is when test fails
● Formal verification
– Lower bound for program quality
● Passed test guarantees absence of some type of
failures in a program

21. Formal verification
requires
● Correctness of
– Language
– Compiler
– “Core” of the program
● The specification is self-consistent

22. Formal verification
applied
boolean isPowerOfTwo(int a) {
return (a&(a-1)) == 0;
}

23. Formal verification
applied
n
∀0< a∈ N : a &(a−1)=0⇔∃ n∈ N : a=2
a > 0 => binary presentation of a has a least one 1 bit
m >= 0
Let's take a binary representation of a: (a1…ak)10....0
m m
a-1 = (a1…ak)01....1 => a&(a-1) = (a1…ak)00....0
a&(a-1) = 0 => a1,...,ak = 0 => a = 2m
a = 2n => m=n, a1,...,ak = 0 => a&(a-1) = 0

24. Formal verification
is
Formal verification is the act of proving or
disproving the correctness of intended algorithms
underlying a system with respect to a certain
formal specification or property, using
formal methods of mathematics.
Another approach is deductive verification. It
consists of generating from the system and its
specifications (and possibly other annotations) a
collection of mathematical proof obligations,
the truth of which imply conformance of the
system to its specification.

25. Deductive Verification
Theorem proving
● Four color theorem (proved in 1976)
● Curry-Howard isomorphism
– (Theorem, Proof) <=> (Type, Program)
● Theorem provers
– Interactive environments for constructing proofs
– Coq, Agda, Isabelle, HOL
● Real-world example
– COMPCERT: C Verified Compiler

26. Using tools
how about ...
● We create a program
– Is capable of proving something about another
program
– Is itself proven (yeah, yeah, a recursion)
● Use the program to prove something about
another program
● Let's call it a “prover”
Is this still a formal verification?
Sure!

27. Formal verification
compiler is a prover on it's own
● Formal verification for Java is performed by
Java compiler
– Types
– Uninitialized variable
– Missing of return statement
– Uncaught exceptions
– etc.
– etc.

28. Annotations in Java
@Stateless @LocalBean
public class GalleryFacade {
@EJB
private GalleryEAO galleryEAO;
@TransactionAttribute(SUPPORTS)
public Gallery findById(Long id) { ... }
@TransactionAttribute(REQUIRED)
public void create(String name) { … }

29. Annotations in Java
● Introduced in Java 5
● Metadata
● May be reflective
– SOURCE, CLASS, RUNTIME
● Standard (e.g. @Override) & custom annotations
● Extensively used nowadays
– JavaEE 6, IoC containers, test harnesses, etc

30. Annotations: pre-Java 8
● Allowed on declarations only
– Class declaration
@A public class Test {
@B private int a = 0;
@C public void m(@D Object o) {
@E int a = 1;
...
}
}

31. Annotations: pre-Java 8
● Allowed on declarations only
– Field declaration
@A public class Test {
@B private int a = 0;
@C public void m(@D Object o) {
@E int a = 1;
...
}
}

32. Annotations: pre-Java 8
● Allowed on declarations only
– Method declaration
@A public class Test {
@B private int a = 0;
@C public void m(@D Object o) {
@E int a = 1;
...
}
}

33. Annotations: pre-Java 8
● Allowed on declarations only
– Method parameter declaration
@A public class Test {
@B private int a = 0;
@C public void m(@D Object o) {
@E int a = 1;
...
}
}

34. Annotations: pre-Java 8
● Allowed on declarations only
– Local variable declaration
@A public class Test {
@B private int a = 0;
@C public void m(@D Object o) {
@E int a = 1;
...
}
}

35. Limitations
● Consider @NonNull annotation
● How to declare a Map with non-null keys and
values?

36. Limitations
● Consider @NonNull annotation
● How to declare a Map with non-null keys and
values?
@NonNull Map<K,V>?

37. Limitations
● Consider @NonNull annotation
● How to declare a Map with non-null keys and
values?
@NonNull Map<K,V>?
NO!

38. Limitations
● Consider @NonNull annotation
● How to declare a Map with non-null keys and
values?
@NonNull Map<K,V>?
NO!
● Map<@NonNull K, @NonNull V>
… but incorrect in Java 7 and earlier
Type annotations in Java 8 for the rescue!

39. Type annotations in Java 8
● Allowed anywhere you would write a type
… including generics and casts
… for array levels and receivers

40. Type annotations in Java 8:
Examples
● Class inheritance
class UnmodifiableList<T>
implements @ReadOnly List<T> { ... }
● Casts
myDate = (@ReadOnly Date) roDate;
● Type tests
myString instanceof @NonNull String;
● Arrays
String @NonNull [] messages;

41. Type annotations in Java 8:
Examples
● Generics
List<@Interned String> messages;
● Type parameter bounds
Collection<? super @Exists File>
● Generic type arguments in a generic method
o.<@NonNull String>toString("...");

42. Pluggable types
● User-defined (pluggable) type system
● Extend built-in type system
– express extra information about types via
type qualifiers
● Permit more expressive compile-time checking
and guarantee the absence of additional errors

43. Checker Framework
● Collection of compiler plugins (“checkers”)
● Relies on Pluggable types and Type
Annotations in Java 8
● Find different sorts of bugs or verify their
absence
– 14 checkers are already provided
● Supports custom compiler plugins (provides
API)
– 5 third-party checkers

44. Example: Nullness Checker
● Annotations
– @NonNull
– @Nullable
● Partial type hierarchy

45. Example: Nullness Checker
● Annotations
– @NonNull
– @Nullable
● Example:
@Nullable Object o1; // might be null
@NonNull Object o2; // never null
o1.toString(); // warning
o2 = o1; // warning
if (o2 == null) // warning: redundant test

46. Example: Nullness Checker
● Annotations
– @NonNull
– @Nullable
● Example:
public <@NonNull T> T process(T);

47. Example: Tainting Checker
● Use case:
– Trusted vs untrusted data
– Verify before use
● Examples
– SQL injection attack
● validate SQL query before executing it
– information leakage
● secret data vs data displayed to a user

48. Example: Tainting Checker
● Annotations
– @Untainted
● A type that includes only untainted, trusted values
– @Tainted
● A type that includes only tainted, untrusted values

49. Example: Tainting Checker
● Annotations
– @Untainted
– @Tainted
● Example
void execute(@Untainted String sql)
throws SQLException;
@Untainted String validate(@Tainted String)
throws SQLException;

50. Credit card number
Annotation
@Documented
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE_USE,
ElementType.TYPE_PARAMETER})
@TypeQualifier
@SubtypeOf(Unqualified.class)
public @interface CreditCard {}

51. Credit card number
Checker
@TypeQualifiers(CreditCard.class)
@SuppressWarningsKey("credit.card")
public class CreditCardChecker
extends BaseTypeChecker {
…
}

52. Credit card number
Usage
public class Account {
private final @CreditCard String cardNumber;
public Account(@CreditCard String number) {
this.cardNumber = number;
}
public @CreditCard String getCardNumber() {
return cardNumber;
}

53. Credit card number
Sources
@SuppressWarnings("credit.card")
@CreditCard String convert(String input) {
if(checkLuhn(input))
return input;
else
throw IllegalArgumentException("...")
}
new Account("4111111111111111");
new Account("4111111111111110");

54. Credit card number
Conclusion
● A card number in an account is always validated
● That is guaranteed at compile time
● You do not need to test with invalid numbers
● You do need to test
– All @SuppressWarnings("credit.card")
– checkLuhn(String cardNum)
● Better all … prove it!

55. More real life examples
String getProperty(@PropertyKey String key);
HashMap <@Adult Person, @NonNull Address>
findSobutylnik(@NonNull Location);
void monitorTemperature()
throws @Critical TemperatureException;

56. Checkers Framework:
Advanced features
● Linear checker
– Implements linear types (based on linear logic)
– control aliasing and prevent re-use
– Single ownership abstraction
● Prevents absence of ownership and multiple owners
● Dependent types
– @Dependent annotation
– Changes the type depending on qualified type of
the receiver (this)
– Example
List[N] – list with it's length encoded into it's type

57. How to start using
● No need to wait Java 8 release
– modified compiler already available
● Incremental program annotation
– Partial program checking
– Warnings during compilation
– Easily convertible into compilation errors
● -Werror flag to javac
– Default annotations for types w/o annotations
● Ability to annotate external libraries

58. Links
● Type Annotations Specification (JSR-308)
http://types.cs.washington.edu/jsr308/specification/java-
● Checker Framework
http://types.cs.washington.edu/checker-framework/curre

59. Q&A

60. Владимир Иванов
vladimir.x.ivanov@oracle.com
Александр Ильин
alexandre.iline@oracle.com

61. Алло, мы ищем таланты!
Приходите к нам работать!
alexandre.iline@oracle.co
m