The document discusses data-driven testing and the RSpock testing framework. Some key points: - RSpock is a testing framework built on Minitest that aims to optimize the time to build a mental model of code and write tests. - It transforms Minitest-style tests into a more declarative style using code blocks like Given, When, Then, Expect, and Where. - This helps group related tests together and makes it easier to understand what each part of a test is doing. - A case study showed the test density (tests per line of code) increased by 400% when converting tests to RSpock style, improving code coverage.

1. RSpock: Data-Driven Testing
Jean-Philippe Duchesne

2. • 80% reading code
• 20% writing code
• Some even say closer to 10:1
Read / Write Code

3. “I Have a Dream”
class MyThing
# ...
def do_something(a, b)
# Implementation details...
end
end

4. “I Have a Dream”
class MyThing
extend(T::Sig)
sig { params(a: Integer, b: Integer).returns(String) }
def do_something(a, b)
# Implementation details...
end
end

5. “I Have a Dream”
class MyThing
extend(T::Sig)
# Documentation
sig { params(a: Integer, b: Integer).returns(String) }
def do_something(a, b)
# Implementation details...
end
end

6. class MyThing
def do_something(a, b)
#...
end
end
Mental Model

7. class MyThing
def do_something(a, b)
#...
end
end
Tests as Specification
class MyThingTest < HermeticTestCase
test "#do_something with arg1 and arg2 does X" do
# ...
end
test "#foo does something amazing" do
# ...
end
test "#do_something does Y if param1 is nil" do
# ...
end
test "#bar does something less amazing than foo" do
# ...
end
test "#do_something does Z" do
# ...
end
end
Mental Model

8. Tests as Specification
class MyThingTest < HermeticTestCase
test "#do_something with arg1 and arg2 does X" do
# ...
end
test "#foo does something amazing" do
# ...
end
test "#do_something does Y if param1 is nil" do
# ...
end
test "#bar does something less amazing than foo" do
# ...
end
test "#do_something does Z" do
# ...
end
end
Mental Model
class MyThing
def do_something(a, b)
#...
end
end

9. Tests as Specification
class MyThingTest < HermeticTestCase
test "#do_something with arg1 and arg2 does X" do
# ...
end
test "#foo does something amazing" do
# ...
end
test "#do_something does Y if param1 is nil" do
# ...
end
test "#bar does something less amazing than foo" do
# ...
end
test "#do_something does Z" do
# ...
end
end
Mental Model
class MyThing
def do_something(a, b)
#...
end
end

10. • Regroup related ideas (tests)
• Make it easier to identify that related tests have the same logic
• Make it easier to identify which part of a test does what
Build Mental Model Faster?

11. RSpock

12. • Testing framework piggybacking on Minitest
• Came to life from 2x Hack Days projects in 2018
• Goals:
• Optimize time to build mental model
• Optimize time to write tests
What is it?

13. • Number of Test Cases / Lines of Code
• Higher test density => related ideas are more grouped together
Measurement: Test Density

14. class UpdaterComparatorTest < HermeticTestCase
class DummyUpdater < BaseUpdater; end
# ... setup block
test "#compare with updater a < b returns -1" do
result = @comparator.compare(UpdaterA.new, UpdaterB.new)
assert_equal(-1, result)
end
test "#compare with updater a > b returns 1" do
result = @comparator.compare(UpdaterB.new, UpdaterA.new)
assert_equal(1, result)
end
test "#compare with updater a == b returns 0" do
result = @comparator.compare(UpdaterA.new, UpdaterA.new)
assert_equal(0, result)
end
test "#compare with a missing updater raises" do
assert_raises ArgumentError do
@comparator.compare(DummyUpdater.new, UpdaterA.new)
end
end
end
Minitest

15. class UpdaterComparatorTest < HermeticTestCase
class DummyUpdater < BaseUpdater; end
# ... setup block
test "#compare with updater a < b returns -1" do
result = @comparator.compare(UpdaterA.new, UpdaterB.new)
assert_equal(-1, result)
end
test "#compare with updater a > b returns 1" do
result = @comparator.compare(UpdaterB.new, UpdaterA.new)
assert_equal(1, result)
end
test "#compare with updater a == b returns 0" do
result = @comparator.compare(UpdaterA.new, UpdaterA.new)
assert_equal(0, result)
end
test "#compare with a missing updater raises" do
assert_raises ArgumentError do
@comparator.compare(DummyUpdater.new, UpdaterA.new)
end
end
end
transform!(RSpock::AST::Transformation)
class UpdaterComparatorTest < HermeticTestCase
class DummyUpdater < CheckoutUpdaters::BaseUpdater; end
# ... setup block
test "#compare #{updater1.class} with #{updater2.class} returns #{result}" do
Expect
@comparator.compare(updater1, updater2) == result
Where
updater1 | updater2 | result
UpdaterA.new | UpdaterB.new | -1
UpdaterB.new | UpdaterA.new | 1
UpdaterB.new | UpdaterB.new | 0
end
test "#compare with unknown updater raises" do
Expect
assert_raises ArgumentError do
@comparator.compare(updater1, updater2)
end
Where
updater1 | updater2
DummyUpdater.new | DummyUpdater.new
DummyUpdater.new | UpdaterA.new
UpdaterA.new | DummyUpdater.new
end
end
RSpockMinitest

16. class UpdaterComparatorTest < HermeticTestCase
class DummyUpdater < BaseUpdater; end
# ... setup block
test "#compare with updater a < b returns -1" do
result = @comparator.compare(UpdaterA.new, UpdaterB.new)
assert_equal(-1, result)
end
test "#compare with updater a > b returns 1" do
result = @comparator.compare(UpdaterB.new, UpdaterA.new)
assert_equal(1, result)
end
test "#compare with updater a == b returns 0" do
result = @comparator.compare(UpdaterA.new, UpdaterA.new)
assert_equal(0, result)
end
test "#compare with a missing updater raises" do
assert_raises ArgumentError do
@comparator.compare(DummyUpdater.new, UpdaterA.new)
end
end
end
transform!(RSpock::AST::Transformation)
class UpdaterComparatorTest < HermeticTestCase
class DummyUpdater < CheckoutUpdaters::BaseUpdater; end
# ... setup block
test "#compare #{updater1.class} with #{updater2.class} returns #{result}" do
Expect
@comparator.compare(updater1, updater2) == result
Where
updater1 | updater2 | result
UpdaterA.new | UpdaterB.new | -1
UpdaterB.new | UpdaterA.new | 1
UpdaterB.new | UpdaterB.new | 0
end
test "#compare with unknown updater raises" do
Expect
assert_raises ArgumentError do
@comparator.compare(updater1, updater2)
end
Where
updater1 | updater2
DummyUpdater.new | DummyUpdater.new
DummyUpdater.new | UpdaterA.new
UpdaterA.new | DummyUpdater.new
end
end
RSpockMinitest

17. class UpdaterComparatorTest < HermeticTestCase
class DummyUpdater < BaseUpdater; end
# ... setup block
test "#compare with updater a < b returns -1" do
result = @comparator.compare(UpdaterA.new, UpdaterB.new)
assert_equal(-1, result)
end
test "#compare with updater a > b returns 1" do
result = @comparator.compare(UpdaterB.new, UpdaterA.new)
assert_equal(1, result)
end
test "#compare with updater a == b returns 0" do
result = @comparator.compare(UpdaterA.new, UpdaterA.new)
assert_equal(0, result)
end
test "#compare with a missing updater raises" do
assert_raises ArgumentError do
@comparator.compare(DummyUpdater.new, UpdaterA.new)
end
end
end
transform!(RSpock::AST::Transformation)
class UpdaterComparatorTest < HermeticTestCase
class DummyUpdater < CheckoutUpdaters::BaseUpdater; end
# ... setup block
test "#compare #{updater1.class} with #{updater2.class} returns #{result}" do
Expect
@comparator.compare(updater1, updater2) == result
Where
updater1 | updater2 | result
UpdaterA.new | UpdaterB.new | -1
UpdaterB.new | UpdaterA.new | 1
UpdaterB.new | UpdaterB.new | 0
end
test "#compare with unknown updater raises" do
Expect
assert_raises ArgumentError do
@comparator.compare(updater1, updater2)
end
Where
updater1 | updater2
DummyUpdater.new | DummyUpdater.new
DummyUpdater.new | UpdaterA.new
UpdaterA.new | DummyUpdater.new
end
end
RSpockMinitest

18. class UpdaterComparatorTest < HermeticTestCase
class DummyUpdater < BaseUpdater; end
# ... setup block
test "#compare with updater a < b returns -1" do
result = @comparator.compare(UpdaterA.new, UpdaterB.new)
assert_equal(-1, result)
end
test "#compare with updater a > b returns 1" do
result = @comparator.compare(UpdaterB.new, UpdaterA.new)
assert_equal(1, result)
end
test "#compare with updater a == b returns 0" do
result = @comparator.compare(UpdaterA.new, UpdaterA.new)
assert_equal(0, result)
end
test "#compare with a missing updater raises" do
assert_raises ArgumentError do
@comparator.compare(DummyUpdater.new, UpdaterA.new)
end
end
end
transform!(RSpock::AST::Transformation)
class UpdaterComparatorTest < HermeticTestCase
class DummyUpdater < CheckoutUpdaters::BaseUpdater; end
# ... setup block
test "#compare #{updater1.class} with #{updater2.class} returns #{result}" do
Expect
@comparator.compare(updater1, updater2) == result
Where
updater1 | updater2 | result
UpdaterA.new | UpdaterB.new | -1
UpdaterB.new | UpdaterA.new | 1
UpdaterB.new | UpdaterB.new | 0
end
test "#compare with unknown updater raises" do
Expect
assert_raises ArgumentError do
@comparator.compare(updater1, updater2)
end
Where
updater1 | updater2
DummyUpdater.new | DummyUpdater.new
DummyUpdater.new | UpdaterA.new
UpdaterA.new | DummyUpdater.new
end
end
RSpockMinitest

19. class UpdaterComparatorTest < HermeticTestCase
class DummyUpdater < BaseUpdater; end
# ... setup block
test "#compare with updater a < b returns -1" do
result = @comparator.compare(UpdaterA.new, UpdaterB.new)
assert_equal(-1, result)
end
test "#compare with updater a > b returns 1" do
result = @comparator.compare(UpdaterB.new, UpdaterA.new)
assert_equal(1, result)
end
test "#compare with updater a == b returns 0" do
result = @comparator.compare(UpdaterA.new, UpdaterA.new)
assert_equal(0, result)
end
test "#compare with a missing updater raises" do
assert_raises ArgumentError do
@comparator.compare(DummyUpdater.new, UpdaterA.new)
end
end
end
transform!(RSpock::AST::Transformation)
class UpdaterComparatorTest < HermeticTestCase
class DummyUpdater < CheckoutUpdaters::BaseUpdater; end
# ... setup block
test "#compare #{updater1.class} with #{updater2.class} returns #{result}" do
Expect
@comparator.compare(updater1, updater2) == result
Where
updater1 | updater2 | result
UpdaterA.new | UpdaterB.new | -1
UpdaterB.new | UpdaterA.new | 1
UpdaterB.new | UpdaterB.new | 0
end
test "#compare with unknown updater raises" do
Expect
assert_raises ArgumentError do
@comparator.compare(updater1, updater2)
end
Where
updater1 | updater2
DummyUpdater.new | DummyUpdater.new
DummyUpdater.new | UpdaterA.new
UpdaterA.new | DummyUpdater.new
end
end
RSpockMinitest

20. 4 26
Test Cases per Lines of Code
test cases w/o RSpock lines of code w/o RSpock
6 30test cases with RSpock lines of code with RSpock

21. 0.15 0.20
Test Density
w/o RSpock with RSpock
33%improvement

22. Case Study:
Multi Location Inventory
Tests

23. 18 268
Multi Location Inventory Tests
test cases w/o RSpock lines of code w/o RSpock
77 220test cases with RSpock lines of code with RSpock

24. 0.07 0.35
Test Density
w/o RSpock with RSpock
400%improvement

25. 20.3 54.5
Coverage
avg. hits per line w/o RSpock avg. hits per line with RSpock
98%test coverage for both

26. RSpock Code Blocks -> BDD Phases
Code Block BDD Phase
Given Setup
When Stimulus
Then Response
Expect
Stimulus +
Response
Cleanup Cleanup

27. How Does it Work?

28. RubyVM::InstructionSequence Hook
module RubyVM::InstructionSequence
def self.load_iseq(source_path)
# ...
end
end

29. Abstract Syntax Tree
1 + 2
int(1)
send(:+)
int(2)
Ruby Code

30. transform! annotation
transform!(RSpock::AST::Transformation)
class MyTest < Minitest::Test
# ...
end

31. RSpock Code Blocks

32. Given
Given "An empty Cart and a Product"
cart = Cart.new
product = Product.new

33. When
When "Adding a product"
cart.add_product(Product.new)

34. Then
Then "The product is added to the cart"
cart.products.size == 1
cart.products.first == product

35. Expect
When "Calling #abs on a negative number"
actual = -2.abs
Then "Value is positive"
actual == 2
Expect "absolute of -2 is 2"
-2.abs == 2
Using Expect
When + Then

36. • Use When + Then to describe methods with side-effects
• Use Expect to describe purely functional methods (query methods)
Expect: Rule of Thumb

37. Cleanup
Given "Open the file"
file = File.new("/invalid/file/path") # raises
# other blocks...
Cleanup
# Use safe navigation operator, since +file+ is nil if an error occurred.
file&.close

38. Where
test "#{a} XOR #{b} results in #{r}" do
Expect
a ^ b == r
Where
a | b | r
0 | 0 | 0
0 | 1 | 1
1 | 0 | 1
1 | 1 | 0
end

39. Code Blocks

40. • Describes ouputs in terms of possible inputs
• Common approach in propositional logic and boolean algebra
Truth Table

41. Truth Table: XOR
case in1 in2 out
1 0 0 0
2 0 1 1
3 1 0 1
4 1 1 0

42. • Ordering test cases makes them easier to find and reason about.
• Ordered data table makes it easier to understand test cases.
• Recommend to order by boolean increment.
Where Blocks

43. Truth Table Generator
$ rake rspock:truth_table -- a=-1,0,1 b=-1,0,1 r='?'
a | b | r
-1 | -1 | '?'
-1 | 0 | '?'
-1 | 1 | '?'
0 | -1 | '?'
0 | 0 | '?'
0 | 1 | '?'
1 | -1 | '?'
1 | 0 | '?'
1 | 1 | '?'

44. Why do this?
test "#{a} XOR #{b} results in #{r}" do
Expect
a ^ b == r
Where
a | b | r
0 | 0 | 0
0 | 1 | 1
1 | 0 | 1
1 | 1 | 0
end

45. test "#{a} XOR #{b} results in #{r}" do
Expect
a ^ b == r
Where
a | b | r
0 | 0 | 0
0 | 1 | 1
1 | 0 | 1
1 | 1 | 0
end
When you can do this:
[
{a: 0, b: 0, r: 0},
{a: 0, b: 1, r: 1},
{a: 1, b: 0, r: 1},
{a: 1, b: 1, r: 1},
].each do |a:, b:, r:|
test "#{a} XOR #{b} results in #{r}" do
assert_equal r, a ^ b
end
end
Why do this?

46. • Executed code != Source code
• Debugging in Pry has slightly different code
What’s the catch?

47. • Better tooling support (i.e. Pry)
• Open-sourcing the project
• More features!!
What’s next?

49. • GitHub: https://github.com/rspockframework/rspock
• RubyGems: https://rubygems.org/gems/rspock
I’m In! How can I use it?

50. Questions?