O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Java code coverage with JCov. Implementation details and use cases.

6.363 visualizações

Publicada em

This slides show
1. How to obtain code coverage information for Java code
2. What kind of code coverage it is possible to get
3. Is 100% block coverage feasible, is it useful
4. How the code coverage could be used for more than discovering a percentage of uncovered code

Publicada em: Software
  • Doctor's 2-Minute Ritual For Shocking Daily Belly Fat Loss! Watch This Video ■■■ http://ishbv.com/bkfitness3/pdf
       Responder 
    Tem certeza que deseja  Sim  Não
    Insira sua mensagem aqui
  • Doctor's 2-Minute Ritual For Shocking Daily Belly Fat Loss! Watch This Video ■■■ https://tinyurl.com/y6qaaou7
       Responder 
    Tem certeza que deseja  Sim  Não
    Insira sua mensagem aqui
  • Doctor's 2-Minute Ritual For Shocking Daily Belly Fat Loss! Watch This Video ♥♥♥ https://tinyurl.com/y6qaaou7
       Responder 
    Tem certeza que deseja  Sim  Não
    Insira sua mensagem aqui
  • I would like to have more information about the 14th slide because you show some commands, great, but you don't show what we need to put instead of "...". By the way, the JCov's wiki page isn't complete at all...
       Responder 
    Tem certeza que deseja  Sim  Não
    Insira sua mensagem aqui

Java code coverage with JCov. Implementation details and use cases.

  1. 1. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  2. 2. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Java Code Coverage with JCov Implementation Details and Use Cases Alexandre (Shura) Iline JDK Test Architect JDK SQE team Oct 1, 2014
  3. 3. Dmitry Fazunenko Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 3 JCov development Alexey Fedorchenko
  4. 4. Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 4
  5. 5. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Program Agenda History, facts Getting the data with JCov Applied to OpenJDK Using the data Links, more information 1 2 3 4 5 5
  6. 6. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 6 Facts • JCov is a Java Code Coverage tool. • The code coverage tool for JCK • The code coverage tool for Oracle JDK. • Other products: –JavaFX –SceneBuilder –...
  7. 7. History • 1996: First prototype • 1997: Integration with JDK 1.1 • 1998: Used “in production” • JDK 1.2 • … • JDK 7 • 2014: Open-source as part of OpenJDK codetools project. • 2014: JCov 3.0 – a public “release” supporting JDK 8 • JDK 9 – in progress Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 7
  8. 8. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. More on JCov dev 1.5 engineer at a time, on average :) Leonid Arbouzov, Alexander Petrov, Vladimir Generalov, Serguei Chukhontsev, Oleg Uliankin, Gregory Steuck, Pavel Ozhdikhin, Konstantin Bobrovsky, Robert Field, Alexander Kuzmin, Leonid Mesnik, Sergey Borodin, Andrey Titov, Dmitry Fazunenko, Alexey Fedorchenko
  9. 9. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Code coverage is ● An information on what source code is exercised in execution 9
  10. 10. most often ... Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Code coverage is ● An information on what source code is exercised in testing 10 Testing is ● Activities to prove that the code behaves as expected where ...
  11. 11. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Other possible usages. • Usage monitoring • Save data in a test environment • Check coverage from generated logic • Fuzzing • Load testing • Regression test generation
  12. 12. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Program Agenda History, facts Getting the data with JCov Applied to OpenJDK Using the data Links, more information 2 12 1 3 4 5
  13. 13. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 13 Getting the data • A simplest use case • A pick or two under the hood • More possibilities – Dynamic vs. static instrumentation –Grabber – Individual test coverage – Abstract coverage – Drop points – Direct coverage – Running big test suites
  14. 14. Simplest use case • Compile java files as usual • “Instrument” the byte code java -jar jcov.jar Instr <application classes> • Run the code java -classpath ...:jcov_file_saver.jar ... • Create a report java -jar jcov.jar RepGen <jcov xml file> demo Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
  15. 15. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. A pick under the hood 15
  16. 16. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. JCov bytecode insertion A pick under the hood public int getX(); Code: 0: ldc #31 // int 2 2: invokestatic #29 // Method com/sun/tdk/jcov/runtime/Collect.hit:(I)V 5: aload_0 6: getfield #2 // Field x:I 9: ireturn 16 demo Warning: the code is a subject to change!!!
  17. 17. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. JCov bytecode insertion As if it was in Java public int getX() { com.sun.tdk.jcov.runtime.Collect.hit(2); return x; } 17 Warning: the code is a subject to change!!!
  18. 18. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Increasing the count One step deeper public class Collect { ... private static long counts[]; ... public static void hit(int slot) { counts[slot]++; } ... } 18 demo
  19. 19. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Saving the data Finally ... Runtime.getRuntime().addShutdownHook(new Thread() { public void run() { ... Collect.disable(); Collect.saveResults(); Collect.enable(); ... } }); 19
  20. 20. Performance impact • Instrumentation phase: –proportional to the application code size. • Execution phase: –Counters increments - very low. Depends on the size of blocks. –Saving data – from significant to blocking. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 20 ● There is a cure! :) More on next slides. • Report generation phase.
  21. 21. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. XML coverage data <class name="Point" supername="java/lang/Object" ...> ... <meth name="getX" vmsig="()I" ...> <bl s="0" e="4"> <methenter s="0" e="4" id="5" count="1"/> <exit s="4" e="4" opcode="ireturn"/> </bl> </meth> </class> 21 demo Warning: the format is a subject to change!!!
  22. 22. Back to the simplest use case • Compile java files as usual • “Instrument” the byte code java -jar jcov.jar Instr <application classes> • Run the code java -classpath ...:jcov_file_saver.jar ... • Create a report java -jar jcov.jar RepGen <jcov xml file> demo Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Incomplete data!!!
  23. 23. Code coverage template • Describes all code there is to cover • Created during instrumentation: java -jar jcov.jar Instr -t template.xml <in> <out> • Created explicitly: java -jar jcov.jar TmplGen -t template.xml <in> • A regular JCov XML file. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. ● Merge ● Filter ● etc. etc.
  24. 24. Simplest use case corrected demo • Compile java files as usual • “Instrument” the byte code java -jar jcov.jar Instr -t template.xml <directory or jar> • Run the code java -classpath ...:jcov_file_saver.jar ... • Merge with the template java -jar jcov.jar Merger -o merge.xml template.xml result.xml • Create a report java -jar jcov.jar RepGen result.xml Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
  25. 25. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Simplest use case with filtering • Compile java code • “Instrument” the byte code java -jar jcov.jar Instr -t template.xml -i com.company.product <application classes> • Run the code, merge with the template • Create a report java -jar jcov.jar RepGen -e com.company.product.test result.xml demo
  26. 26. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. More on getting the data 26
  27. 27. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Dynamic instrumentation • Compile java files as usual • Run with an extra VM option java -classpath ... -javaagent:jcov.jar ... • result.xml generated. • Merge and filter by the template java -jar jcov.jar Merger -o merge.xml -t template.xml result.xml demo demo
  28. 28. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Dynamic vs. static instrumentation Dynamic Static Modification of the tested application Not needed Needed Coverage collected for All code(*) Instrumented code only Test execution Modified to pass more options Modified to add jcov code to the classpath (**) Performance Slows the classloading Faster 28 * There are options to limit the instrumented code ** Or inject the classes into the application itself
  29. 29. “Network” grabber demo • Start the grabber on the network java -jar jcov.jar Grabber … -hostname host123 -port 3333 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. -t template.xml • Run the tests java -classpath ... -javaagent:jcov.jar=grabber,host=host123,port=3333 ... • Stop the grabber – a data file is generated java -jar jcov.jar GrabberManager -kill • Merge, generate report
  30. 30. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Individual code coverage • Track code coverage on a per-test level • “Usefulness” of the tests Test 2 Code Code Code Test 1
  31. 31. Individual test coverage with JCov • Data is encoded into the XML file: <class name="Component" supername="java/lang/Object" ...> ... Copyright © 2014, Oracle and/or its affiliates. All rights reserved. <meth name="&lt;init&gt;" vmsig="()V" ...> <bl s="0" e="151"> <methenter s="0" e="151" id="20" count="90359" scale="08fffffffffee7d8effffffffffffffffffffffffffffffffffffffffffffffff fffff3efffffefffffffffffffffffffffffffffffffffffffefffbfffffffffffffffbf fffffffffffffffffd83fffffffffffffffffffffff1"/> <exit s="151" e="151" opcode="return"/> </bl> </meth> ... </class>
  32. 32. Test scales with files • Run tests separately, save result.xml's $ java -classpath ... -javaagent:jcov.jar ... my.tests.TestN $ cp result.xml result_TestN.xml • Merge data files $ java -jar jcov.jar Merger -o merge.xml Copyright © 2014, Oracle and/or its affiliates. All rights reserved. -outTestList testlist.txt -t template.xml result_*.xml demo
  33. 33. Test scales with grabber • Start the grabber $ java -jar jcov.jar -scale -outTestList testlist.txt -t template.xml • Run tests separately, let the grabber know test names $ java -classpath ... -javaagent:jcov.jar:grabber Copyright © 2014, Oracle and/or its affiliates. All rights reserved. -Djcov.testname=TestN ... my.tests.TestN ● Stop the grabber, generate report. demo
  34. 34. Field coverage • Generate template java -jar jcov.jar TmplGen -field on -t template.xml <in> • Instrumentation ldc // int 1951 invokestatic demo // Method com/sun/tdk/jcov/runtime/CollectDetect.invokeHit:(I)getfield // Field x:I Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
  35. 35. Abstract API coverage demo • Generate template java -jar jcov.jar TmplGen -abstract on -t template.xml <in> • Run with abstract java -classpath ... -javaagent:jcov.jar=abstract=on ... • Instrumentation ldc // int 3012 invokestatic // Method com/sun/tdk/jcov/runtime/CollectDetect.invokeHit:(I)invokeinterface // InterfaceMethod ... Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
  36. 36. Save points • Instrument java -jar jcov.jar Instr -savebegin <a method name> demo Copyright © 2014, Oracle and/or its affiliates. All rights reserved. -t template.xml <classes> • Instrumentation for the method: 0: ldc // int 14 2: invokestatic // Method com/sun/tdk/jcov/runtime/Collect.hit:(I)V 5: invokestatic // Method com/sun/tdk/jcov/runtime/Collect.saveResults:()V
  37. 37. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Direct coverage • Only report code called directly from tests • “Fair” coverage ● Controlled environment ● Meaningful parameters Test Code Code Code
  38. 38. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Direct coverage with no “inner invocation” • Instrument code only (not tests) java -jar jcov.jar Instr -type method -innerinvocation off -t template.xml <classes> • Instrumentation ldc // int -1 invokestatic // Method com/sun/tdk/jcov/runtime/CollectDetect.setExpected:(I)V ... // Some other code ldc // int 0 invokestatic // Method com/sun/tdk/jcov/runtime/CollectDetect.setExpected:(I)V return demo
  39. 39. demo Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Direct coverage with “caller include” • Run tests, instrumenting code and the tests java -classpath ... -javaagent:jcov.jar=ci=<test packages>,type=method ... • Instrumentation of test code invokestatic // Method com/sun/tdk/jcov/runtime/CollectDetect.setExpected:(I)V invokevirtual // some product code call • Instrumention of product code invokestatic // Method com/sun/tdk/jcov/runtime/CollectDetect.hit:(III)V
  40. 40. Collecting data for big suites • Decide on dynamic vs static • Save data periodically –Even in case of same vm –Otherwise risking loosing the data • Using file –One file for all tests – getting bigger in time – huge performance impact, if multiple VMs –Separate files for tests – takes a lot of time to merge • Use the grabber! –Run test consequently for individual test coverage. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
  41. 41. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 41 Getting the data • Instrument dynamically or statically • Save the data onto a file or grabber • Individual test coverage • Abstract coverage • Fields coverage • Data save points • “Direct” coverage
  42. 42. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. There is more. Exec Executes a command collecting coverage data in a grabber Agent print help on usage jcov in dynamic mode Instr instruments classfiles and creates template.xml JREInstr instrumenter designed for instumenting rt.jar ProductInstr Instr2 instrumenter designed for abstract, native methods and TmplGen generates the jcov template.xml Grabber gathers information from JCov runtime via sockets GrabberManager control commands to the Grabber server Merger merges several jcov data files RepMerge merges jcov data files at method level not caring of blocks Filter filters out result data DiffCoverage check whether changed lines were covered RepGen generates text or HTML (or custom) reports JCov gets product coverage with one command
  43. 43. There is more. TmplGen Verbosity: -verbose Template specification: -template(t) 'string value' Type of template: -type [all|branch|block|method] Filtering conditions: -include(i) 'string value', -exclude(e) 'string value', -include_list 'string value', -exclude_list 'string value' Specify which items should be additionally included in template: -abstract [on|off], -native [on|off], -field [on|off] -synthetic [on|off], -anonym [on|off] Flush instrumented classes: -flush 'string value' Basic options: -help(h, ?), -help-verbose(hv) -print-env(env), -propfile 'string value' -plugindir 'string value', -log.file 'string value' -log.level(log) 'string value' Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
  44. 44. There is more. Instr Output: -instr.output(output, o) 'string value' Verbose mode: -verbose Type of template: -type [all|branch|block|method] Filtering conditions: -include(i) 'string value’, -include_list 'string value’, -exclude(e) 'string value’, -caller_include(ci) 'string value’, -caller_exclude(ce) 'string value’, -exclude_list 'string value' Save points: -savebegin 'string value’, -saveatend 'string value' Template specification: -template(t) 'string value’, -subsequent Items to be included: -abstract [on|off], -native [on|off], -field [on|off], -synthetic [on|off], -anonym [on|off], -innerinvocation [on|off] Flush instrumented classes: -flush 'string value' Runtime management: -implantrt(rt) 'string value’, -recursive Basic options: -help(h, ?), -help-verbose(hv), -print-env(env), -propfile 'string value’, -plugindir 'string value’, -log.file 'string value’, -log.level(log) 'string value' Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
  45. 45. There is more. Merger. Output file: -merger.output(output, o) 'string value' File to read jcov input files from: -filelist 'string value' Filtering conditions: -include(i) 'string value' -exclude(e) 'string value', -fm 'string value' -include_list 'string value', -exclude_list 'string value' -fm_list 'string value' Process/generate test scales: -scale, -outTestList 'string value' Verbose mode: -verbose(v) Looseness level: -loose [0|1|2|3|blocks] Compress test scales: -compress Break on error: -breakonerror [file|error|test|skip|none], -critwarn Template path: -template(tmpl, t) 'string value' Skipped files: -outSkipped 'string value' Basic options: -help(h, ?) -help-verbose(hv) -print-env(env) -propfile 'string value' -plugindir 'string value -log.file 'string value' -log.level(log) 'string value Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
  46. 46. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Program Agenda History, facts Getting the data Applied to OpenJDK Using the data Links, more information 2 46 1 3 4 5
  47. 47. OpenJDK coverage. Very simple demo • Template java -jar jcov.jar TmplGen -t template.xml rt.jar • Start grabber java -jar jcov.jar Grabber -v -t template.xml start • Execute tests jtreg ... -javaoptions:"-javaagent:$JCOV_JAR=grabber" <tests> • Stop grabber, merge, generate report java -jar jcov.jar GrabberManager -kill java -jar jcov.jar Merger -o merged.xml -t template.xml Copyright © 2014, Oracle and/or its affiliates. All rights reserved. result.xml java -jar jcov.jar RepGen -src jdk/src/share/classes merged.xml
  48. 48. OpenJDK coverage. Static. • Instrument java -jar jcov.jar JREInstr ... -implant jcov.jar Copyright © 2014, Oracle and/or its affiliates. All rights reserved. -t template.xml .../jre • Run tests, etc
  49. 49. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Program Agenda History, facts Getting the data Applied to OpenJDK Using the data Links, more information 2 49 1 3 4 5
  50. 50. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 50 Using the data • 100% coverage. Needed? Possible? Enough? • Prioritizing • Public API • Monitoring coverage • Speed up test execution • More on test development prioritization • Comparing suites, runs
  51. 51. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 51 Block coverage target value Cost of testing - Defects found * COD = Return
  52. 52. 100% block coverage 1 false • 1 test • 100% pass rate • 100% coverage. • The bug is not discovered Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 52
  53. 53. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 53 100% block and branch coverage 1 true -1 false • 2 tests • 100% pass rate • 100% block coverage. 100% branch coverage. • The bug is not discovered
  54. 54. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 54 Prioritizing test development • 100% coverage is not the goal • Too much code to choose from • Filter the coverage data to leave only code to cover –Public API –UI –Controller code (as in MVC) • Prioritize code –By age –By complexity –By bug density
  55. 55. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 55 Public API
  56. 56. Public API • “Methods which supposed to be called directly from user code” • Every method in public API needs to be called at least once (*) • 100% public API does not prove anything • Necessary. Not sufficient. • Rather blunt solution: java -jar jcov.jar RepGen -publicapi ... –In JDK8: public and protected methods of public and protected class in Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 56 java.* and javax.* (**) –In JDK9: … something different * More later ** Not completely accurate
  57. 57. demo JCov API • com.sun.tdk.jcov.instrument.DataRoot –Load, save • com.oracle.java.sqe.inspection.JCovFinder, JCovInspector, JCovVisitor –walk over coverage data • com.sun.tdk.jcov.instrument.DataBlock, DataBranch, DataClass, DataMethod, DataPackage –Investigate, edit • com.sun.tdk.jcov.filter.MemberFilter, FilterSPI, Filter JCov vommand –filter Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 57
  58. 58. Abstract public API • Code coverage • Good API is abstract • An abstract public API is covered when at least one of its implementations is covered. • Done with JCov abstract coverage and filtering Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 58
  59. 59. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 59 “Public API implementation”
  60. 60. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 60 Public API implementation coverage • Create template with abstract API • Filter, only leaving –Implementations of public API –Extensions of public API • Filter coverage data by the template
  61. 61. Public API implementation closure • It may be required to extend the original set //original public API public interface Action { public abstract void perform();} //some implementation public class MySomethingAction implements Action{ Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 61 @Override public void perform() { //prepare something doPerform(); } protected abstract void doPerform(); } add to filter
  62. 62. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 62 Public API closure • A set of methods implementing library interface –Non-abstract public API –Implementations of abstract public API –Overrides of non-abstract public API –“Delegation” API ● closure of the delegation API demo
  63. 63. demo Sane public API “Every method in public API needs to be called at least once” - Is this so? • Some code is trivial –One line getters –Overloads • Different code requires different techniques. –java.lang.Object.hashCode() • Some code has low impact –Exception constructors • More Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 63
  64. 64. UI code coverage • Similar to public API: –If there is a form in the product UI, it needs to be covered at least once • Identify UI code –constructing UI objects –displaying UI objects –actions with UI Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 64 ● javax.sing.Action.actionPerformed
  65. 65. Controller code coverage • Per MVC –Controller accepts inputs and and converts to commands to view or Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 65 model • Very little boilerplate code • In some cases identifiable by classes and packages • Could be marked explicitly. @Important
  66. 66. Linking CC to other characteristics of the source code Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 66 Characteristic Reasoning Age New code is better to be tested before getting to customer. Old code is either already tested or not needed. :) Number of changes More times the code was changed, more atomic improvements were implemented Bug density Many bugs already found means there are more hidden by existing bugs and more could be introduced with fixes Complexity Assuming similar engineering talent and the same technology … more bugs are likely to exist in complex code
  67. 67. Linking CC to other characteristics of the source code Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 67 ● A formula (1 – cc) * (a 1 *x1 + a *x2 + a 2 *x3 3 + ...) –cc – code coverage (0, 1) –x- characteristic i –a i – importance coefficient • Coefficients are important
  68. 68. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 68 Code coverage monitoring • Build by build • Platform to platform • JCov comparison report • Homegrown database solution (*) • Requires an apparatus to compare the coverage data * More at the end of the presentation
  69. 69. Compare two run on the same build • Comparison report java -jar jcov.jar RepGen file1.xml file2m.xml demo Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 69
  70. 70. demo Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 70 Compare two runs of different builds • Bytecode is not comparable. • Loose the details java -jar jcov.jar Merger -loose blocks file1.xml file2m.xml • Only methods left <meth name="isInvalid" vmsig="()Z" ... id="18" count="150"/> <meth name="getPrefixLength" vmsig="()I" ... id="30" count="28"/> Comparison report java -jar jcov.jar RepGen file1.xml file2m.xml
  71. 71. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 71 Subtraction • Compare two suites A and B • In (cc a – cc b ) only that code which is covered in cc a and not in cc b • Using JCov API –Load cc , cc a b –Walk over cc a –Search in cc b –If found – null the coverage • Review (cc a – cc ) and (cc b b – cc ) a demo
  72. 72. Test base reduction • Collect coverage for tests individually and together. • Compare coverage from every test to the combined coverage. –if a test does not add coverage, execute it less • Conscious choice. –Remember that code coverage proves nothing. –Only use for acceptance test cycles. –Do not throw tests away – run every test although less often. –Analyze tests with give no additional coverage. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 72
  73. 73. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 73 Using the data • 100% coverage. Needed? Possible? Enough? • Getting coverage subsets to cover 100% • Public API • UI • Controller • Monitoring coverage • Test base reduction • Prioritizing test dev by ranking • Comparing suites, runs
  74. 74. More info • Wiki: https://wiki.openjdk.java.net/display/CodeTools/jcov • Source: http://hg.openjdk.java.net/code-tools/jcov • Tutorial: http://hg.openjdk.java.net/code-tools/jcov/raw-file/tip/examples/tutorial/• How to build JCov: https://wiki.openjdk.java.net/display/CodeTools/How+To+Build+• Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 74 “Pragmatic code coverage” on slideshare.net
  75. 75. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 75 More to come • Plugins –IDE: Netbeans, Eclipse, Intellij Idea –maven • Backend –Coverage storage database –Trends, statistics –Ranking • Test base reduction Hey, it's open-source! Contribute.
  76. 76. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Java Code Coverage with JCov Implementation Details and Use Cases Alexandre (Shura) Iline JDK Test Architect JDK SQE team Sept 1, 2014

×