One of the things I was always wondering about is if we can use the JDWP protocol to do code coverage in testing. I imaging a debugger that instead of stopping at breakpoints, just tracks what code is running and what branches are covered, and outputs a coverage report.
Today all (I think) code coverage tools use code instrumentation, potentially changing test results (as we run a modified version of the code we test). The impact of this is that most coverage tools suggest running tests twice - once without coverage tracking (the actual tests) and again with coverage, a practice that slows down builds.
Using the debugger interface for tracking code coverage means we are running unchanged code in tests, those not requiring a second run of tests just for coverage.
In addition, using the debugger interface, we can look at variable values and estimate coverage based on different states of variables in a statement.
Have you come across JaCoCo [1]? It does its instrumentation via a JVM agent at runtime-- no need to recompile your code just to test for coverage.
I've been using it for the last year or two and love it! One of my favorite features is how easy it is to generate separate coverage information for unit tests vs. integration tests. Sonar provides excellent support for it, as well.
It is a great tool, and I have used it. I have also used Emma.
Both use instrumentation at loading time, which is great (because it is transparent). However, both recommend to run tests twice because of the instrumentation. In both cases, the Maven plugin runs tests twice - once for testing, and once for coverage - which is a total waist of time.
I was thinking on using the debug interface to track coverage without instrumentation, those without the risk of modifying the tests.
Am I missing something or does this post have a really poor title? The article is a very brief and high-level overview of some of the main topic headings related to writing debuggers. You are barely any closer to writing your own debugger after reading this article.
Yes, I thought that this would have more info that would be useful to get going.
Does anyone know of some open source java debugger code that I could hook up with a ncurses front-end ? that would be useful when I'm debugging customer issues.
Today all (I think) code coverage tools use code instrumentation, potentially changing test results (as we run a modified version of the code we test). The impact of this is that most coverage tools suggest running tests twice - once without coverage tracking (the actual tests) and again with coverage, a practice that slows down builds.
Using the debugger interface for tracking code coverage means we are running unchanged code in tests, those not requiring a second run of tests just for coverage.
In addition, using the debugger interface, we can look at variable values and estimate coverage based on different states of variables in a statement.