Page history Edit this page How do I edit this website?

2011-10-07 - Unit tests for ImageJ 1.45

When we launched the ImageJ2 project, we began by writing many unit tests to protect ImageJ 1.x from regression bugs. At this writing, there are unit tests in place for 50 core IJ1 classes, intended to detect bugs introduced during ImageJ development. These tests are located in the ij1-tests repository.

Originally, the tests were designed to run against a modified version of IJ1 that we were developing to facilitate integration with IJ2. However, as development continued, we settled on a design that achieves maximum compatibility by embedding IJ1 as-is within IJ2, enabling IJ1 to continue developing in parallel with IJ2. We eliminated the modified version of IJ1, with only the unit tests being retained. Unfortunately, we did not finish fully transitioning the tests, and they were not compiling against stock versions of IJ1.

As of now, we have fixed up the tests to compile (r4092) and run (r4096, r4098) against stock versions of ImageJ v1.44 and later, and posted instructions for running them.

We then ran the tests against all the 1.45 development versions. The results are below.

Over time, a small but increasing number of tests are failing, which indicates changing behavior creeping into the code. Of course, sometimes changing behavior is good, such as when bugs are fixed. But sometimes these sorts of changes indicate regression bugs being introduced. The only way to know for sure is to examine each failure on an individual basis. For each one, either the test needs to be updated (in the case of a bugfix causing the test to fail when testing for the previous buggy behavior), or the code needs to be fixed to preserve the prior behavior (in the case of a regression bug having been introduced).

Once all the tests pass, Jenkins can run them automatically whenever a new version of ImageJ 1.x is released, and report any failures via email, so that we become aware as early as possible when regression bugs have been introduced.

Version(s)

Results

1.43u

DOES NOT COMPILE (version too old)

1.44o

ALL PASS

1.45a

ALL PASS

1.45b

2 FAILURES:

  1. testFitSplineForStraightening(ij.gui.PolygonRoiTest): array lengths differed, expected.length=4 actual.length=8
  2. testGetUncalibratedLength(ij.gui.PolygonRoiTest): expected:<1.0> but was:<6.324555320336759>

1.45c - 1.45h

4 FAILURES:

  1. testFitSplineForStraightening(ij.gui.PolygonRoiTest): array lengths differed, expected.length=4 actual.length=8
  2. testGetUncalibratedLength(ij.gui.PolygonRoiTest): expected:<1.0> but was:<6.324555320336759>
  3. testGetPixels(ij.VirtualStackTest): arrays first differed at element [2]; expected:<0> but was:<120>
  4. testGetProcessor(ij.VirtualStackTest): arrays first differed at element [2]; expected:<0> but was:<120>

1.45i - 1.45n

5 FAILURES:

  1. testDrawPixelsImageProcessor(ij.gui.ArrowTest): expected:<0> but was:<73>
  2. testFitSplineForStraightening(ij.gui.PolygonRoiTest): array lengths differed, expected.length=4 actual.length=8
  3. testGetUncalibratedLength(ij.gui.PolygonRoiTest): expected:<1.0> but was:<6.324555320336759>
  4. testGetPixels(ij.VirtualStackTest): arrays first differed at element [2]; expected:<0> but was:<120>
  5. testGetProcessor(ij.VirtualStackTest): arrays first differed at element [2]; expected:<0> but was:<120>

1.45o - 1.45q

7 FAILURES:

  1. testDrawPixelsImageProcessor(ij.gui.ArrowTest): expected:<0> but was:<73>
  2. testFitSplineForStraightening(ij.gui.PolygonRoiTest): array lengths differed, expected.length=4 actual.length=8
  3. testGetUncalibratedLength(ij.gui.PolygonRoiTest): expected:<1.0> but was:<6.324555320336759>
  4. testGetConvexHull(ij.gui.ShapeRoiTest)
  5. testGetFeretValues(ij.gui.ShapeRoiTest): arrays first differed at element [0]; expected:<10.44016> but was:<10.44030650891055>
  6. testGetPixels(ij.VirtualStackTest): arrays first differed at element [2]; expected:<0> but was:<120>
  7. testGetProcessor(ij.VirtualStackTest): arrays first differed at element [2]; expected:<0> but was:<120>

1.45r+

See the ImageJ 1.46 unit tests blog post