[ImageJ-devel] Labeling - Revised Branch

[Johannes Schindelin]

Hi all,

On Tue, 20 Mar 2012, Johannes Schindelin wrote:

> On Tue, 20 Mar 2012, Tobias Pietzsch wrote:
> 
> > On 03/20/2012 05:26 PM, Johannes Schindelin wrote:
> > >
> > > On Tue, 20 Mar 2012, Tobias Pietzsch wrote:
> > >
> > > > Did you do any benchmarks to see how much runtime-overhead you
> > > > have there?
> > >
> > > I did a little DuckDuckGo search and it seems as if there is a
> > > Performance Plugin for Jenkins. It can use the output of JUnit, so
> > > maybe it would be a fun thing to implement these performance tests
> > > as JUnit tests? Then we could just configure our Jenkins to test the
> > > performance with every new commit (or at least with as many as
> > > Jenkins catches through polling) and see how performance improves...
> >
> > That would be awesome!  Is there anything special you need to
> > implement, or does it just use every test?
> 
> As far as I know, it is plain JUnit tests. If you look at the output
> Eclipse provides, it also has some numbers. So if you want to test
> performance realistically, I guess that you have to either make big
> enough artificial data or run the algorithm often enough to result in
> some multi-second numbers. But if I read the documentation for the
> Performance plugin correctly, it can pick up those tests, run them, and
> show nice graphs over time and stuff, like statistics, trends, etc. And
> if it does not, keep in mind, this is Open Source, I can most likely
> change it so it does.

I dropped some hints earlier that I installed the Performance plugin to
our jolly Jane Jenkins, and I thought I should follow up with some
details:

* it is installed:

	http://jenkins.imagej.net/job/ImageJ/performance/

	http://jenkins.imagej.net/job/ImgLib/performance/

* it seems as if the Performance plugin mistakes the test classes to
  implement identical tests and tries to determine their average running
  time and median. Our tests classes, however, contain *related* tests,
  i.e. we do not expect them to run in roughly the same time

* Note that the finer-grained information is present in the files (so once
  I find some time, I will enhance the Performance plugin to show
  finer-grained JUnit timings)

Ciao,
Dscho