[ImageJ-devel] Regarding Target Java 1.5 for imglib2 core

[Tobias Pietzsch]

Hi Curtis,

Thanks a lot for the explanations. That was really helpful.

I thought it was all somehow caused by targeting java 1.5.
I did not know that Fiji used the 49.0 class version anyway.
So I'm much more comfortable with that now. I will still run
the benchmark to compare the class versions, just to know for
sure.

best regards,
Tobias

On 04/01/2012 05:29 AM, Curtis Rueden wrote:
> Hi Tobias & everyone,
>
>     This has all been happening a bit too fast for me and I'm kind of lost
>     right now about what exactly is going on here.  So could someone please
>     explain what happened?
>
>
> Yes, I am sorry I could not send a mail explaining the situation
> earlier, but we were fully occupied just fixing all the build problems
> that had accumulated over the past few days.
>
> Your confusion is understandable, because there were several unrelated
> issues that required solving.
>
>
> *== 1) TrakEM2 precompiled was out of date ==
> *
> Probably the least significant problem was that building Fiji without
> having modules/TrakEM2 initialized caused compile errors, due to the
> precompiled TrakEM2 jar not matching the rest of the codebase. Fixing
> this required rebuilding TrakEM2 from source and committing the new jar
> (https://github.com/fiji/fiji/commit/34ae3188). Doing so was complicated
> only by the fact that the Fiji build system needed a little TLC first
> (see #2 next).
>
>
> *== 2) Bugs in the Fiji build system and ImageJ launcher ==
> *
> Johannes has been working very hard on the ImageJ launcher, in
> preparation for inclusion with ImageJ2. Initially, much of that work was
> done in fiji.git on ImageJ.c. The code has been steadily improving, but
> as with all such changes, new bugs can be introduced or brought to light
> as a result:
>
> A) Dscho & I tracked down a bug with the launcher on OS X, noticed by
> Steffi: https://github.com/fiji/fiji/commit/a7d90f38
>
> B) Dscho fixed another bug specific to OS X:
> https://github.com/fiji/fiji/commit/25e58077
>
> C) Steffi fixed a problem with Fiji build on Windows:
> https://github.com/fiji/fiji/commit/a85acab1
>
> Once those problems were fixed, we were able to finish solving the
> tougher issues, below.
>
>
> *== 3) The OpenJDK6 Javac bug ==
> *
> The OpenJDK6 problem has nothing to do with Java5. There are several
> usages of generics in ImgLib that Fiji's OpenJDK6 compiler failed to
> handle properly. To resolve it, Johannes upgraded Fiji's version of
> OpenJDK6 to the latest available
> (https://github.com/fiji/fiji/commit/402dce11). However, this latest
> version introduced some new problems when compiling SPIM registration,
> so Johannes partially reverted it
> (https://github.com/fiji/fiji/commit/82bd2950). But still a few bugs
> remained when compiling ImgLib. To work around them, I introduced the
> ugly "HACK" changes (https://github.com/imagej/imglib/commit/5748a1be).
>
> I know these HACKs are unpleasant, but the fact is that we are targeting
> three different compilers: Oracle's Java6, OpenJDK6, and Eclipse's JDT
> compiler. If ImgLib fails to compile with any of them, then our current
> workflows have problems.
>
> In the future, as OpenJDK7 becomes the gold standard for compilation,
> and Java7 is more easily available for all major platforms, the
> dichotomy between compilation with Fiji build (which uses OpenJDK6) and
> Maven on the command line (which uses your installed Java, often
> Oracle's Java6, depending on your platform) should disappear. We will
> likely need to remain cautious indefinitely with respect to Eclipse's
> JDT compiler, though.
>
>     Is there anything I have to be careful about now, so that I don't
>     break anything?
>
>
> As always, just because something works in Eclipse does not mean it will
> work with Oracle's Java6, or with OpenJDK6. In a perfect world, we would
> all test our changes with all three compilers (on all three OSes too!),
> but personally I think asking everyone to always do that is unrealistic.
> That's what Jenkins is for. Just don't be surprised if sometimes the
> build breaks even though Eclipse had no problems. And if you see that
> one of your changes broke the build, it is extremely helpful if you can
> find a fix/workaround, rather than waiting for Dscho or me to fix it.
> J-Y asked in another thread ("Fiji does not compile anymore under Mac
> and Windows") about the best way to test compilation in Fiji build,
> etc.—on Monday I will reply there with more details, and create a new
> page on the Fiji wiki about it.
>
> As a side note, it can happen the other way too: yesterday Johannes
> merged the ImageJ launcher to ImageJ2, and it is all working fine from
> the command line, but not from Eclipse. ;-) The reality is that it takes
> work to keep everything running smoothly across the different paradigms.
>
>
> *== 4) The Java5 compatibility snafu ==
> *
> Recently, I asked whether we should migrate the imglib2-io project into
> SCIFIO (a.k.a. Bio-Formats), and people responded favorably. So Mark
> Hiner & I did it on a topic branch, and when Melissa Linkert visited
> last week, we merged the changes
> (https://github.com/openmicroscopy/bioformats/pull/39, ). However, we
> forgot one important detail: SCIFIO still uses a Java5 compiler to
> build. (Actually, the entire OME software stack does.) So the OME
> Jenkins failed to build SCIFIO with the new ImgLib I/O classes, and we
> scrambled to find a solution. We created a temporary Java5 version of
> imglib2.jar that we could use, and fixed a couple of other oversights
> (https://github.com/openmicroscopy/bioformats/pull/45).
>
> That fixed the build problems, but Dscho & I wanted to find a more
> permanent solution to the build compatibility problem between SCIFIO and
> ImgLib2. I had an idea for a compromise—target Java 1.5 for the core
> imglib2 project—which I implemented on Thursday
> (https://github.com/imagej/imglib/commit/c66f26e6). I did not know that
> Eclipse would misbehave when importing the project until afterwards—but
> that problem is now fixed (see below).
>
> As an aside, I would like to point out that compiling ImgLib to target
> Java5 is nothing new. Fiji build has done this from the beginning, and
> not just for imglib2 core, but for *all* of ImgLib
> (https://github.com/fiji/fiji/commit/702c921b). You'll notice the line
> "javaVersion=1.5" which tells Fiji build to compile the code to class
> version 49.0 (Java 1.5). So those of you building ImgLib via Fiji build
> are already using class version 49.0.
>
>
>     Are there plans to figure out the performance implications of using
>     the old class format?
>
>
> As Dscho says, we have no plans to test it ourselves. However, it would
> be very easy to run the ImgLib2 benchmarks
> (https://github.com/imagej/imglib/blob/master/imglib2/ij/src/test/scripts/benchmark.sh)
> with a Java5 jar vs. a Java6 one.
>
> I have absolutely no idea if performance is impacted by using a
> Java5-compatible jar. Personally, I doubt it. However, Johannes
> mentioned offhand that it might be a possibility, and I agreed, so I
> mentioned it in the commit message. But it is pure speculation. ImgLib
> has proven to be extremely sensitive to small changes (because the JIT
> is sensitive), so we thought it worth mentioning. But I did not mean to
> suggest that such performance problems were necessarily likely—merely
> conceivable.
>
>
>     Can I (locally) just remove the maven-compiler-plugin section from
>     the pom and have everything working as before?
>
>
> The question is now moot, because in a commit earlier today, I disabled
> the Java5 targeting by default
> (https://github.com/imagej/imglib/commit/b67f246f). It must now be
> enabled explicitly using a profile. Things should now work in Eclipse
> again, exactly as before. Jenkins now turns on the Java5 targeting when
> it builds, so that the imglib2.jar deployed to the Maven repository
> (which SCIFIO builds against) will remain Java5 compatible. But those of
> us using Eclipse will not suffer any inconvenience.
>
> Regarding making local changes to a POM: in general I would discourage
> it. If you find yourself needing to do so, we should discuss what
> problem you are trying to solve, and what an easier and more effective
> solution might be.
>
>
> *== 5) Continuing changes to ImgLib2 I/O classes ==
> *
> We have been testing ImageJ2's I/O plugins, which use the ImgLib2
> ImgOpener and ImgSaver classes. The ImgSaver in particular is very new,
> and still under active development. Unfortunately, the turnaround time
> to merge changes into SCIFIO, and make them available from ImageJ's
> Maven repository, is days to weeks. So while there is another pull
> request pending to update the ImgLib I/O code in SCIFIO
> (https://github.com/openmicroscopy/bioformats/pull/49), for the
> impending ImageJ2 beta we needed a solution more rapidly.
>
> So what we did is to commit the latest version back to the imglib2-io
> project of imglib.git as well. Now, the code lives in both projects,
> with two different package names (net.imglib2.io
> <http://net.imglib2.io>, and ome.scifio.img). At the moment, the code is
> in sync. The long term plan is still to remove the imglib2-io project in
> favor of it being part of SCIFIO, but it will have to wait until we can
> achieve the speed of integrated multi-project development we are
> accustomed to with Fiji. The good news is that there is now a working
> ImgSaver available for use (feedback welcome!).
>
>
> Hopefully this clarifies the recent changes to Fiji and ImgLib. Please
> let us know if anything else is unclear.
>
> Regards,
> Curtis
>
>
> On Sat, Mar 31, 2012 at 5:10 AM, Tobias Pietzsch <pietzsch at mpi-cbg.de
> <mailto:pietzsch at mpi-cbg.de>> wrote:
>
>     Hi guys,
>
>     This has all been happening a bit too fast for me and I'm kind of lost
>     right now about what exactly is going on here.  So could someone please
>     explain what happened?
>
>     There have been ugly "HACK"s introduced like the following:
>
>     -final T buffer = Util.getTypeFromInterval( fftImage ).createVariable();
>     +// HACK: Explicit assignment is needed for OpenJDK javac.
>     +final T fftImageType = Util.getTypeFromInterval( fftImage );
>     +final T buffer = fftImageType.createVariable();
>
>     It seems, that this has been limited (fortunately!!!) to
>     imglib2-algorithms-gpl.  But why? Surely, there are similar
>     constructs in
>     other parts of imglib. Why are they not affected?
>
>     Are there plans to figure out the performance implications of using the
>     old class format?  Is there anything I have to be careful about now, so
>     that I don't break anything?  Can I (locally) just remove the
>     maven-compiler-plugin section from the pom and have everything
>     working as before?
>
>     And honestly, I don't like that this was done without any advance
>     warning or discussion. This seems to be a big thing with potential
>     performance implications.
>
>     best regards,
>     Tobias
>
>