Hi Tobias & everyone,<br><br><blockquote style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote">This has all been happening a bit too fast for me and I'm kind of lost<br>
right now about what exactly is going on here. So could someone please<br>
explain what happened?<br></blockquote>
<br>
Yes, I am sorry I could not send a mail explaining the situation earlier, but we were fully occupied just fixing all the build problems that had accumulated over the past few days.<br><br>Your confusion is understandable, because there were several unrelated issues that required solving.<br>
<br><br><b>== 1) TrakEM2 precompiled was out of date ==<br></b><br>Probably the least significant problem was that building Fiji without having modules/TrakEM2 initialized caused compile errors, due to the precompiled TrakEM2 jar not matching the rest of the codebase. Fixing this required rebuilding TrakEM2 from source and committing the new jar (<a href="https://github.com/fiji/fiji/commit/34ae3188" target="_blank">https://github.com/fiji/fiji/commit/34ae3188</a>). Doing so was complicated only by the fact that the Fiji build system needed a little TLC first (see #2 next).<br>
<br><br><b>== 2) Bugs in the Fiji build system and ImageJ launcher ==<br></b><br>Johannes has been working very hard on the ImageJ launcher, in preparation for inclusion with ImageJ2. Initially, much of that work was done in fiji.git on ImageJ.c. The code has been steadily improving, but as with all such changes, new bugs can be introduced or brought to light as a result:<br>
<br>A) Dscho & I tracked down a bug with the launcher on OS X, noticed by Steffi: <a href="https://github.com/fiji/fiji/commit/a7d90f38" target="_blank">https://github.com/fiji/fiji/commit/a7d90f38</a><br><br>B) Dscho fixed another bug specific to OS X: <a href="https://github.com/fiji/fiji/commit/25e58077" target="_blank">https://github.com/fiji/fiji/commit/25e58077</a><br>
<br>C) Steffi fixed a problem with Fiji build on Windows: <a href="https://github.com/fiji/fiji/commit/a85acab1" target="_blank">https://github.com/fiji/fiji/commit/a85acab1</a><br><br>Once those problems were fixed, we were able to finish solving the tougher issues, below.<br>
<br><br><b>== 3) The OpenJDK6 Javac bug ==<br></b><br>The OpenJDK6 problem has nothing to do with Java5. There are several usages of generics in ImgLib that Fiji's OpenJDK6 compiler failed to handle
properly. To resolve it, Johannes upgraded Fiji's version of OpenJDK6 to the latest available (<a href="https://github.com/fiji/fiji/commit/402dce11" target="_blank">https://github.com/fiji/fiji/commit/402dce11</a>). However, this latest version introduced some new problems when compiling SPIM registration, so Johannes partially reverted it (<a href="https://github.com/fiji/fiji/commit/82bd2950" target="_blank">https://github.com/fiji/fiji/commit/82bd2950</a>). But still a few bugs remained when compiling ImgLib. To work around them, I introduced the ugly "HACK" changes (<a href="https://github.com/imagej/imglib/commit/5748a1be" target="_blank">https://github.com/imagej/imglib/commit/5748a1be</a>).<br>
<br>I know these HACKs are unpleasant, but the fact is that we are targeting three different compilers: Oracle's Java6, OpenJDK6, and Eclipse's JDT compiler. If ImgLib fails to compile with any of them, then our current workflows have problems.<br>
<br>In the future, as OpenJDK7 becomes the gold standard for compilation, and Java7 is more easily available for all major platforms, the dichotomy between compilation with Fiji build (which uses OpenJDK6) and Maven on the command line (which uses your installed Java, often Oracle's Java6, depending on your platform) should disappear. We will likely need to remain cautious indefinitely with respect to Eclipse's JDT compiler, though.<br>
<br><div> </div><blockquote style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote">Is there anything I have to be careful about now, so that I don't break anything?</blockquote>
<div><br>As always, just because something works in Eclipse does not mean it will work with Oracle's Java6, or with OpenJDK6. In a perfect world, we would all test our changes with all three compilers (on all three OSes too!), but personally I think asking everyone to always do that is unrealistic. That's what Jenkins is for. Just don't be surprised if sometimes the build breaks even though Eclipse had no problems. And if you see that one of your changes broke the build, it is extremely helpful if you can find a fix/workaround, rather than waiting for Dscho or me to fix it. J-Y asked in another thread ("Fiji does not compile anymore under Mac and Windows") about the best way to test compilation in Fiji build, etc.—on Monday I will reply there with more details, and create a new page on the Fiji wiki about it.<br>
<br>As a side note, it can happen the other way too: yesterday Johannes merged the ImageJ launcher to ImageJ2, and it is all working fine from the command line, but not from Eclipse. ;-) The reality is that it takes work to keep everything running smoothly across the different paradigms.<br>
<br><br><b>== 4) The Java5 compatibility snafu ==<br></b><br>Recently, I asked whether we should migrate the imglib2-io project into SCIFIO (a.k.a. Bio-Formats), and people responded favorably. So Mark Hiner & I did it on a topic branch, and when Melissa Linkert visited last week, we merged the changes (<a href="https://github.com/openmicroscopy/bioformats/pull/39">https://github.com/openmicroscopy/bioformats/pull/39</a>, ). However, we forgot one important detail: SCIFIO still uses a Java5 compiler to build. (Actually, the entire OME software stack does.) So the OME Jenkins failed to build SCIFIO with the new ImgLib I/O classes, and we scrambled to find a solution. We created a temporary Java5 version of imglib2.jar that we could use, and fixed a couple of other oversights (<a href="https://github.com/openmicroscopy/bioformats/pull/45">https://github.com/openmicroscopy/bioformats/pull/45</a>).<br>
<br>That fixed the build problems, but Dscho & I wanted to find a more permanent solution to the build compatibility problem between SCIFIO and ImgLib2. I had an idea for a compromise—target Java 1.5 for the core imglib2 project—which I implemented on Thursday (<a href="https://github.com/imagej/imglib/commit/c66f26e6">https://github.com/imagej/imglib/commit/c66f26e6</a>). I did not know that Eclipse would misbehave when importing the project until afterwards—but that problem is now fixed (see below).<br>
<br>As an aside, I would like to point out that compiling ImgLib to target Java5 is nothing new. Fiji build has done this from the beginning, and not just for imglib2 core, but for *all* of ImgLib (<a href="https://github.com/fiji/fiji/commit/702c921b">https://github.com/fiji/fiji/commit/702c921b</a>). You'll notice the line "javaVersion=1.5" which tells Fiji build to compile the code to class version 49.0 (Java 1.5). So those of you building ImgLib via Fiji build are already using class version 49.0.<br>
<br><br><blockquote style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote">
Are there plans to figure out the performance implications of using the old class format?</blockquote><br>
As Dscho says, we have no plans to test it ourselves. However, it would be very easy to run the ImgLib2 benchmarks (<a href="https://github.com/imagej/imglib/blob/master/imglib2/ij/src/test/scripts/benchmark.sh" target="_blank">https://github.com/imagej/imglib/blob/master/imglib2/ij/src/test/scripts/benchmark.sh</a>) with a Java5 jar vs. a Java6 one.<br>
<br>I have absolutely no idea if performance is impacted by using a Java5-compatible jar. Personally, I doubt it. However, Johannes mentioned offhand that it might be a possibility, and I agreed, so I mentioned it in the commit message. But it is pure speculation. ImgLib has proven to be extremely sensitive to small changes (because the JIT is sensitive), so we thought it worth mentioning. But I did not mean to suggest that such performance problems were necessarily likely—merely conceivable.<br>
<br><br></div><blockquote style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote">Can I (locally) just remove the
maven-compiler-plugin section from the pom and have everything working
as before?<br></blockquote><br>The question is now moot, because in a commit earlier today, I disabled the Java5 targeting by default (<a href="https://github.com/imagej/imglib/commit/b67f246f" target="_blank">https://github.com/imagej/imglib/commit/b67f246f</a>). It must now be enabled explicitly using a profile. Things should now work in Eclipse again, exactly as before. Jenkins now turns on the Java5 targeting when it builds, so that the imglib2.jar deployed to the Maven repository (which SCIFIO builds against) will remain Java5 compatible. But those of us using Eclipse will not suffer any inconvenience.<br>
<br>Regarding making local changes to a POM: in general I would discourage it. If you find yourself needing to do so, we should discuss what problem you are trying to solve, and what an easier and more effective solution might be.<br>
<br><br><b>== 5) Continuing changes to ImgLib2 I/O classes ==<br></b><br>We have been testing ImageJ2's I/O plugins, which use the ImgLib2 ImgOpener and ImgSaver classes. The ImgSaver in particular is very new, and still under active development. Unfortunately, the turnaround time to merge changes into SCIFIO, and make them available from ImageJ's Maven repository, is days to weeks. So while there is another pull request pending to update the ImgLib I/O code in SCIFIO (<a href="https://github.com/openmicroscopy/bioformats/pull/49">https://github.com/openmicroscopy/bioformats/pull/49</a>), for the impending ImageJ2 beta we needed a solution more rapidly.<br>
<br>So what we did is to commit the latest version back to the imglib2-io project of imglib.git as well. Now, the code lives in both projects, with two different package names (<a href="http://net.imglib2.io">net.imglib2.io</a>, and ome.scifio.img). At the moment, the code is in sync. The long term plan is still to remove the imglib2-io project in favor of it being part of SCIFIO, but it will have to wait until we can achieve the speed of integrated multi-project development we are accustomed to with Fiji. The good news is that there is now a working ImgSaver available for use (feedback welcome!).<br>
<br><br>Hopefully this clarifies the recent changes to Fiji and ImgLib. Please let us know if anything else is unclear.<br><br>Regards,<br>Curtis<br><br><br><div class="gmail_quote">On Sat, Mar 31, 2012 at 5:10 AM, Tobias Pietzsch <span dir="ltr"><<a href="mailto:pietzsch@mpi-cbg.de" target="_blank">pietzsch@mpi-cbg.de</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi guys,<br>
<br>
This has all been happening a bit too fast for me and I'm kind of lost<br>
right now about what exactly is going on here. So could someone please<br>
explain what happened?<br>
<br>
There have been ugly "HACK"s introduced like the following:<br>
<br>
-final T buffer = Util.getTypeFromInterval( fftImage ).createVariable();<br>
+// HACK: Explicit assignment is needed for OpenJDK javac.<br>
+final T fftImageType = Util.getTypeFromInterval( fftImage );<br>
+final T buffer = fftImageType.createVariable();<br>
<br>
It seems, that this has been limited (fortunately!!!) to imglib2-algorithms-gpl. But why? Surely, there are similar constructs in<br>
other parts of imglib. Why are they not affected?<br>
<br>
Are there plans to figure out the performance implications of using the<br>
old class format? Is there anything I have to be careful about now, so<br>
that I don't break anything? Can I (locally) just remove the maven-compiler-plugin section from the pom and have everything working as before?<br>
<br>
And honestly, I don't like that this was done without any advance warning or discussion. This seems to be a big thing with potential<br>
performance implications.<br>
<br>
best regards,<br>
Tobias<br>
</blockquote></div><br>