[ImageJ-devel] Avian VM and Bio-Formats

[Yili Zhao]

Hi Stephan,
  thanks for the information!

  I should have read them in more detail :)

  And thanks for the ITK introduction. I used VTK a bit, and CMake
somtimes, and now I know they are trinity.


2013/10/19 Stephan Saalfeld <saalfeld at mpi-cbg.de>

> Hi Yili,
>
> that is you didn't read beyond the first paragraph of the introduction:
>
> http://bioinformatics.oxfordjournals.org/content/28/22/3009.full
>
> , or the first paragraph of the conclusion in the ImgLib1 workshop
> paper:
>
> http://fly.mpi-cbg.de/~preibisch/pubs/imagejpaper2010.pdf
>
> ?  Depressing for the authors ;).
>
> The other significant source of inspiration is ITK, if you don't know it
> yet, check it out, also very cool stuff:
>
> http://www.itk.org/
>
> Best,
> Stephan
>
>
>
>
> On Sat, 2013-10-19 at 17:13 +0800, Yili Zhao wrote:
> > Hi,
> >   >> Its design was the most important source of inspiration for ImgLib2.
> >   Thanks for this information.
> >   When I read ImgLib2, I have a impression that its design has some
> similar
> > with other library.
> >   Now I know this library is VIGRA :)
> >
> >   By the way, I can not access
> > http://hci.iwr.uni-heidelberg.de/vigra<
> http://hci.iwr.uni-heidelberg.de/vigra/%E2%80%8E>/
> > currently.
> > Do others have this problem?
> >
> >
> > 2013/10/17 Johannes Schindelin <Johannes.Schindelin at gmx.de>
> >
> > > Hi Ullrich,
> > >
> > > a little background information for everybody else: Ullrich is the
> > > inventor and maintainer of VIGRA, an incredibly powerful C++ library to
> > > process and analyze images. Its design was the most important source of
> > > inspiration for ImgLib2. As a consequence, I was really delighted to
> meet
> > > Ullrich at the EuBIAS meeting in Barcelona last week. Among other
> things,
> > > we talked extensively about ways to interoperate between ImgLib2 and
> > > VIGRA, and the lack of support in VIGRA to read file formats via the
> > > Bio-Formats library.
> > >
> > > Also at that meeting, I got into contact with the OpenMole project,
> and in
> > > particular with Mark Hammons who not only explained patiently the
> benefits
> > > of the Scala language to me, but also pointed out the existence of the
> > > Avian VM, a small, embeddable, BSD-licensed Java Virtual Machine
> written
> > > in C++, with a tiny Just-In-Time compiler, just enough to make it a
> > > practical choice for running limited Java inside a C++ program/library:
> > > http://oss.readytalk.com/avian/
> > >
> > > It is actually very, very, *very* limited, as I found out when I tried
> to
> > > get it to run the bfconvert tool of Bio-Formats.
> > >
> > > But I did get it to run bfconvert. To be precise, in my hands, the 5MB
> > > executable compiled from my fork of Avian was able to run my
> > > (minimally-diverging) Bio-Formats v4.4.8 fork's
> > > loci.formats.tools.ImageConverter class to convert Fiji's icon.png (my
> > > standard example) to a .tiff file (although the colors are off, but
> > > running the same example with plain Java results in a byte-identical
> > > file).
> > >
> > > Although it is too early for proper benchmarking, things seem to be
> > > comparable between Java and Avian: while Java is slightly faster in
> what
> > > time(1) calls "real" time (~0.54s vs ~0.72s), in terms of "user" time
> > > roles are reversed (~0.70s vs ~0.63s). Roughly the same holds true for
> the
> > > dynamic Avian executable (which is only 11K and links to Avian's
> libjvm.so
> > > weighing in with a whopping 1.2M). The major benefit will most likely
> be
> > > space: linking to libjvm.so should be enough, the standard Java classes
> > > are included therein. So you can get Java support in VIGRA (or any
> other
> > > C++ project) by adding a library that is slightly larger than one
> > > megabyte.
> > >
> > > Now to some more detailed explanations about the challenges I faced,
> for
> > > the technically-inclined.
> > >
> > > On the Bio-Formats side, there are two major and one minor (and one
> micro)
> > > issue:
> > >
> > > - to enable logging, Bio-Formats uses a cute hack called
> > >   ReflectedUniverse, which is basically a scripting language for Java
> > >   itself. However, it uses regular expressions, something that Avian's
> > >   class library does not yet support.
> > >
> > > - Bio-Formats uses a concurrent hashmap in loci.common.Location.
> > >   Concurrency (JSR-166) is not supported by Avian's class library yet.
> > >
> > > - in loci.formats.gui.LegacyQTTools, the value of the java.library.path
> > >   property is used without checking whether the property is null
> (unset).
> > >
> > > - some debug logging in Bio-Formats uses String.format() (not yet
> > >   supported by Avian's class library).
> > >
> > > All of these issues could be worked around in the code calling
> > > Bio-Formats: there is no need to use the ImageConverter class (which
> > > enables the logging), the concurrent hashmap could be faked by
> extending
> > > the non-concurrent hashmap (and just not bothering with the concurrency
> > > because we will most likely instantiate one VM per Bio-Formats call),
> the
> > > java.library.path property could be initialized to a dummy value and
> the
> > > logging could be switched off completely.
> > >
> > > On the Avian side, I am happy to report that I only had to extend the
> > > class library (apart from one bug fix that I will contribute upstream
> > > later this week). In particular, I had to
> > >
> > > - make RandomAccessFile support writing
> > >
> > > - provide three dummy AWT classes because Bio-Formats actually links to
> > >   them, but does not use them by default
> > >
> > > - provide a couple of interfaces and exceptions that were not yet
> included
> > >   in Avian
> > >
> > > - implement a minimal java.io.DataOutputStream
> > >
> > > - provide a FileChannel for RandomAccessFiles (i.e. implement
> getChannel)
> > >
> > > - implement java.lang.Boolean's parse(String) method
> > >
> > > - implement java.lang.Character's isISOControl(char) method
> > >
> > > - implement fake byte order methods for ByteBuffers (they are
> hardcoded to
> > >   big endian, but that is what Bio-Formats required in my test)
> > >
> > > - provide a dummy SimpleDateFormat which actually does not work, but
> does
> > >   not need to
> > >
> > > - add toString(long[]) to java.util.Arrays
> > >
> > > - teach Avian's pseudo regular expressions about "\\", i.e. the
> backslash
> > >
> > > For the moment, all of these changes are contained in hodge podge
> > > work-in-progress commits which I will clean up over the next few days.
> You
> > > can see them here (the linked pages will update whenever I get around
> to
> > > clean up the commits):
> > >
> > > https://github.com/dscho/avian/compare/bio-formats
> > > https://github.com/dscho/bioformats/compare/v4.4.8...avian
> > >
> > > Since VIGRA and ImgLib2 share design concepts, and since SCIFIO
> supports
> > > ImgLib2 natively, I think that my plan to extend the test to SCIFIO
> after
> > > cleaning up my Avian fork is sound.
> > >
> > > Once that is done, I will add unit tests that our trusty Jenkins will
> run
> > > whenever SCIFIO (or Avian) changes, to ensure that things continue to
> > > work. These unit tests will be extended as needed, e.g. when someone
> finds
> > > out that an important use case requires more changes in Avian, still.
> > >
> > > This way to call Bio-Formats and SCIFIO could become the default C++
> entry
> > > point into those libraries.
> > >
> > > Summary: I showed that Bio-Formats/SCIFIO support for VIGRA and other
> C++
> > > libraries is feasible through the BSD-licensed embeddable Java virtual
> > > machine "Avian". I am confident that I can clean up my patches this
> week
> > > still, to be contributed to the Avian project.
> > >
> > > Ciao,
> > > Johannes
> > >
> > > _______________________________________________
> > > ImageJ-devel mailing list
> > > ImageJ-devel at imagej.net
> > > http://imagej.net/mailman/listinfo/imagej-devel
> > >
> >
> >
> >
> > _______________________________________________
> > ImageJ-devel mailing list
> > ImageJ-devel at imagej.net
> > http://imagej.net/mailman/listinfo/imagej-devel
>
>


-- 
Yili Zhao
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://imagej.net/pipermail/imagej-devel/attachments/20131023/98c1370a/attachment.html>