[ImageJ-devel] IJ2 Plugin Proposal

[Aivar Grislis]

I also just sent this to ImageJX, forgot to cc: them originally.

-------- Original Message --------
Subject: 	IJ2 Plugin Proposal
Date: 	Wed, 24 Nov 2010 15:16:14 -0600
From: 	Aivar Grislis <grislis at wisc.edu>
To: 	imagej-devel at imagejdev.org



I'm proposing a more dynamic way of running plugins and sending images
from one plugin to another that would lend itself to workflows and
pipelines.  This could also be used within plugins and within IJ2 as an
architectural building block to chain chunks of image processing code
together.

Basically I propose that a plugin can have multiple named input and
output images and that whenever an output image is ready it is passed on
to become the input image to the next plugin in the chain.

IMPETUS

I feel the need for this from a couple of plugins I have worked on
(actually my only plugins!).

I worked on a Deep Zoom plugin.  This takes a huge image, tiles it, then
the tiles are written out in a particular directory/file name
structure.  Next we halve the original image and repeat, until the image
gets halved to a miniscule size.

While working on this plugin I thought about writing the code to halve
the image and to tile it in the most general-purpose, reusable way.
Also I wanted to be memory-efficient: I didn't want to just make all the
halves, then go through them and make all the tiles, then go throughout
those and write out all the files.

What I came up with at that time was a chainable plugin.  In pseudo-code:

interface IChainablePlugin {

   // chains one plugin to next; called from outside the plugin
   public chain(IChainablePlugin plugin);

   // gets the input image; called from within plugin
   Image get();

   // puts the output image; called from within plugin
   put(Image image);
}

An abstract base class kept track of the next, chained plugin instance
and handed off the output image to that next plugin, using the same thread.

I wrote a chainable plugin class to halve the image, one to tile it, and
had a third one that was just an inner class to my Deep Zoom plugin that
handled the specifics of writing out the tiled image files.  Presumably
the first two classes would be reusable code and the third was just
specific to Deep Zoom.

Obviously as-is this is limited to one input and one output.  I started
thinking about having an image dispatcher class that would keep track of
which images a plugin required and hand them off from one plugin to another.

I'll just briefly mention the SLIM plugin I am working on.  Since the
SLIM fitting process is time consuming I display a very coarse colorized
version of the fitted results that is refined as the fit progresses.  So
there I want to pass on my output image before the plugin finishes and I
want to put out successive versions of this image.  This is similar to
my wanting to process the individual tiles before the whole image is tiled.

PROPOSAL

We could label the input and output images using Java annotations.
(Note that the current Declarative Plugin annotations allow one to label
input and output image member variables but the outputs are harvested
only once the plugin is finished. There is no way to put out an output
image within the plugin code.)

Here is some sample code for what I'm proposing:

"@Input
@Output({ @Img(Plugin1.BLUE), @Img(Plugin1.GREEN) })
public class Plugin1 extends AbstractPlugin {
   public static final String BLUE = "Blue";
   public static final String GREEN = "Green";

   void process() {
       Image image = get();
       put(BLUE, image);
       put(GREEN, image);
   }
}"

Here the annotations result in an input name set of ["Default"] and an
output set of ["Blue","Green"].  (If there are no parameters the default
name is used.)  Of course, there could also be named input images (i.e.
"get(ORANGE)") or default output images (i.e. "put()").

Instances of plugins such as these could be chained together:
   plugin1.chain(Plugin1.BLUE, plugin2);  // named output goes to
default input
   plugin1.chain(Plugin1.GREEN, plugin2, Plugin2.ORANGE); // named
output goes to named input
   plugin2.chain(plugin3); // default output goes to default input
   plugin2.chain(plugin4, Plugin4.PURPLE); // default output goes to
named input

Using strings as identifiers might seem less than ideal.  However, the
strings are unique to the plugin they belong to.  If they are specified
as static constants, as above, that would prevent spelling mismatches.
Once a chain of plugins is established and before we run it we could
make sure that all of the inputs have been chained to.  We can detect if
a plugin asks for an input or output that has not been annotated and
warn the plugin programmer.

Of course, once we have named input and output images we could build on
that and give the IJ end-user ways to link up plugins in workflows or
pipelines, a la CellProfiler.

I have a prototype implementation of this, so the sample code above
works, details upon request.

Aivar

(P.S. It's the Thanksgiving holiday in the U.S. so you might not hear
from us until Monday.  Happy Thanksgiving!)








-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://imagej.net/pipermail/imagej-devel/attachments/20101124/2e7f47ae/attachment.html>