[ImageJ-devel] IJ2 Plugin Proposal

[Lee Kamentsky]

Hi Aivar,
I'm sort of doing this with the @Parameter attributes now and 
CellProfiler. I use reflection to get the class of the tagged variable. 
The plugin's image inputs are variables of type ImagePlus (and probably 
later, imglib's Image) that are not marked with output=true and the 
outputs are the ones with output=true. Perhaps it's a bit of a hack to 
infer that something is intended to be used as an image by its class 
type, but that's what I did; the other possibility would be to indicate 
that it is an image explicitly in @Parameter.

CellProfiler works beautifully with this - maybe Curtis can show you how 
it does with ImageJ plugins.

I can imagine writing your tool (or using something like Knime) with the 
current infrastructure. You have some sort of hookup gui (and Knime does 
this for you) that allows you to attach image inputs to outputs and it's 
up to the tool to figure out when a plugin's inputs are ready.

--Lee

On 11/24/2010 4:16 PM, Aivar Grislis wrote:
> I'm proposing a more dynamic way of running plugins and sending images 
> from one plugin to another that would lend itself to workflows and 
> pipelines.  This could also be used within plugins and within IJ2 as 
> an architectural building block to chain chunks of image processing 
> code together.
>
> Basically I propose that a plugin can have multiple named input and 
> output images and that whenever an output image is ready it is passed 
> on to become the input image to the next plugin in the chain.
>
> IMPETUS
>
> I feel the need for this from a couple of plugins I have worked on 
> (actually my only plugins!).
>
> I worked on a Deep Zoom plugin.  This takes a huge image, tiles it, 
> then the tiles are written out in a particular directory/file name 
> structure.  Next we halve the original image and repeat, until the 
> image gets halved to a miniscule size.
>
> While working on this plugin I thought about writing the code to halve 
> the image and to tile it in the most general-purpose, reusable way.  
> Also I wanted to be memory-efficient: I didn't want to just make all 
> the halves, then go through them and make all the tiles, then go 
> throughout those and write out all the files.
>
> What I came up with at that time was a chainable plugin.  In pseudo-code:
>
> interface IChainablePlugin {
>
>   // chains one plugin to next; called from outside the plugin
>   public chain(IChainablePlugin plugin);
>
>   // gets the input image; called from within plugin
>   Image get();
>
>   // puts the output image; called from within plugin
>   put(Image image);
> }
>
> An abstract base class kept track of the next, chained plugin instance 
> and handed off the output image to that next plugin, using the same 
> thread.
>
> I wrote a chainable plugin class to halve the image, one to tile it, 
> and had a third one that was just an inner class to my Deep Zoom 
> plugin that handled the specifics of writing out the tiled image 
> files.  Presumably the first two classes would be reusable code and 
> the third was just specific to Deep Zoom.
>
> Obviously as-is this is limited to one input and one output.  I 
> started thinking about having an image dispatcher class that would 
> keep track of which images a plugin required and hand them off from 
> one plugin to another.
>
> I'll just briefly mention the SLIM plugin I am working on.  Since the 
> SLIM fitting process is time consuming I display a very coarse 
> colorized version of the fitted results that is refined as the fit 
> progresses.  So there I want to pass on my output image before the 
> plugin finishes and I want to put out successive versions of this 
> image.  This is similar to my wanting to process the individual tiles 
> before the whole image is tiled.
>
> PROPOSAL
>
> We could label the input and output images using Java annotations.  
> (Note that the current Declarative Plugin annotations allow one to 
> label input and output image member variables but the outputs are 
> harvested only once the plugin is finished. There is no way to put out 
> an output image within the plugin code.)
>
> Here is some sample code for what I'm proposing:
>
> "@Input
> @Output({ @Img(Plugin1.BLUE), @Img(Plugin1.GREEN) })
> public class Plugin1 extends AbstractPlugin {
>   public static final String BLUE = "Blue";
>   public static final String GREEN = "Green";
>
>   void process() {
>       Image image = get();
>       put(BLUE, image);
>       put(GREEN, image);
>   }
> }"
>
> Here the annotations result in an input name set of ["Default"] and an 
> output set of ["Blue","Green"].  (If there are no parameters the 
> default name is used.)  Of course, there could also be named input 
> images (i.e. "get(ORANGE)") or default output images (i.e. "put()").
>
> Instances of plugins such as these could be chained together:
>   plugin1.chain(Plugin1.BLUE, plugin2);  // named output goes to 
> default input
>   plugin1.chain(Plugin1.GREEN, plugin2, Plugin2.ORANGE); // named 
> output goes to named input
>   plugin2.chain(plugin3); // default output goes to default input
>   plugin2.chain(plugin4, Plugin4.PURPLE); // default output goes to 
> named input
>
> Using strings as identifiers might seem less than ideal.  However, the 
> strings are unique to the plugin they belong to.  If they are 
> specified as static constants, as above, that would prevent spelling 
> mismatches.  Once a chain of plugins is established and before we run 
> it we could make sure that all of the inputs have been chained to.  We 
> can detect if a plugin asks for an input or output that has not been 
> annotated and warn the plugin programmer.
>
> Of course, once we have named input and output images we could build 
> on that and give the IJ end-user ways to link up plugins in workflows 
> or pipelines, a la CellProfiler.
>
> I have a prototype implementation of this, so the sample code above 
> works, details upon request.
>
> Aivar
>
> (P.S. It's the Thanksgiving holiday in the U.S. so you might not hear 
> from us until Monday.  Happy Thanksgiving!)
>
>
>
>
>
>
>
>
> _______________________________________________
> ImageJ-devel mailing list
> ImageJ-devel at imagejdev.org
> http://imagejdev.org/mailman/listinfo/imagej-devel