[ImageJ-devel] IJ2 Plugin Proposal

Wed Nov 24 16:45:43 CST 2010

Hi Lee,

What I want to do differently is pass on the output image from a plugin 
whenever its ready.  You won't get this externally by noting that a 
member variable is annotated, the plugin has to make a call to say this 
output image is done.  In this way the plugin could produce a series of 
images one by one (rather than having to pass back an array of output 
images).

I looked at KNIME, Pipeline Pilot, and MyGrid Taverna, which were all 
mentioned in a December 2009 ImageJX discussion.  They seem overblown to 
me for what we need.  There's also licensing issues: GPL, odd 
proprietary eval/yearly lease, LPGL respectively.  Some of them involve 
authoring XML files, etc.  The annotation scheme is very lightweight.

To me the GUI drag & drop UI is mostly eye candy.  I like how 
CellProfiler sets up  workflows.  You should be able to author workflows 
by example:  you load an image, run a plugin, run another plugin, and 
that becomes a workflow.  We save the plugins used, there are inputs, 
chained output/inputs, and outputs.

Aivar

On 11/24/10 3:31 PM, Lee Kamentsky wrote:
> Hi Aivar,
> I'm sort of doing this with the @Parameter attributes now and 
> CellProfiler. I use reflection to get the class of the tagged 
> variable. The plugin's image inputs are variables of type ImagePlus 
> (and probably later, imglib's Image) that are not marked with 
> output=true and the outputs are the ones with output=true. Perhaps 
> it's a bit of a hack to infer that something is intended to be used as 
> an image by its class type, but that's what I did; the other 
> possibility would be to indicate that it is an image explicitly in 
> @Parameter.
>
> CellProfiler works beautifully with this - maybe Curtis can show you 
> how it does with ImageJ plugins.
>
> I can imagine writing your tool (or using something like Knime) with 
> the current infrastructure. You have some sort of hookup gui (and 
> Knime does this for you) that allows you to attach image inputs to 
> outputs and it's up to the tool to figure out when a plugin's inputs 
> are ready.
>
> --Lee
>
> On 11/24/2010 4:16 PM, Aivar Grislis wrote:
>> I'm proposing a more dynamic way of running plugins and sending 
>> images from one plugin to another that would lend itself to workflows 
>> and pipelines.  This could also be used within plugins and within IJ2 
>> as an architectural building block to chain chunks of image 
>> processing code together.
>>
>> Basically I propose that a plugin can have multiple named input and 
>> output images and that whenever an output image is ready it is passed 
>> on to become the input image to the next plugin in the chain.
>>
>> IMPETUS
>>
>> I feel the need for this from a couple of plugins I have worked on 
>> (actually my only plugins!).
>>
>> I worked on a Deep Zoom plugin.  This takes a huge image, tiles it, 
>> then the tiles are written out in a particular directory/file name 
>> structure.  Next we halve the original image and repeat, until the 
>> image gets halved to a miniscule size.
>>
>> While working on this plugin I thought about writing the code to 
>> halve the image and to tile it in the most general-purpose, reusable 
>> way.  Also I wanted to be memory-efficient: I didn't want to just 
>> make all the halves, then go through them and make all the tiles, 
>> then go throughout those and write out all the files.
>>
>> What I came up with at that time was a chainable plugin.  In 
>> pseudo-code:
>>
>> interface IChainablePlugin {
>>
>>   // chains one plugin to next; called from outside the plugin
>>   public chain(IChainablePlugin plugin);
>>
>>   // gets the input image; called from within plugin
>>   Image get();
>>
>>   // puts the output image; called from within plugin
>>   put(Image image);
>> }
>>
>> An abstract base class kept track of the next, chained plugin 
>> instance and handed off the output image to that next plugin, using 
>> the same thread.
>>
>> I wrote a chainable plugin class to halve the image, one to tile it, 
>> and had a third one that was just an inner class to my Deep Zoom 
>> plugin that handled the specifics of writing out the tiled image 
>> files.  Presumably the first two classes would be reusable code and 
>> the third was just specific to Deep Zoom.
>>
>> Obviously as-is this is limited to one input and one output.  I 
>> started thinking about having an image dispatcher class that would 
>> keep track of which images a plugin required and hand them off from 
>> one plugin to another.
>>
>> I'll just briefly mention the SLIM plugin I am working on.  Since the 
>> SLIM fitting process is time consuming I display a very coarse 
>> colorized version of the fitted results that is refined as the fit 
>> progresses.  So there I want to pass on my output image before the 
>> plugin finishes and I want to put out successive versions of this 
>> image.  This is similar to my wanting to process the individual tiles 
>> before the whole image is tiled.
>>
>> PROPOSAL
>>
>> We could label the input and output images using Java annotations.  
>> (Note that the current Declarative Plugin annotations allow one to 
>> label input and output image member variables but the outputs are 
>> harvested only once the plugin is finished. There is no way to put 
>> out an output image within the plugin code.)
>>
>> Here is some sample code for what I'm proposing:
>>
>> "@Input
>> @Output({ @Img(Plugin1.BLUE), @Img(Plugin1.GREEN) })
>> public class Plugin1 extends AbstractPlugin {
>>   public static final String BLUE = "Blue";
>>   public static final String GREEN = "Green";
>>
>>   void process() {
>>       Image image = get();
>>       put(BLUE, image);
>>       put(GREEN, image);
>>   }
>> }"
>>
>> Here the annotations result in an input name set of ["Default"] and 
>> an output set of ["Blue","Green"].  (If there are no parameters the 
>> default name is used.)  Of course, there could also be named input 
>> images (i.e. "get(ORANGE)") or default output images (i.e. "put()").
>>
>> Instances of plugins such as these could be chained together:
>>   plugin1.chain(Plugin1.BLUE, plugin2);  // named output goes to 
>> default input
>>   plugin1.chain(Plugin1.GREEN, plugin2, Plugin2.ORANGE); // named 
>> output goes to named input
>>   plugin2.chain(plugin3); // default output goes to default input
>>   plugin2.chain(plugin4, Plugin4.PURPLE); // default output goes to 
>> named input
>>
>> Using strings as identifiers might seem less than ideal.  However, 
>> the strings are unique to the plugin they belong to.  If they are 
>> specified as static constants, as above, that would prevent spelling 
>> mismatches.  Once a chain of plugins is established and before we run 
>> it we could make sure that all of the inputs have been chained to.  
>> We can detect if a plugin asks for an input or output that has not 
>> been annotated and warn the plugin programmer.
>>
>> Of course, once we have named input and output images we could build 
>> on that and give the IJ end-user ways to link up plugins in workflows 
>> or pipelines, a la CellProfiler.
>>
>> I have a prototype implementation of this, so the sample code above 
>> works, details upon request.
>>
>> Aivar
>>
>> (P.S. It's the Thanksgiving holiday in the U.S. so you might not hear 
>> from us until Monday.  Happy Thanksgiving!)
>>
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> ImageJ-devel mailing list
>> ImageJ-devel at imagejdev.org
>> http://imagejdev.org/mailman/listinfo/imagej-devel
>
>
> _______________________________________________
> ImageJ-devel mailing list
> ImageJ-devel at imagejdev.org
> http://imagejdev.org/mailman/listinfo/imagej-devel