[ImageJ-devel] Plugin Philosophy?

Fri Jan 31 16:50:41 CST 2014

Hi Jay,

> I have the bridge between JEX and IJ2 pretty well roughed together.

That is great!

> When the plugins have a service as a parameter, it often leaves vague
> what that service will specifically provide for the function. For
> example, the "Fill" plugin (i.e., the FillDataValues class) takes an
> OptionsService as a parameter. But, when using the CommandService and
> discovering this input, it is not obvious at all that the plugin is
> really looking for a color with which to fill the specified region.

That is an excellent point. Really, any object type can be annotated as an
@Parameter input. In the example you gave, the Fill command should really
annotate a ColorRGB as the input. And then to fill with the foreground
color we should have a separate "FillWithForeground" command that takes the
correct Options plugin as input (not just the OptionsService) and extracts
the foreground from that and invokes the Fill command via its API.

This sort of refactoring is straightforward but requires someone to observe
specifically where the API is weak and file issues so that we can
prioritize and track that effort.

> This concept extrapolates somewhat as well to the use of an
> ImageDisplay as an input, it appears that often an ImageDisplay object
> is used to help specify both a Dataset and an image plane index.

Absolutely, totally agreed. Again, making the commands more fine-grained
and modular is certainly a goal.

> Further, if I don't have an ImageJ2 ImageDisplay viewed, I can't
> change the particular plane that is being operated on without creating
> a one-off wrapper for this plugin to specify that information.
> Likewise, if I create a widget to display every time an ImageDisplay
> is asked for to ask for an image plane index, there are myriad other
> properties that I would also need to provide widgets for if I want to
> give the user flexibility for all the types of things that can be
> learned from an ImageDisplay.

Right, I don't think that's the way to go at all. The commands shipped with
ImageJ2 need to be more low-level and fine-grained, with more
"default-centric" commands that delegate to them using the current state of
the ImageJ system (e.g., the currently active ImageDisplay).

> The purpose of the "DataTransformPlugins" would be to do the actual
> editing / grunt work while the "MenuPlugins" use the services and IJ2
> ui elements to provide the "DataTransformPlugins" (and other plugins)
> good guesses or standard choices for inputs to the algorithms and
> could keep the behavior of IJ2 in sync with IJ1 as needed (e.g., the
> Edit -> Fill menu option would fill with the IJ2 foreground color).

Yep, that's essentially what I'm proposing above. We don't necessarily need
separate interfaces, although there could be one or more marker interfaces
for certain things (would need to cogitate further).

> The DataTransformPlugins would then be much more straight forward for
> groups such as myself, CellProfiler, Icy, and Knime to more
> transparently present the algorithms of IJ2 to users in other ui's
> (i.e., displaying widgets and borrowing or implementing services for
> providing good initial guesses through our own "menus").

Yes, although that is what the "headless" flag is for. Any module that
declares itself usable headless should be able to be exposed in an external
framework such as any of those you mentioned above. Of course, you can also
filter based on the types of inputs the module requires. If it needs a
FooBarFizzBob and you don't know what that is, then just exclude the module
from your list.

> In JEX, an object or data manager panel helps to keep the user aware
> of all the objects they have at their disposal and provides a means by
> which to select items for inputs to functions other than just active
> windows etc.

That is exactly the purpose of the SciJava ObjectService. It forms one of
the central pillars of ImageJ2. All Datasets, DatasetViews, ImageDisplays,
and many other kinds of objects floating around all get registered with the
ObjectService (specifically, with its ObjectIndex). In fact, that is how
ImageJ2's ObjectWidget (the final fall-back widget when no more appropriate
widget can be found) knows what to populate into its drop-down list of
choices. So I think we are most of the way there.

> in the ROIManager, when you draw an ROI on an image you have to
> specifically add it to the ROIManager instead of it showing
> automatically.

That is true in ImageJ1, but not ImageJ2. Try running ImageJ2's Overlay
Manager and you'll see that *all* overlays you draw on every dataset are
shown there.

> This would provide a super easy way to keep track of items and supply
> them to functions via selection and multi-selection (it could even be
> its own widget... in fact this is much like the drop down in Knime
> showing all the table columns on which to operate).

Sure. I don't really have a good vision of how that UI would function, but
I think it would be doable in the current ImageJ2 paradigm.

> I feel like the context dependent nature of ImageJ is one reason why
> the first version got so disorganized under the hood and why many of
> the plugin implementations were so buggy or not obvious in how to be
> used. I would love to see this reduced and maybe such a manager would
> be a step in the right direction.

Well, I think (though I'm certainly biased ;-) that ImageJ2 does a pretty
good job in tracking the objects that exist thanks to the ObjectService.
And it is designed to keep each application context separately
encapsulated, in case multiple such contexts become necessary in the same
JVM for any reason.

Thanks very much for the detailed comments! I think your ideas are great,
and I hope we can find time to refactor commands in those directions.

We are currently using Trac (http://trac.imagej.net/) to track the ImageJ2
issues, but I increasingly want to migrate everything to GitHub Issues
(would be at https://github.com/imagej/imagej/issues). If I turn on the
ImageJ GitHub Issues tracker, would you be comfortable filing some of these
specific examples of commands being "not typed enough" as issues in there?

Regards,
Curtis

On Fri, Jan 17, 2014 at 11:55 AM, Jay Warrick <jay.w.warrick at gmail.com>wrote:

> Hi Guys,
>
> I have the bridge between JEX and IJ2 pretty well roughed together. Now
> exploring for nuances and building up bridge in the area of ROIs. I had
> some thoughts I wanted to share given my recent experience as a Guinea pig
> in case they might help the ImageJ2 project as a whole. Sorry for the
> length and feel free to copy whomever you wish. Let me know if any of these
> avenues has potential that you would like to discuss further (I have some
> more thoughts on ROIs :-). I know you all have a billion things that are of
> higher priority so my hopes aren't high for seeing any of these things come
> to be but I really do just hope to increase the impact of this software.
> Any help I can provide as far as implementing related changes is also
> available to you.
>
> When the plugins have a service as a parameter, it often leaves vague what
> that service will specifically provide for the function. For example, the
> "Fill" plugin (i.e., the FillDataValues class) takes an OptionsService as a
> parameter. But, when using the CommandService and discovering this input,
> it is not obvious at all that the plugin is really looking for a color with
> which to fill the specified region. Likewise, but maybe to a lesser degree,
> there are many potential properties/values/objects that could be supplied
> by each service used as input parameters. In the case of the "Fill" plugin,
> the user must change the foreground option of IJ2, which is also a plugin
> (but not a command plugin), in order to change the fill color. Instead,
> wouldn't it seem more appropriate to have a color input (e.g., a color
> chooser widget) that is defaulted to the IJ2 foreground color. If I were to
> implement my own OptionsService for using as an input to the function, I
> would not know the small piece of information that my OptionsService would
> actually need to provide to run successfully.
>
> This concept extrapolates somewhat as well to the use of an ImageDisplay
> as an input, it appears that often an ImageDisplay object is used to help
> specify both a Dataset and an image plane index. Couldn't the plugin take a
> Dataset and a plane choice (i.e., 0,1,2,3,All) as inputs? The service could
> provide a good guess (i.e., the current plane) but leaving the chosen plane
> as an option for the user. This way, the required information for the
> plugin to run is more transparent while still using the ui information.
> Further, if I don't have an ImageJ2 ImageDisplay viewed, I can't change the
> particular plane that is being operated on without creating a one-off
> wrapper for this plugin to specify that information. Likewise, if I create
> a widget to display every time an ImageDisplay is asked for to ask for an
> image plane index, there are myriad other properties that I would also need
> to provide widgets for if I want to give the user flexibility for all the
> types of things that can be learned from an ImageDisplay. In general, most
> of this information is ignored, which would again lead to an ambiguous
> interface for plugins that need an ImageDisplay.
>
> I realize these examples don't cover the extent of cases that exist and,
> in many cases, the plugins are meant to act on the ui itself and don't
> really transform any data (e.g. contrast)... but what if the plugins shown
> in the menu are (I'm trying to make up a new class so any overlap with
> actual class names is inadvertent) "MenuPlugins" and the raw data
> transforming operations or grunt work are implemented as
> "DataTransformPlugins". The purpose of the "DataTransformPlugins" would be
> to do the actual editing / grunt work while the "MenuPlugins" use the
> services and IJ2 ui elements to provide the "DataTransformPlugins" (and
> other plugins) good guesses or standard choices for inputs to the
> algorithms and could keep the behavior of IJ2 in sync with IJ1 as needed
> (e.g., the Edit -> Fill menu option would fill with the IJ2 foreground
> color). The DataTransformPlugins would then be much more straight forward
> for groups such as myself, CellProfiler, Icy, and Knime to more
> transparently present the algorithms of IJ2 to users in other ui's (i.e.,
> displaying widgets and borrowing or implementing services for providing
> good initial guesses through our own "menus"). This could be done in a way
> that people (e.g. Knime) that are using the current plugins, which I think
> of more as "MenuPlugins", wouldn't be affected by the addition of the
> intermediate implementation layer. In the case of the "Fill" plugin,
> current use cases may even be enhanced by providing the same service
> defaults as before but it might now show the color chooser widget,
> providing more flexibility. I'd be happy to help if this seems feasible or
> desired (which it may very well not be :-) as it is a small addition of a
> middle man to what is already implemented for many things.
>
> This also brings up a personal desire I have for ImageJ2, which also means
> next to nothing, but I have found it useful for JEX and thought I would
> bring it up in case you wished to discuss it more. In JEX, an object or
> data manager panel helps to keep the user aware of all the objects they
> have at their disposal and provides a means by which to select items for
> inputs to functions other than just active windows etc...  This should
> include ROIs and Tables which I feel are currently second class citizens in
> imagej. The result is that *too much of the current context needs to
> discovered for functions to run rather than via outright specification*(e.g., creating color images from multiple open grayscale images, which
> will be which R, G, and B? I've seen one implementation that uses the order
> of the image windows to decide, good luck guessing right or remembering).
> The ROIManager is something of a beginning for ROIs. However in the
> ROIManager, when you draw an ROI on an image you have to specifically add
> it to the ROIManager instead of it showing automatically. Thus, although it
> is not part of the ROIManager it is still a potential input to a function.
> Why not show all objects somewhere (e.g., a simple list organized by type).
> This would provide a super easy way to keep track of items and supply them
> to functions via selection and multi-selection (it could even be its own
> widget... in fact this is much like the drop down in Knime showing all the
> table columns on which to operate). I feel like the context dependent
> nature of ImageJ is one reason why the first version got so disorganized
> under the hood and why many of the plugin implementations were so buggy or
> not obvious in how to be used. I would love to see this reduced and maybe
> such a manager would be a step in the right direction.
>
> As always, thanks for taking the time to read and reply. I really
> appreciate the work of the ImageJ2 team and these conversations. Look
> forward to hearing from you.
>
> Regards,
>
> Jay
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://imagej.net/pipermail/imagej-devel/attachments/20140131/4ca2c0c4/attachment-0001.html>