[ImageJ-devel] ImageJ2 Question

[Curtis Rueden]

Hi Jay,

I am CCing the imagej-devel list, where these discussions should best take
place for the public good.

> I'm trying to incorporate ImageJ plugins/functionality into JEX. I got
> an example working where I use an instance if ImageJ and the
> .command() service to discover, get, and execute a command type
> plugin. As I move forward I will be implementing widgets for each type
> of input and provide my own ui and put the info into the CommandInfo
> inputs, set the dataset and retrieve the output. I think this will
> suffice for most of the commands that take a Dataset. Right?

I think so. Though please note you will be the first guinea^H^H^H^H^Hperson
to be implementing a full-blown custom UI with its own widgets. I have
implemented four so far, though only one (Swing) is fully fleshed out; the
others are incomplete prototypes.

I am very happy to help resolve any issues you see in the imagej.ui design
(UIService, UserInterface plugin type, etc.). Personally I am not satisfied
with the design yet, but for the time being there are more pressing
development priorities.

> However, I am not sure how to build an ImageDisplay object (e.g., from
> a tif file on disk) for use with plugins that require an
> ImageDisplay object.

At the moment there is some unfortunate casting.

To wrap a Dataset in an ImageDisplay:
ImageDisplay display = (ImageDisplay) ij.display().createDisplay(dataset);

To wrap a Dataset in a DatasetView:
DatasetView view = (DatasetView) ij.imageDisplay().createDataView(d);

To wrap a DatasetView in an ImageDisplay:
ImageDisplay display = (ImageDisplay) ij.display().createDisplay(view);

But that should get you started, at least.

> Is there a fundamental difference between plugin
> commands that prefer to work with ImageDisplay objects (i.e. they only
> alter how things are displayed? and not the underlying data? or maybe
> commands that take ImageDisplay objects might need a roi object while
> commands that use Datasets don't?)

Yes:

- A Dataset contains image pixels and metadata, but is stateless. E.g., it
does not know the current plane being visualized.

- A DatasetView wraps a Dataset and knows its current position (Localizable
and Positionable in ImgLib2 terms). The idea is that the same data can be
visualized simultaneously in different ways; e.g., a 3-view of
cross-sections over different dimensions of a multi-dimensional dataset.
ImgLib2 makes this sort of thing really feasible without copying any data
around.

- An ImageDisplay is a collection of DataViews (DatasetViews and/or
OverlayViews). So if you want to show an image with some ROIs on it, or
multiple images as tiles, the ImageDisplay can bind all that together into
one thing.

> For example, why does the "Rotate..." command take a Dataset while the
> "Rotate Left" and "Rotate Right" commands take an ImageDisplay? If the
> command takes and ImageDisplay, can the image operation still be
> accomplished without displaying the image?

We try to use the "minimal object type" for each operation; i.e., Dataset
when possible. If ImageDisplay is required, it's almost always because
there is some feature of the ImageDisplay API that is needed by the
command. If DatasetView is required, it's probably because the command
wants to know the current position of the visualization, or wants to
manipulate the current canvas zoom or color scheme (all features of the
view, not the data itself).

If you look at the source of Rotate90DegreesLeft, you'll see that it asks
the ImageDisplay for its current selection bounds. This is so that the
rotation operation can process pixels inside only the current selection,
such as a rectangle ROI.

> Also, will plugins like TrackMate (is that the name? Also not sure if
> this one is ImageJ2 ready... is it?) that may encompass a much larger
> framework, be implemented as command plugins or will they be more
> general modules.

TrackMate is not yet an ImageJ2 command, but it will be. Right now it still
targets ImageJ1. More generally, TrackMate is an API for programmatic
tracking, with its own SciJava plugin types for segmentation and "linking"
(i.e., joining segmented objects together into a graph).

> If they are modules, can I still generally just ask
> for the inputs, display the widgets to the user, pass the info, set
> the Dataset, run them, and get outputs like for the command plugins
> but just using the module service. Is it more hopeless than that?

An ImageJ2 command is merely a type of ImageJ2 module. The code you have
written for discovering the inputs and outputs of available modules applies
equal to commands and to other modules.

The only thing special about a command is that it is Java code which
implements the Command interface and supports declaring the inputs and
outputs via the @Parameter annotation. From the standpoint of an API
consumer, you don't care at all about that, and can just interrogate the
ModuleInfo for what you need, regardless of whether the module in question
is a Command.

> What is the general difference in paradigm I need to understand (if
> any) in order to also incorporate these much larger/more involved
> plugins?

By "more involved" I assume you mean "more than just a single module."
Because in terms of lines of code, it makes no difference. But often what
will happen is that an extension such as TrackMate will end up consisting
of several modules, possibly define other plugin types (such as "segmentor"
and "linker"), and plugin implementations (e.g., "simple threshold" as a
segementor, or "overlapping pixels" as a linker).

But really, who cares? If TrackMate provides 18 modules, and your code
discovers and exposes all available modules, you will provide 18
TrackMate-related modules to your users. No worries there, right?

> I always at least provide default values for each input of a plugin,
> will I require any other preprocessing?

Without getting very nitty-gritty, I couldn't say. Try it.

> Can I get an example of how preprocessing and post processing are used
> or their typical purpose for the types of plugins that I'm looking to
> bring into JEX (image processing and quantification as opposed to
> results table editing / preferences / image display settings / etc
> plugins).

Here is an example showing how to implement your own preprocessing plugin:


https://github.com/imagej/imagej-tutorials/tree/master/custom-preprocessor-plugin

Postprocessing plugins are analogous.

That said, I am not sure whether you will need to implement any custom pre-
or postprocessing steps. Perhaps.

> I couldn't find and "all-inclusive" imagej.jar file on the website for
> download so that I could just import a single jar into my project.

Sorry, it is a little buried:

  http://developer.imagej.net/faq#t22n213

But that's rather intentional; using uber-JARs is fraught with peril, and
it's better if you don't. Structure your project as a Maven project and let
it manage your dependencies, and the need for an uber-JAR largely
disappears.

Regards,
Curtis


On Thu, Jan 9, 2014 at 1:26 PM, Jay Warrick <warrick at wisc.edu> wrote:

> Hi Johannes,
>
> Let the annoying questions from me begin ;-)... I'm trying to incorporate
> ImageJ plugins/functionality into JEX. I got an example working where I use
> an instance if ImageJ and the .command() service to discover, get, and
> execute a command type plugin. As I move forward I will be implementing
> widgets for each type of input and provide my own ui and put the info into
> the CommandInfo inputs, set the dataset and retrieve the output. I think
> this will suffice for most of the commands that take a Dataset. Right?
>
> However, I am not sure how to build an ImageDisplay object (e.g., from a
> tif file on disk) for use with plugins that require an ImageDisplay object.
> Is there a fundamental difference between plugin commands that prefer to
> work with ImageDisplay objects (i.e. they only alter how things are
> displayed? and not the underlying data? or maybe commands that take
> ImageDisplay objects might need a roi object while commands that use
> Datasets don't?) Just trying to wrap my mind around the thinking so I can
> craft a general way to work with these particular sets of commands. For
> example, why does the "Rotate..." command take a Dataset while the "Rotate
> Left" and "Rotate Right" commands take an ImageDisplay? If the command
> takes and ImageDisplay, can the image operation still be accomplished
> without displaying the image?
>
> Also, will plugins like TrackMate (is that the name? Also not sure if this
> one is ImageJ2 ready... is it?) that may encompass a much larger framework,
> be implemented as command plugins or will they be more general modules. If
> they are modules, can I still generally just ask for the inputs, display
> the widgets to the user, pass the info, set the Dataset, run them, and get
> outputs like for the command plugins but just using the module service. Is
> it more hopeless than that? What is the general difference in paradigm I
> need to understand (if any) in order to also incorporate these much
> larger/more involved plugins? If I always at least provide default values
> for each input of a plugin, will I require any other preprocessing? Can I
> get an example of how preprocessing and post processing are used or their
> typical purpose for the types of plugins that I'm looking to bring into JEX
> (image processing and quantification as opposed to results table editing /
> preferences / image display settings / etc plugins).
>
> Thanks!!!!
>
> Jay



On Thu, Jan 9, 2014 at 1:44 PM, Jay Warrick <warrick at wisc.edu> wrote:

> Sorry for another one :-)
>
> I couldn't find and "all-inclusive" imagej.jar file on the website for
> download so that I could just import a single jar into my project. I'm fine
> with importing each separately if need be but figured you guys probably
> have one somewhere. In one of your pages you refer to something of the sort
> but I didn't see it on the linked download page currently.
>
> Thanks,
>
> Jay
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://imagej.net/pipermail/imagej-devel/attachments/20140113/a3c8952f/attachment.html>