[ImageJ-devel] ImageJ - KNIME Integration: open questions

Fri Jun 28 15:04:51 CDT 2013

Hi Michael & Lee,

Sorry for the delay in reply; it's a long thread. :-/

Michael wrote:
> Would it be possible to add a flag that surpresses previews checkboxes
> during dialog building?

It's tricky because right now, each preview checkbox is an input parameter
to the command itself. But personally I don't like that approach. It is not
a "real" input to the command, but rather intended to be toggled during
input harvesting only. One option might be to label the visibility of those
parameters as ItemVisibility.INVISIBLE. Then we could add a flag somewhere
to hide INVISIBLE parameters, maybe.

But really, I am leaning toward a simpler solution: just remove the preview
checkboxes, in favor of ImageJ2 *always* doing a preview (and of course,
KNIP doesn't have to). And if the previews are too slow, we can optimize
the preview() method to address it, rather than having this hacky checkbox
like ImageJ1 does.

Lee wrote:
> I hadn't looked at the interactive commands with regard to
> CellProfiler. It looks like "buttons" could be interpreted as possible
> headless actions that could be performed... at least in some cases.

Yes, each button generally performs some action, which we should really
make sure are always exposed as standalone commands (see next paragraph).
We are going to add logic to make multiple sequential Button widgets render
as a single button bar in our UI layout. Won't affect you though, Lee,
since you do your own UI in Python.

Michael wrote:
> Why is the threshold  plugin an interactive command?

Let me explain the motivation here. Anything that extends
InteractiveCommand is not *intended* to be run in headless mode. Rather, it
is intended as a sort of "meta-command" that allows you to launch other
commands. It will be filled with callback methods when various inputs
change, which should, in a properly designed InteractiveCommand, trigger
real command executions that are more well-behaved from a headless
standpoint.

Anything that extends InteractiveCommand should *not* be flagged with
"headless = true" since it is not intended for that purpose.

So in the case of threshold, we need a "real" threshold command that has
sensible inputs and outputs. Then the command exposed in the ImageJ UI as
"threshold" is actually "Interactive Threshold" which fires off Threshold
executions.

This is not all fully in place, but it is the direction we are going. Sound
reasonable?

Lee wrote:
> For Threshold, I can see reasons for executing the callbacks for
> "auto", "apply" and "delete" in a headless context

The above would also address your comments here, Lee. The interactive
threshold command would not need to be exposed within CP because those
buttons would each have a corresponding standalone headless-friendly
command which *would* be exposed.

Michael wrote:
> is there a way to determine if a command is dynamic?

Other than the hacky instanceof test you are doing, nope. I agree we should
add API for it to ModuleInfo. Especially because it is possible for modules
to be dynamic without extending the DynamicCommand class...

Could you please file a ticket for it on the ImageJ Trac? I just made
ImageJ Trac accounts for you, Christian & Martin; you should have an
automated email about it. I know Trac is not ideal (GitHub Issues is nice)
but we have all our ImageJ tickets in Trac now, migrating everything would
be very involved, and I don't really want to start splitting between two
issue trackers. Someday we might move...

> Are you planing to re-organize the plugins, e.g. such that plugins,
> which are interesting for any application reside in their own
> jar-file?

This is probably a good idea. However, I agree with Lee that in some cases
it may not be totally clear-cut [1]. You think splitting ij-commands into
two JAR files is sufficient? If so, care to send a list of plugins you feel
are *not* generally useful, and I can split them out?

> Will the package path of the plugins be stable after ImageJ2 1.0.0 was
> released?

Yes, when ImageJ 2.0.0 final is released. From then on, we will always
deprecate any necessary moves, to keep old code working. And of course will
also try not to do it at all unless deemed really important.

That said, I don't really like the current package prefix of
"imagej.core.*" for various "core plugins." It is confusing that we have
"ij-core" which is the core *library* and that code does *not* have prefix
"imagej.core", whereas we have "core plugins" which *do* have package
prefix "imagej.core". I would welcome a better idea!

> Are there  plans how the "dimension selection" will be integrated into
> ImageJ2, i.e. how algorithms can be run on arbitrary dimensions?

Yes. The img-metadata branch [2] is relevant here as far as axis names go.

I was thinking for starters, we could simply create new (headless-friendly)
commands that narrow the view of a given dataset using the ImgLib2 Views
class. Then from an end user perspective, if you want to limit the
execution of the algorithm, you just run that command first. This fits in
well with KNIME's node concept, where the answer to any given problem is
usually to just insert another node for preprocessing or whatever.

Do you think more than that is needed?

Lee wrote:
> There isn't any mechanism to handle AxesNames algebra ... annotation
> improvements here would be nice.

I agree, but do not have time to work on it. Proposals and PRs very
welcome. It would probably make the most sense to base any work on the Axis
metadata design from the img-metadata branch of ImgLib.

Michael wrote:
> ROIs: Are there plans to support Labelings in ImageJ2?

Lee wrote:
> We're also looking forward to labelings being first-class entities

This is something we should pursue further for beta9, which is intended to
focus on ROIs.

Michael wrote:
> Is "new ImageJ(services...)" replaced by  "new Context(...services)"?

Yes. Strictly speaking, that is correct.

However, there is a new ImageJ "gateway" class in ij-app so that you can
still say "new ImageJ(...)". And actually from that object you will get
lots more functionality: you can query any built-in service with a
compile-safe method. See the ImageJ tutorials for details [3].

Michael wrote:
> How can we query the ImageJ version?

I apologize that it is somewhat convoluted now. First, the code:

    ImageJ ij = new ImageJ();
    App ijApp = ij.app().getApp(ImageJApp.NAME);
    String ijApp.getVersion();

This uses the AppService feature of SciJava Common. The rationale is that
you may have a single SciJava Context with multiple associated applications
such as ImageJ, SCIFIO and SciJava Common itself. And obviously they don't
all have the same version. So you first have to ask the AppService for your
application of interest, then query its version.

We actually spit out this sort of stuff for *all* available applications in
the SystemInformation command:
https://github.com/imagej/imagej/blob/imagej-2.0.0-beta-7/plugins/commands/src/main/java/imagej/core/commands/debug/SystemInformation.java#L84

There are all sorts of goodies you can query, many of which are extracted
from either associated Maven POMs or JAR manifests. :-)

Michael wrote:
> the log level system property "System.setProperty("ij.log.level",
> "error");" doesn't seem to  work any more

With the migration to SciJava Common, the property changed to
"scijava.log.level" but it works the same way.

Michael wrote:
> we had to move ModuleService behind  MenuService in the constructor
> call to avoid a null pointer during initialization
> (onEvent(ModulesAddedEvent) was called before initialize).

That is a bug, thanks for reporting. However, moving the module service
after the menu service will not work because MenuService needs
CommandService which needs ModuleService.

What's weird to me is, I am not sure why we don't see this bug when running
the end-user ImageJ application...

I have some service initialization changes pending on ImageJ's
startup-speed branch, which may affect this. So we should debug further
once that branch is merged. Please feel free to file a Trac ticket for it
and assign to me.

Michael wrote:
> Is there a defined service order?

The order is defined by two things:
1) If a service has an @Parameter dependency on another service, the
dependency gets initialized first.
2) Otherwise, the "priority" property of the @Plugin annotation is used to
order service initialization.

> ((So far, we have tons of ideas where we could do more things
> together. Thank you in advance for answering all these questions ;) ))

No problem, and again, sorry if the long delay reduces your momentum and/or
enthusiasm. If you have any more questions, please fire away!

Regards,
Curtis

[1] You know what that easter egg does, right? Don't you *want* to generate
ASCII images using KNIME??? I know I do!
[2] https://github.com/imagej/imglib/compare/master...img-metadata
[3] E.g.:
https://github.com/imagej/imagej-tutorials/blob/885892b6b225fba2e1b32c4ca14ab330e39f384a/load-and-display-dataset/src/main/java/LoadAndDisplayDataset.java

On Wed, May 15, 2013 at 9:35 AM, Lee Kamentsky <leek at broadinstitute.org>wrote:

> On Wed, May 15, 2013 at 9:08 AM, Michael Zinsmaier <
> michael.zinsmaier at gmx.de> wrote:
>
>>  Hi all,
>>
>>  after you guys have made so much progress during the hackathon, we felt
>> motivated to review our KNIME-ImageJ2 integration and try to further
>> improve it, as we want to have everything set-up when you release ImageJ2
>> ;-)
>>
>> Anyway, we discussed several things and a lot of questions came up
>> concerning ImageJ2:
>>
>> Thx guys, lots of stuff below also applies to CellProfiler running
> ImageJ2.0 plugins.
>
>>
>
>>  * Why is the threshold  plugin an interactive command? This for
>> instance prevents the plugin from  beeing executing in headless mode (and
>> hence from appearing as a KNIME node), which might be desirable in some
>> use  cases.
>>
>
> I hadn't looked at the interactive commands with regard to CellProfiler.
> It looks like "buttons" could be interpreted as possible headless actions
> that could be performed... at least in some cases. I have CellProfiler
> analogs for most of the parameter types, but that one is new. I'm thinking
> that I might detect whether a plugin is interactive and, if so, display
> each button as a radio button, so that the button's callback could be
> executed in the context of a CellProfiler headless pipeline execution. For
> Threshold, I can see reasons for executing the callbacks for "auto",
> "apply" and "delete" in a headless context - unfortunately, CellProfiler
> would convert the ThresholdOverlay to a binary mask, so "delete" doesn't
> make much sense for us.
>
>>
>>
>> * In KNIME we are currently loading all available plugins in the
>> classpath which are headless executable. But therewith plugins like "Help",
>> "Easteregg", "LoadDataSet" will appear as KNIME nodes as well. However they
>> can't do anything useful in the KNIME context (as they are  ImageJ2
>> specific). Are you planing to re-organize the plugins, e.g. such that
>> plugins, which are interesting for any application reside in their own
>> jar-file?
>>
>> We're using the plugin menu system to display the available plugins
> hierarchically in CellProfiler. I guess that some plugins are less useful
> or arguably useless but who am I to judge (I'm still waiting for someone to
> publish a paper citing use of CellProfiler's "game of life" morphology
> operation). Hopefully, the hierarchy leads the users to the most useful
> plugins. I could see some other sort of annotation, e.g. "tags", but I
> don't want to be the one who manages the tag ongology ;-).
>
>
>>
>  * Are there  plans how the "dimension selection" will be integrated into
>> ImageJ2,  i.e. how algorithms can be run on arbitrary dimensions? (Mapping
>> from  AxesNames to indicies of dimensions in images). Can we help here? (see
>> Doc hackathon!?)
>>
>> CellProfiler does use the AxesNames to transform the N-dimensional arrays
> to our representation. I think that there's enough functionality in the
> restructuring commands to let power users pull out individual hyperplanes
> for processing in lower dimensions. Perhaps a "ExtractData" and
> "ReplaceData" plugin need to be added in order to create and reinsert
> lower-dimension datasets?
>
> There isn't any mechanism to handle AxesNames algebra - you don't know
> whether a plugin will reduce or augment a dataset's dimensionality or
> whether a plugin is only suitable for two dimensions. CellProfiler and, I'm
> guessing, Knime do a lot of matching inputs to outputs which is a contrast
> to ImageJ. I think our users can handle this ambiguity (they'd use only 2-D
> or N-D plugins adaptable to 2-D), but annotation improvements here would be
> nice.
>
>  * ROIs: Are there plans to support Labelings in ImageJ2? Or will
>> Labelings somehow be replaced in the future?
>>
>
> We're also looking forward to labelings being first-class entities in
> ImageJ, partially my fault personally that this is not better developed.
> I'm mostly interested in us having some agreement for the datatype for a
> labeling parameter - I think that once we have that sort of
> interoperability, we'll see lots of progress in both segmentation methods
> and use of segmentations in downstream analysis.
>
>>
>>
>> * In the current snapshot (compared to beta6) some functionality was
>> moved from  ImageJ to sifio / scijava (e.g. the Services).  Regarding
>> this some questions:
>>   - The services seem to have some  ordering requirements now, we had to
>> move
>>     ModuleService behind  MenuService in the constructor call to avoid a
>> null pointer during
>>     initialization (onEvent(ModulesAddedEvent) was called before
>> initialize). Is there a defined
>>     service order?
>>
> It would be nice to have a headless service profile, something a little
> easier than leaving out the AWT dependent jars and crossing fingers.
> CellProfiler might instantiate both GUI and headless contexts in the same
> JVM - we'd appreciate being able to choose the UI configuration in a simple
> way.
>
>>
>>
>
>> ((So far, we have tons of ideas where we could do more things together.
>> Thank you in advance for answering all these questions ;) ))
>>
>
> Likewise, thank you, thank you - don't misconstrue any of the above as
> demands. We all appreciate how all of this is being done and the time it
> all takes.
>
>>
>>  Cheers,
>>
>>  Martin, Christian and Michael Z.
>>
>> _______________________________________________
>> ImageJ-devel mailing list
>> ImageJ-devel at imagej.net
>> http://imagej.net/mailman/listinfo/imagej-devel
>>
>>
>
> _______________________________________________
> ImageJ-devel mailing list
> ImageJ-devel at imagej.net
> http://imagej.net/mailman/listinfo/imagej-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://imagej.net/pipermail/imagej-devel/attachments/20130628/87ede65c/attachment-0001.html>