<div dir="ltr">Hi Stefan,<div><br></div><div><div><div>> Maybe I was unclear on this issue. May question is, what does a method</div><div>> of a Command (e.g. its initializer, the run method) or rather the</div><div>
> implementor may assumed to have "happend".</div></div></div><div><br></div><div>No worries, you were clear. The term "preconditions" is a technical software engineering term, so I know what you meant. I just dodged the question a bit there, figuring my detailed answers later would clarify everything. But I guess it still was not clear, sorry...</div>
<div><br></div><div>* The run() method of a module can assume that all required input parameters have been filled (i.e., are non-null). And the corresponding post-condition of run() is that all required output parameters are now filled.</div>
<div><br></div><div>* The initialize() method of a module cannot necessarily assume anything about the state of its parameters. As you pointed out, ImageJ2 has various commands right now that *do* assume certain parameters have been filled via preprocessing. This works in practice (at least from the ImageJ application), but is not particularly rigorous.</div>
<div><br></div><div>* Similarly, ImageJ2 has several commands which assume their initializer has been called, but it is not stated as a formal precondition. Most probably it makes sense to state that in the javadoc, would you agree? In other words: if you aren't going to include the InitPreprocessor in the preprocessing chain, you are still somehow responsible for calling module.initialize() before calling module.run(). Is that fair?</div>
<div><br></div><div>So the main sticky point is what to do about initialize() requiring certain parameters to be already filled. This is useful for dynamic modules, but also just for computing defaults -- e.g., setting default brightness/contrast min & max values to the current display min max of the input image. So it seems there are two "layers" of input parameters: those needed before initialize() and those not needed for it.</div>
<div><br>Maybe we need a way to formally flag these "required before initialize" parameters in the annotation? What do you think? Would that help you?</div><div><br></div><div>In any case, now is the perfect time to get the module framework fully solidified, since we are on the cusp of migrating it into the (non-image-specific) SciJava layer. After the June 1 release, we will not be able to introduce any more breaking changes.</div>
<div><br></div><div><div><div>> Besides interesting for a programmer of a Command we are interested in</div><div>> this issue to be able to wrap IJ2 commands in order to run them from</div><div>> our GUI (originaly design for alida operators), our command line</div>
<div>> runner, and to support them in our graphical editor for alida/mitobo</div><div>> operators and hopefully ij2 commands (at least head less and non</div><div>> DynamicCommands)</div></div><div><br></div><div>
One thing that may interest you is the recent ImageJ OPS announcement:</div><div> <a href="http://developer.imagej.net/2014/04/04/announcing-imagej-ops" target="_blank">http://developer.imagej.net/2014/04/04/announcing-imagej-ops</a></div>
<div><br></div><div>An Op is just an ImageJ command intended to be fully functional -- i.e., all inputs and outputs declared explicitly as parameters, with no side effects. They are essentially "lower level" commands intended for image processing, and they might be another good integration point for Alida. Really, your current implementation should gain access to them "for free" since they are just a subtype of Commands, but perhaps it makes sense to special case them somehow since they are in many ways friendlier for headless & non-dynamic situations?</div>
<div><br></div><div><div>> I am not sure whether we can recycle all IJ2s preprocessors as there</div><div>> might be differences in the concept of parameter handling.</div></div><div><br></div><div>Actually, there is a fair chance you could. It's just that the ActiveFooPreprocessor style things may either: A) always return null for you; or B) be populated via some alternate means that you can specify by extending the DefaultDisplayService, DefaultImageDisplayService, etc., with overridden method implementations that draw the "active" objects from somewhere else.</div>
<div><br></div><div>Alternately, it is just fine if they are always null, as long as you pass them in via the moduleService#run method "Object..." or "Map<String, Object>" arguments.</div><div><br>
</div><div></div></div><div><div>> In the attached zip of a tiny maven project using ij2 (2.0.0-beta-7.8)</div><div>> there is a command RunTestOp which tries to invoke a IJTestOp command</div><div>> "by hand".</div>
</div><div><br></div><div><span style="font-family:arial,sans-serif;font-size:13px">It seems the mailing list stripped your attachment. Can you please post on GitHub?</span></div><div><br></div><div><div>> I first create a CommandInfo for IJTestOp, then a CommandModule for this info,</div>
<div>> collects all preprocessors and (try to) invoke them for the CommandModule created.</div></div><div><div>> I disabled invoking of the InitPreprocessor, as it crashes with a null<br></div><div><div>> pointer exception:</div>
<div>> Exception in thread "main" java.lang.NullPointerException at</div><div>> imagej.command.CommandModule.initialize(CommandModule.java:144)</div></div><div><br></div><div>That error indicates that the CommandModule did not have a context injected properly. From your description, it sounds like your code is more low-level than it needs to be. But I'd have to see the code to be certain.</div>
<div><br></div><div>For customizing which pre- and post-processors get run, it is enough to do:</div><div><br></div><div>pre = pluginService.createInstancesOfType(PreprocessorPlugin.class);</div><div>post = pluginService.createInstancesOfType(PostprocessorPlugin.class);</div>
<div>// then loop over pre and post, removing blacklisted entries<br></div><div>// alternately, loop over pre and post, populating whitelists with desired entries</div><div>moduleService.run(moduleInfo, pre, post, inputName1, inputValue1, inputName2, inputValue2, ...etc...)<br>
</div><div><br></div><div>Is that how you were doing it?</div><div><br></div><div><div>// or if you really want to avoid creating instances which you then throw away...</div><div>// you can write:</div><div>// infos = pluginService.getPluginsOfType(PreprocessorPlugin.class)</div>
<div>// and/or:</div><div>// info = pluginService.getPlugin(InitPreprocessor.class)</div><div>// get the infos you want onto a list and then call:</div><div>// pre = pluginService.createInstances(info);</div><div>// or even this way:</div>
<div>// pre.add(pluginService.createInstance(InitPreprocessor.class);</div><div>// the advantage being that all context is injected automatically for you.</div></div><div><br></div></div><div>> And the ImageDisplayService is also still null.</div>
<div><br></div><div>Regardless of which preprocessors you use, such service parameters should not be null, as long as the context has that service associated with it; see below. Again, I'm guessing you didn't inject the context...</div>
<div><br></div><div><div>> BTW: when diving a little bit into the code I realized that Service</div><div>> and Context @Parameters of Commands are nor reflected in the</div><div>> CommandInfo. Somehow I was confused when I saw, that the</div>
<div>> ServicePreprocessor's process method looks for Service and Context</div><div>> parameters in the moduls CommandInfo.inputs(). Could there ever by</div><div>> any?</div></div><div><br></div><div>Yes. :-) You have found a wrinkle that I didn't expect anyone to notice yet.</div>
<div><br></div><div>In the case of Modules generally (not just Commands), there can certainly be Service and Context parameters. The prime example there is scripts: we want to annotate scripts written various languages such as Jython with a header comment that declares all the typed input and output parameters, including services. The ServicePreprocessor will take care of filling them, and the ModuleInfo API will include them when iterating the inputs.</div>
<div><br></div><div>However, in the case of Commands specifically, the Service and Context fields annotated with @Parameter will actually *not* be reported as parameters by the ModuleInfo API. It was changed 8 months ago:<br>
</div><div><br></div><div> <a href="https://github.com/imagej/imagej/commit/5a86785506f94dc0e58d5860559903b23fb64187" target="_blank">https://github.com/imagej/imagej/commit/5a86785506f94dc0e58d5860559903b23fb64187</a></div>
<div><br></div>
<div>Unfortunately, I did not bother to elaborate in my commit message on *why* "we do then need to have a special exclusion for such parameters in the CommandInfo wrapper" and I've forgotten what problems it caused not to have that special exclusion.</div>
<div><br></div><div>So this is certainly inconsistent and confusing. Do you think that Commands should also include service and context inputs when iterating them? If so, we can try removing that exclusion and see what explodes... ;-)</div>
<div><br></div><div><div>> I was aware of the option to create a context but was/am reluctant to<br></div></div><div><div>> use it (in excess) due to overhead considerations. Just looking at the</div><div>> logging output it seems pretty time consuming. Is this impression</div>
<div>> wrong?</div></div><div><br></div><div>Regardless of how you do things, you should be able to keep reusing the same context. You can either create one static one as I described, or you can create one non-static one and then pass it around everywhere to things that need it (either via constructors, setters, or by injecting it as a @Parameter -- whatever pattern suits your fancy).</div>
<div><br></div><div>That said, we have put a fair amount of effort into optimizing context creation. What it comes down to is which services are part of the context. Creating a full-blown ImageJ context (all available services, huge classpath) on my system takes about 2000ms right now; creating one with only core SciJava services takes 450ms. This is actually much slower than last I checked, probably due to the Eclipse Helper making absolutely sure the annotation metadata is up to date. We will certainly be optimizing this more in the future -- e.g., if you run outside of the Eclipse development environment, the Eclipse Helper should not bog things down.</div>
<div><br></div><div>Also, as of <a href="https://github.com/scijava/scijava-common/commit/0fd0e2916d39505e792f1bbd8e56e3de0f1f8904" target="_blank">https://github.com/scijava/scijava-common/commit/0fd0e2916d39505e792f1bbd8e56e3de0f1f8904</a>, the log spam when creating a context has been eliminated by default. (It got to the point where those log messages were really annoying me. ;-) The next release will of course include this change.</div>
<div><br></div><div><div>> In addition I am able to create only on instance of imagej.ImageJ, the</div><div>> second constructor crashes. See the file createContext.log in the zip</div><div>> which results from running RunTestOp. (It seems to me the creating a</div>
<div>> second instance of DefaultLegacyService fails, and I faintly remeber</div><div>> to have read somewhere that only one instance of the</div><div>> DefaultLegacyService may exist ??)</div></div><div><br></div>
<div>Indeed, trying to create two contexts each with a DefaultLegacyService fails right now. The reasons are complex, but in short, it is a bug. And unfortunately one we will not be able to address before the June release. The problem can be avoided by either: A) calling context.dispose() on the first context before creating the second one; or B) creating additional contexts beyond the first without a LegacyService (admittedly awkward to code since you can no longer use the default Context() constructor).</div>
<div><br></div><div>If you urgently need support for multiple simultaneous ImageJ contexts created with the default constructor while ij-legacy is on the classpath, and you want to try fixing the bug yourself, we would be happy to elaborate further so that you can give it a shot.</div>
<div><br></div><div><div>> The Command EquationDataValues.java ist annotated as headless and has</div><div>> a @Parameter of type Button. Is this as intended?</div></div><div><br></div><div>Yes, the "Button" is just a marker interface indicating that the input harvester UI should render a button with that label which when pressed calls that callback method. In the case of no input harvester UI, there will be no button, and the callback will hence never occur. But that should not interfere with headless operation. Such Button parameters can be safely ignored on your end.</div>
<div><br></div><div>Note that we recently made the Button interface extend the Optional interface to clarify the fact that Button "values" are always allowed to be null, even if the @Parameter of the Button does not declare "required = false". See: <a href="https://github.com/imagej/imagej/commit/09ed9f9b6178312d86083094732b2a1201f82e16" target="_blank">https://github.com/imagej/imagej/commit/09ed9f9b6178312d86083094732b2a1201f82e16</a></div>
<div><br></div><div>Let's hope our next exchange does not need to be quite so long winded...</div><div><br></div><div>Regards,</div><div>Curtis</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Apr 10, 2014 at 9:01 AM, Stefan Posch <span dir="ltr"><<a href="mailto:posch@informatik.uni-halle.de" target="_blank">posch@informatik.uni-halle.de</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Curtis,<br>
<br>
thank you very much for the details answers (and I did not mean to press you with<br>
regard to "delays" - this goes, of course, also for the future).<br>
<br>
Though very helpful a few issues are still open:<br>
<div><br>
>> What are the precoditions assumed when running a (headless) command<br>
> What happens is highly dependent on *how* you invoke the command...<br>
<br>
</div> Maybe I was unclear on this issue. May question is, what does a method<br>
of a Command (e.g. its initializer, the run method) or rather the implementor<br>
may assumed to have "happend".<br>
<br>
Or other way round: in case the run-method of OpenFile command used in the tutorial<br>
requires the initializer to have been executed:<br>
Is the run-method responsible to check, if it has already been executed (e.g. if called<br>
via the invokeWithArgs or invokeWihtMap methods of the tutorial) or not<br>
(if e.g. called via invokeFrmoJava).<br>
Or is it the callers responsibility to ensure the the initializer has been already been<br>
executed before actually running the OpenFile command (if it indeed requires the initializer to<br>
have been invoked beforehand).<br>
<br>
The same questions goes for the initializer method: may it assume that a single Dataset<br>
parameter has already been set? And if so, may it asume other prerequisites as well?<br>
E.g. it seems to me that the initializer of DuplicateImage would probably crash if<br>
the inputDisplay is null.<br>
Or is it the other way round: That whoever (indirectly) invokes the initializer is<br>
responsible for certain parameters to be set in advance.<br>
<br>
Besides interesting for a programmer of a Command we are interested in this issue<br>
to be able to wrap IJ2 commands in order to run them from our GUI (originaly design<br>
for alida operators), our command line runner,<br>
and to support them in our graphical editor for alida/mitobo operators and hopefully<br>
ij2 commands (at least head less and non DynamicCommands)<br>
<div><br>
> It should be case that Alida can reuse the default pre- and post-processing<br>
> plugin stack -- in other words, you should be able to pass "process=true"<br>
> to the ModuleService#run and everything will "just work". Let us know if<br>
> not, and we can troubleshoot. As long as no UI has been shown, you will be<br>
> in headless mode and no dialogs should ever be shown. (If one does pop up,<br>
> it is probably a bug.)<br>
<br>
</div> I am not sure whether we can recycle all IJ2s preprocessors as there might be differences<br>
in the concept of parameter handling.<br>
<br>
But trying to reused (some of) them "by hand", not by using the Modules run method,<br>
I somehow got stuck, as obviously I do not understand the concepts<br>
properly. Maybe you are willing to help with this.<br>
<br>
In the attached zip of a tiny maven project using ij2 (2.0.0-beta-7.8)<br>
there is a command RunTestOp which tries to invoke a IJTestOp command "by hand".<br>
I first create a CommandInfo for IJTestOp, then a CommandModule for this info,<br>
collects all preprocessors and (try to) invoke them for the CommandModule created.<br>
I disabled invoking of the InitPreprocessor, as it crashes with a null pointer exception:<br>
Exception in thread "main" java.lang.NullPointerException<br>
at imagej.command.CommandModule.initialize(CommandModule.java:144)<br>
at imagej.module.process.InitPreprocessor.process(InitPreprocessor.java:61)<br>
at mainroutine.RunTestOp.main(RunTestOp.java:49)<br>
<br>
Additionally it seems to be that creating the CommandInfo already invokes preprocessors,<br>
however the initializer of IJTestOp is not invoked.<br>
<br>
And the ImageDisplayService is also still null.<br>
<br>
BTW: when diving a little bit into the code I realized that<br>
Service and Context @Parameters of Commands are nor reflected in the CommandInfo.<br>
Somehow I was confused when I saw, that the ServicePreprocessor's process method<br>
looks for Service and Context parameters in the moduls CommandInfo.inputs().<br>
Could there ever by any?<br>
<div><br>
> modules should not be dynamic. Dynamic commands are *much* more challenging<br>
> to support across many different environments (CellProfiler, KNIME, OMERO,<br>
> Alida, etc.) *much* more challenging.<br>
</div> Is it easily possible to state what makes a Command a DynamicCommand,<br>
and what is the difference to interactive() commands?<br>
<div><br>
> On the CLI, we'll harvest values from the user at the same point in time<br>
> that we currently do it via the UI. So most of the other preprocessing will<br>
> be done; there will be a "CLIInputHarvesterPlugin" that prompts the user to<br>
> type in these values using System.in or similar. We have not yet created<br>
> this preprocessor plugin, but it would be very straightforward. If you need<br>
> this, let me know -- it would be a fun side project. :-)<br>
</div> We have one "command line oprunner" for alida<br>
(which does not work interactively, thus can be used also from shell-scripts,<br>
e.g. for parameter tuning). And as noted above would like to support also<br>
ij2 commands.<br>
<div><br>
> The SciJava context is definitely not a singleton. But you can use it that<br>
> way if you want: just create one a static variable in your own codebase<br>
> somewhere; e.g.:<br>
><br>
><br>
> public static final imagej.ImageJ IJ = new imagej.ImageJ();<br>
<br>
</div> I was aware of the option to create a context but was/am reluctant to use<br>
it (in excess) due to overhead considerations. Just looking at the logging output<br>
it seems pretty time consuming. Is this impression wrong?<br>
<br>
In addition I am able to create only on instance of imagej.ImageJ, the second<br>
constructor crashes. See the file createContext.log in the zip which results from<br>
running RunTestOp.<br>
(It seems to me the creating a second instance of DefaultLegacyService fails,<br>
and I faintly remeber to have read somewhere that only one instance of the<br>
DefaultLegacyService may exist ??)<br>
<br>
A new (and for the moment last) question:<br>
The Command EquationDataValues.java ist annotated as headless and has a @Parameter<br>
of type Button. Is this as intended?<br>
<br>
Again thanks a lot<br>
<br>
Best regards Stefan<br>
<div><div><br>
_______________________________________________<br>
ImageJ-devel mailing list<br>
<a href="mailto:ImageJ-devel@imagej.net" target="_blank">ImageJ-devel@imagej.net</a><br>
<a href="http://imagej.net/mailman/listinfo/imagej-devel" target="_blank">http://imagej.net/mailman/listinfo/imagej-devel</a><br>
</div></div></blockquote></div><br></div></div>