[ImageJ-devel] [fiji-devel] Status reporting in ImgLib Algorithms

[Curtis Rueden]

Hi Tobias,

> I had a quick look and saw that scijava-common is itself not without
> dependencies. I found it depends on org.bushe.eventbus.

Yes. SciJava Common also depends on SezPoz. However, both of these
dependencies are encapsulated, meaning that no downstream code is
"infected" with any direct references to classes from either SezPoz or
EventBus. All event logic goes through the SciJava Common EventService. And
all SezPoz logic is transparent to the developer, via APT.

> On the eventbus website I found the notice:
> Update 3/2013: The EventBus has gone stale

Thanks for the info; I hadn't seen that. Fortunately, the version of
EventBus we use does everything we need. We are unlikely to need upstream
support or new versions. At worst, we can fork the project and cut our own
new release (and/or integrate the EventBus code into SciJava Common itself)
in case a future critical bug is discovered.

> I would favour a way where we could keep the imglib2 code focused on
> processing pixels, not caring about framework issues.

What about Mark's option (A): "imglib2 needs its own status reporting
mechanism (Listeners and Events etc.)"?

> I think the overhead will be quite high in some cases. For instance in
> your forked ComputeMinMax algorithm you do some work on updating
> status for every single pixel, which I think is definitely overdoing
> it.

Yes, there is always a tradeoff, and a balance, for this problem. I agree
that status-per-pixel is overkill.

What about polling? Another option might be for ImgLib2 algorithms to
update some (standardized) internal state, but not inform anyone about such
changes. Then downstream code could poll that state every X milliseconds.

Regards,
Curtis


On Mon, Aug 5, 2013 at 11:01 AM, Tobias Pietzsch <pietzsch at mpi-cbg.de>wrote:

> Hi all,
>
> First of all, I am very reluctant to add any dependency to
> imglib2-algorithms core.
>
> I admit that I did not really look into scijava-common so far, so I cannot
> really comment on how severe I think this would be.
> I had a quick look and saw that scijava-common is itself not without
> dependencies. I found it depends on org.bushe.eventbus. On the eventbus
> website I found the notice:
> Update 3/2013: The EventBus has gone stale and is unlikely to be extended
> to other platforms. Suitable, though more primitive, replacements can be
> found for GWT (GWT EventBus<http://google-web-toolkit.googlecode.com/svn/javadoc/2.1/com/google/gwt/event/shared/EventBus.html>),
> Javascript (TIBCO PageBus - an OpenAjax Hub implementation<http://developer.tibco.com/pagebus/default.jsp>)
> and Flex (Parsley's Messaging<http://www.spicefactory.org/parsley/docs/3.0/manual/>
> )
> So would that mean dragging in a dead project as a transitive dependency?
>
> As far as I can tell, the net.imglib2.multithreading stuff is a leftover
> from imglib1 times and I think I have mentioned several times that I would
> like to get rid of it… :-)
> I guess that scijava-common ThreadService is more thought-through than the
> SimpleMultiThreading class. So replacing that I would see as an advantage.
> In general, I think ExecutorServices are a good fit for multithreading
> pixel processing algorithms, along the lines of submitting a runnable for
> every line/plane/.
>
> I see that you have a need for status reporting in ImageJ2 but I'm not
> sure that it should be in imglib-algorithms.
> I would favour a way where we could keep the imglib2 code focused on
> processing pixels, not caring about framework issues.
> In principle, status reporting can be performed before entering and after
> leaving a imglib2 "work-unit".
> If fine grained reporting is required, then this would mean splitting up
> the work into smaller units and reporting in between.
>
> In any case, even if we add scijava-common as a dependency, I strongly
> suggest to also keep non-reporting versions of the algorithms.
> If noone is listening to the status reports thats just wasted computation
> time. I think the overhead will be quite high in some cases.
> For instance in your forked ComputeMinMax algorithm you do some work on
> updating status for every single pixel, which I think is definitely
> overdoing it.
>
> As a compromise I could image putting an empty protected method report()
> that is called periodically by the core algorithm implementation.
> The JIT would hopefully just optimize that away if you use the core
> algorithm. Then for the reporting version you would have to extend the core
> algorithm and override report().
>
> best regards,
> Tobias
>
>
>
>
> On Jul 26, 2013, at 5:09 PM, Mark Hiner <hinerm at gmail.com> wrote:
>
> Hi all (especially ImgLib2 developers)
>
>   Up until now, when running ImgLib2 algorithms on large datasets in
> ImageJ2, the UI was basically frozen for the complete compute time (e.g.
> when computing Min/Max values in the Brightness/Contrast plugin).
>
>   As of this commit<https://github.com/imagej/imagej/commit/dde5c3529e4b5daf78eb899a781635d4730ecb85>UI-based refreshing and event callback methods are now handled on a
> separate thread, allowing them to send UI updates (e.g. via the
> StatusService).
>
>   For the moment, to avoid adding a Scijava-common dependency to ImgLib2
> algorithms, I forked the ComputeMinMax algorithm<https://github.com/imagej/imagej/blob/ffb3d651811089e582c0698701a1a1ea19151db9/core/data/src/main/java/imagej/data/minmax/DefaultMinMaxMethod.java>in IJ2 to use the StatusService to report computation progress.
>
>   Going forward though, we should agree on a long term solution:
>
>   A) imglib2 needs its own status reporting mechanism (Listeners and
> Events etc.)
>   B) it needs a SciJava context to use the StatusService
>   C) we have to fork every core algorithm in ImgLib2 to add StatusService
>
>   If possible I would like to avoid C, but if it becomes a necessity it
> would be helpful if the algorithms were refactored for extensibility
> (having protected field accessors, for example).
>
>   I personally am in favor of B because it would limit code repetition and
> pave the way for further unification (Part of why I completely forked
> ComputeMinMax was to use the Scijava-common ThreadService, which has some -
> I think unnecessary - overlap with the SimpleMultiThreading utility class).
>
>   But I understand that adding this dependency is not a minor request, and
> care would have to be taken to limit the impact on performance.
>
>   So what do other people think?
>
> Thanks,
> - Mark
>
> --
> --
> Please avoid top-posting, and please make sure to reply-to-all!
>
> Mailing list web interface: http://groups.google.com/group/fiji-devel
>
> ---
> You received this message because you are subscribed to the Google Groups
> "Fiji-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to fiji-devel+unsubscribe at googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
>
>  --
> --
> Please avoid top-posting, and please make sure to reply-to-all!
>
> Mailing list web interface: http://groups.google.com/group/fiji-devel
>
> ---
> You received this message because you are subscribed to the Google Groups
> "Fiji-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to fiji-devel+unsubscribe at googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://imagej.net/pipermail/imagej-devel/attachments/20130805/7aeb26ce/attachment.html>