[ImageJ-devel] [fiji-devel] Imglib2: using threadpools in core algorithms

[Curtis Rueden]

Hi all,

After sleeping on it, I do not think SJC ThreadService is actually the way
to go for providing better threading configuration of algorithms. But
unlike Tobias, I do *not* think this is because SJC contexts are
burdensome. Specifically, in response to Tobias's comment:

> [SJC] is absolutely not the right thing if you are building a library in
my opinion.

I strongly disagree; SJC was created to benefit SCIFIO, and it has done so
tremendously. I can elaborate on this if you like, but suffice to say that
even libraries have configuration and state and services that they need to
provide, and a unified context is an excellent way to do this. If you
instead rely on statics, you will create something that cannot be extended.
We learned this from painful experience with Bio-Formats!

Anyway, the reason I think SJC ThreadService is wrong for this use case is
due to another bullet point I mentioned earlier:

> Easy to override threading behavior on a case-by-case basis
> (i.e., pass different ExecutorServices to different algorithms).

That is, SJC services are singletons, intended to store state for the
application context. The threading configuration for running an algorithm
might be different than that for another algorithm running in the same
context. We need to be able to configure it individually. While
ThreadService might be able to serve as a sensible default configuration
(i.e., "just use SJC's ExecutorService by default"), it should not be
forced on every algorithm.

> As I pointed out, this has been done exactly right in Java 7 Fork/Join
> framework (by people who presumably put more thought and experience
> into it than we could). If the majority insists that we reimplement
> something similar, then at least let us use the same interfaces.

I agree that it would be great if we could use something that has been
vetted by the greater Java community. However, this makes porting to
non-JavaSE-7 platforms much more difficult. Tobias, I know you pointed out
some java.util.concurrent leakages into ThreadService -- we weren't being
overly careful about it yet -- but my point stands that SJC has the
potential to make those sorts of ports easier if we put effort into it.
More Jenkins builds against alternative Java implementations would be a
great start toward that.

Anyway, it's not that I *want* to reimplement something similar to Java 7's
Fork/Join... it's that I don't have a good alternative if we also want to
support those other scenarios. One possible compromise would be design our
own agnostic interface, and provide an implementation in its own module
which uses Java 7's Fork/Join. That way, it will be possible to provide an
alternative implementation on platforms which don't support it. This sort
of interface-driven extensible design is *exactly* what SJC seeks to make
possible with its plugin framework.

Regards,
Curtis


On Fri, Dec 6, 2013 at 12:39 PM, Tobias Pietzsch <pietzsch at mpi-cbg.de>wrote:

> Hi,
>
> On Dec 6, 2013, at 7:28 PM, Johannes Schindelin <
> Johannes.Schindelin at gmx.de> wrote:
>
> > Hi Lee,
> >
> > On Fri, 6 Dec 2013, Lee Kamentsky wrote:
> >>
> >> On Fri, Dec 6, 2013 at 10:10 AM, Tobias Pietzsch <pietzsch at mpi-cbg.de
> >wrote:
> >>
> >>> Problem: Did anyone think about the possibility of deadlocks?
> >>
> >> Nice catch, Tobias
> >
> > Does anybody have a different reaction than "Oh well, that's right, we
> > cannot use the ExecutorService, then, but instead need to adapt the
> > ThreadService so it handles this one right"?
>
> Here, I have a different reaction:
> As I pointed out, this has been done exactly right in Java 7 Fork/Join
> framework (by people who presumably put more thought and experience into it
> than we could).
> If the majority insists that we reimplement something similar, then at
> least let us use the same interfaces.
>
> best regards,
> Tobias
>
> >
> > Ciao,
> > Dscho
> >
> > --
> > --
> > Please avoid top-posting, and please make sure to reply-to-all!
> >
> > Mailing list web interface: http://groups.google.com/group/fiji-devel
> >
> > ---
> > You received this message because you are subscribed to the Google
> Groups "Fiji-devel" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to fiji-devel+unsubscribe at googlegroups.com.
> > For more options, visit https://groups.google.com/groups/opt_out.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://imagej.net/pipermail/imagej-devel/attachments/20131206/c616d6de/attachment.html>