[ImageJ-devel] [fiji-devel] Imglib2: using threadpools in core algorithms

[Stephan Saalfeld]

Hi all,

what about having a Java 7 Fork/Join inspired ThreadService as an
independent component forming a dependency of SJC?  As SJC grows,
cutting it into pieces that can optionally be grouped by parent-poms is
probably due sooner or later anyways.  May be this is the day of birth
for SJCC (SJC-concurrency).

Tobias, thanks for bringing up the deadlock issue!

Best,
Stephan



On Fri, 2013-12-06 at 12:52 -0600, Curtis Rueden wrote: 
> Hi all,
> 
> After sleeping on it, I do not think SJC ThreadService is actually the way
> to go for providing better threading configuration of algorithms. But
> unlike Tobias, I do *not* think this is because SJC contexts are
> burdensome. Specifically, in response to Tobias's comment:
> 
> > [SJC] is absolutely not the right thing if you are building a library in
> my opinion.
> 
> I strongly disagree; SJC was created to benefit SCIFIO, and it has done so
> tremendously. I can elaborate on this if you like, but suffice to say that
> even libraries have configuration and state and services that they need to
> provide, and a unified context is an excellent way to do this. If you
> instead rely on statics, you will create something that cannot be extended.
> We learned this from painful experience with Bio-Formats!
> 
> Anyway, the reason I think SJC ThreadService is wrong for this use case is
> due to another bullet point I mentioned earlier:
> 
> > Easy to override threading behavior on a case-by-case basis
> > (i.e., pass different ExecutorServices to different algorithms).
> 
> That is, SJC services are singletons, intended to store state for the
> application context. The threading configuration for running an algorithm
> might be different than that for another algorithm running in the same
> context. We need to be able to configure it individually. While
> ThreadService might be able to serve as a sensible default configuration
> (i.e., "just use SJC's ExecutorService by default"), it should not be
> forced on every algorithm.
> 
> > As I pointed out, this has been done exactly right in Java 7 Fork/Join
> > framework (by people who presumably put more thought and experience
> > into it than we could). If the majority insists that we reimplement
> > something similar, then at least let us use the same interfaces.
> 
> I agree that it would be great if we could use something that has been
> vetted by the greater Java community. However, this makes porting to
> non-JavaSE-7 platforms much more difficult. Tobias, I know you pointed out
> some java.util.concurrent leakages into ThreadService -- we weren't being
> overly careful about it yet -- but my point stands that SJC has the
> potential to make those sorts of ports easier if we put effort into it.
> More Jenkins builds against alternative Java implementations would be a
> great start toward that.
> 
> Anyway, it's not that I *want* to reimplement something similar to Java 7's
> Fork/Join... it's that I don't have a good alternative if we also want to
> support those other scenarios. One possible compromise would be design our
> own agnostic interface, and provide an implementation in its own module
> which uses Java 7's Fork/Join. That way, it will be possible to provide an
> alternative implementation on platforms which don't support it. This sort
> of interface-driven extensible design is *exactly* what SJC seeks to make
> possible with its plugin framework.
> 
> Regards,
> Curtis
> 
> 
> On Fri, Dec 6, 2013 at 12:39 PM, Tobias Pietzsch <pietzsch at mpi-cbg.de>wrote:
> 
> > Hi,
> >
> > On Dec 6, 2013, at 7:28 PM, Johannes Schindelin <
> > Johannes.Schindelin at gmx.de> wrote:
> >
> > > Hi Lee,
> > >
> > > On Fri, 6 Dec 2013, Lee Kamentsky wrote:
> > >>
> > >> On Fri, Dec 6, 2013 at 10:10 AM, Tobias Pietzsch <pietzsch at mpi-cbg.de
> > >wrote:
> > >>
> > >>> Problem: Did anyone think about the possibility of deadlocks?
> > >>
> > >> Nice catch, Tobias
> > >
> > > Does anybody have a different reaction than "Oh well, that's right, we
> > > cannot use the ExecutorService, then, but instead need to adapt the
> > > ThreadService so it handles this one right"?
> >
> > Here, I have a different reaction:
> > As I pointed out, this has been done exactly right in Java 7 Fork/Join
> > framework (by people who presumably put more thought and experience into it
> > than we could).
> > If the majority insists that we reimplement something similar, then at
> > least let us use the same interfaces.
> >
> > best regards,
> > Tobias
> >
> > >
> > > Ciao,
> > > Dscho
> > >
> > > --
> > > --
> > > Please avoid top-posting, and please make sure to reply-to-all!
> > >
> > > Mailing list web interface: http://groups.google.com/group/fiji-devel
> > >
> > > ---
> > > You received this message because you are subscribed to the Google
> > Groups "Fiji-devel" group.
> > > To unsubscribe from this group and stop receiving emails from it, send
> > an email to fiji-devel+unsubscribe at googlegroups.com.
> > > For more options, visit https://groups.google.com/groups/opt_out.
> >
> >
> 
> --