[ImageJ-devel] [fiji-devel] Imglib2: using threadpools in core algorithms

[Tobias Pietzsch]

Hi Curtis,

> > [SJC] is absolutely not the right thing if you are building a library in my opinion.
> 
> I strongly disagree; SJC was created to benefit SCIFIO, and it has done so tremendously. I can elaborate on this if you like, but suffice to say that even libraries have configuration and state and services that they need to provide, and a unified context is an excellent way to do this. If you instead rely on statics, you will create something that cannot be extended. We learned this from painful experience with Bio-Formats!


Yes, could you please elaborate on this!
Maybe I should say that I have just recently (two days ago to be honest) started to seriously get into scijava-common and I like it very much so far. But I still don't see where it would benefit imglib2 core. I would like to have imglib2 be as close to purely functional as possible, no state, no side effects. As much as possible, at least.
Could you please explain where SCIFIO benefits from being stateful and where being stateful would probably help imglib?

best regards,
Tobias

On Dec 6, 2013, at 7:52 PM, Curtis Rueden <ctrueden at wisc.edu> wrote:

> Hi all,
> 
> After sleeping on it, I do not think SJC ThreadService is actually the way to go for providing better threading configuration of algorithms. But unlike Tobias, I do *not* think this is because SJC contexts are burdensome. Specifically, in response to Tobias's comment:
> 
> > [SJC] is absolutely not the right thing if you are building a library in my opinion.
> 
> I strongly disagree; SJC was created to benefit SCIFIO, and it has done so tremendously. I can elaborate on this if you like, but suffice to say that even libraries have configuration and state and services that they need to provide, and a unified context is an excellent way to do this. If you instead rely on statics, you will create something that cannot be extended. We learned this from painful experience with Bio-Formats!
> 
> Anyway, the reason I think SJC ThreadService is wrong for this use case is due to another bullet point I mentioned earlier:
> 
> > Easy to override threading behavior on a case-by-case basis
> > (i.e., pass different ExecutorServices to different algorithms).
> 
> That is, SJC services are singletons, intended to store state for the application context. The threading configuration for running an algorithm might be different than that for another algorithm running in the same context. We need to be able to configure it individually. While ThreadService might be able to serve as a sensible default configuration (i.e., "just use SJC's ExecutorService by default"), it should not be forced on every algorithm.
> 
> > As I pointed out, this has been done exactly right in Java 7 Fork/Join
> > framework (by people who presumably put more thought and experience
> > into it than we could). If the majority insists that we reimplement
> > something similar, then at least let us use the same interfaces.
> 
> I agree that it would be great if we could use something that has been vetted by the greater Java community. However, this makes porting to non-JavaSE-7 platforms much more difficult. Tobias, I know you pointed out some java.util.concurrent leakages into ThreadService -- we weren't being overly careful about it yet -- but my point stands that SJC has the potential to make those sorts of ports easier if we put effort into it. More Jenkins builds against alternative Java implementations would be a great start toward that.
> 
> Anyway, it's not that I *want* to reimplement something similar to Java 7's Fork/Join... it's that I don't have a good alternative if we also want to support those other scenarios. One possible compromise would be design our own agnostic interface, and provide an implementation in its own module which uses Java 7's Fork/Join. That way, it will be possible to provide an alternative implementation on platforms which don't support it. This sort of interface-driven extensible design is *exactly* what SJC seeks to make possible with its plugin framework.
> 
> Regards,
> Curtis
> 
> 
> On Fri, Dec 6, 2013 at 12:39 PM, Tobias Pietzsch <pietzsch at mpi-cbg.de> wrote:
> Hi,
> 
> On Dec 6, 2013, at 7:28 PM, Johannes Schindelin <Johannes.Schindelin at gmx.de> wrote:
> 
> > Hi Lee,
> >
> > On Fri, 6 Dec 2013, Lee Kamentsky wrote:
> >>
> >> On Fri, Dec 6, 2013 at 10:10 AM, Tobias Pietzsch <pietzsch at mpi-cbg.de>wrote:
> >>
> >>> Problem: Did anyone think about the possibility of deadlocks?
> >>
> >> Nice catch, Tobias
> >
> > Does anybody have a different reaction than "Oh well, that's right, we
> > cannot use the ExecutorService, then, but instead need to adapt the
> > ThreadService so it handles this one right"?
> 
> Here, I have a different reaction:
> As I pointed out, this has been done exactly right in Java 7 Fork/Join framework (by people who presumably put more thought and experience into it than we could).
> If the majority insists that we reimplement something similar, then at least let us use the same interfaces.
> 
> best regards,
> Tobias
> 
> >
> > Ciao,
> > Dscho
> >
> > --
> > --
> > Please avoid top-posting, and please make sure to reply-to-all!
> >
> > Mailing list web interface: http://groups.google.com/group/fiji-devel
> >
> > ---
> > You received this message because you are subscribed to the Google Groups "Fiji-devel" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to fiji-devel+unsubscribe at googlegroups.com.
> > For more options, visit https://groups.google.com/groups/opt_out.
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://imagej.net/pipermail/imagej-devel/attachments/20131206/6b396197/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 455 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://imagej.net/pipermail/imagej-devel/attachments/20131206/6b396197/attachment.pgp>