[ImageJ-devel] [fiji-devel] slow opening of tif files using scifio-based ImgOpener

[Tobias Pietzsch]

Hi Melissa,

That is awesome! Thanks a lot for tackling these problems.

Out of curiosity, could you briefly explain (if there is an easy explanation) why the 100.000-files-issue happens?
Is bioformats analyzing the directory the TIFF file lives in? Is there a way to turn this off?

best regards,
Tobias

On Feb 6, 2013, at 3:08 AM, Melissa Linkert wrote:

> Hi Tobias,
> 
>> recently I ran across a problem with the scifio-based ImgOpener of ImgLib2.
>> I wanted to load stacks from series of tif files which were all in the same directory (about 100.000 of them).
>> With ImgOpener, this was imbearably slow, so I decided to load the files using ImageJ1 and wrapping the resulting ImagePluses into imglib2 containers.
>> 
>> Today, I created a benchmark, that illustrates the problem.
>> You can find it here: https://github.com/tpietzsch/TifBenchmark
>> It is derived from Johannes' https://github.com/imagej/minimal-ij1-plugin, so you can simply import the maven project in Eclipse and try it out easily.
>> 
>> The results are disillusioning.
>> With just a single tif, scifio opens the file 10x slower than ImageJ1. Bad, but maybe bearable.
>> With 100.000 tif files in the same directory, scifio is 400x slower! That is 0.1s for loading 50 tifs from ImageJ vs 55s using scifio!!!
>> 
>> Can anyone explain this behavior?
> 
> Bio-Formats/SCIFIO is known to be slower than ImageJ when opening plain
> TIFF files.  See:
> 
> http://trac.openmicroscopy.org.uk/ome/ticket/9228
> 
> That ticket (and several other performance improvements in Bio-Formats) are at
> the top of my to-do list for the next few weeks.
> 
> Regards,
> -Melissa
> 
> On Wed, Feb 06, 2013 at 01:10:45AM +0100, Tobias Pietzsch wrote:
>> Hi,
>> 
>> recently I ran across a problem with the scifio-based ImgOpener of ImgLib2.
>> I wanted to load stacks from series of tif files which were all in the same directory (about 100.000 of them).
>> With ImgOpener, this was imbearably slow, so I decided to load the files using ImageJ1 and wrapping the resulting ImagePluses into imglib2 containers.
>> 
>> Today, I created a benchmark, that illustrates the problem.
>> You can find it here: https://github.com/tpietzsch/TifBenchmark
>> It is derived from Johannes' https://github.com/imagej/minimal-ij1-plugin, so you can simply import the maven project in Eclipse and try it out easily.
>> 
>> The results are disillusioning.
>> With just a single tif, scifio opens the file 10x slower than ImageJ1. Bad, but maybe bearable.
>> With 100.000 tif files in the same directory, scifio is 400x slower! That is 0.1s for loading 50 tifs from ImageJ vs 55s using scifio!!!
>> 
>> Can anyone explain this behavior?
>> This needs to be fixed if we expect anyone to seriously use the ImgOpener…
>> 
>> best regards,
>> Tobias
>> 
>> -- 
>> -- 
>> Please avoid top-posting, and please make sure to reply-to-all!
>> 
>> Mailing list web interface: http://groups.google.com/group/fiji-devel
>> 
>> --- 
>> You received this message because you are subscribed to the Google Groups "Fiji-devel" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to fiji-devel+unsubscribe at googlegroups.com.
>> For more options, visit https://groups.google.com/groups/opt_out.
>> 
>>