[ImageJ-devel] About memory usage with Fiji/JVM

[Hadrien Mary]

Thank you Mark for the answer.

About ImgMode I wonder what is the difference between these three modes :

- ArrayImgFactory
- CellImgFactory
- PlanarImgFactory

Sorry if it is something obvious but I often see these modes without
knowing how they work.

Best,

--
Hadrien Mary

Ph.D student in Biology
Tournier-Gachet Team
CNRS - LBCMCP - UMR 5088

Université de Toulouse - Bât. 4R3B1
118, route de Narbonne - 31062 Toulouse


On Fri, Mar 13, 2015 at 7:43 PM, Mark Hiner <hiner at wisc.edu> wrote:
> Hi Hadrien,
>
>>My hypothesis is that when loading an image, Fiji/SCIFIO look at free
>>memory in the system but not in the JVM. Is that possible ?
>
> Yes, this is almost certainly what's happening. Unfortunately I am not aware
> of a way to differentiate between memory that is still in use, and memory
> that just hasn't been deallocated.
>
> By default in these situations SCIFIO will look at the available memory, see
> that there's not enough to open the dataset and open it virtually. Note that
> it actually would be possible to open the full dataset - as the memory
> allocation approached the JVM's limit, it would automatically be released.
>
> Right now this can be worked around programmatically by specifying a PLANAR
> or ARRAY[1] ImgMode in the SCIFIOConfig[2]. The Bio-Formats importer lets
> you specify whether the image should be opened virtually or not. The only
> other option is to click the status bar to force garbage collection.
>
> We also have plans to improve the SCIFIO importer[3] to expose these
> settings in the UI. Ultimately the hope is that virtual (cell) images will
> incur minimal performance penalties though[4], making this issue moot.
>
> Thanks!
> - Mark
>
> [1]
> https://github.com/scifio/scifio/blob/scifio-0.21.1/src/main/java/io/scif/config/SCIFIOConfig.java#L124
> [2]
> https://github.com/scifio/scifio/blob/scifio-0.21.1/src/main/java/io/scif/config/SCIFIOConfig.java#L370-373
> [3] https://github.com/imagej/imagej-plugins-commands/issues/4
> [4] https://github.com/scifio/scifio/issues/201
>
>
> On Mon, Jan 26, 2015 at 5:57 AM, Hadrien Mary <hadrien.mary at gmail.com>
> wrote:
>>
>> Hi,
>>
>> New SCIFIO I/O system in Fiji/IJ2 automatically detects wether virtual
>> stack is needed according to system memory available and image size
>> (if I understood correctly).
>>
>> Virtual stack is fine but it always better to work without when you
>> can. For example working with timelapse stacks, sliding time is much
>> faster when image is not a virtual stack (and it makes sense since RAM
>> memory access is much faster than hard drive memory access...).
>>
>> Saying that I noticed something: when I close a big image (no virtual
>> stack because moving time slider is fast), java does not release
>> memory to system (and it's a normal behaviour according to this
>>
>> http://stackoverflow.com/questions/22912063/automatically-release-unused-memory-in-imagej-fiji).
>> Now when I load the same kind of image (same size), the JVM should use
>> the free memory released by the previously closed image isn't it ? I
>> have the feeling that it does not because, when I move the time slider
>> on the second image, it's very slow....
>>
>> Some numbers:
>>
>> - image size : 4.6GB
>> - Fiji memory after start : 0.3GB
>> - Fiji memory after loading first image: 6.9GiB
>> - Fiji memory after closing first image: 6.9GiB
>> - Fiji memory after loading second image: 6.9GiB
>>
>> My hypothesis is that when loading an image, Fiji/SCIFIO look at free
>> memory in the system but not in the JVM. Is that possible ? Did you
>> already noticed that ?
>>
>> PS: I have attached to this mail some properties of my system.
>>
>> Best,
>>
>> --
>> Hadrien Mary
>>
>> Ph.D student in Biology
>> Tournier-Gachet Team
>> CNRS - LBCMCP - UMR 5088
>>
>> Université de Toulouse - Bât. 4R3B1
>> 118, route de Narbonne - 31062 Toulouse
>>
>> _______________________________________________
>> ImageJ-devel mailing list
>> ImageJ-devel at imagej.net
>> http://imagej.net/mailman/listinfo/imagej-devel
>>
>