[ImageJ-devel] About memory usage with Fiji/JVM

[Mark Hiner]

Hi Hadrien,

>My hypothesis is that when loading an image, Fiji/SCIFIO look at free
>memory in the system but not in the JVM. Is that possible ?

Yes, this is almost certainly what's happening. Unfortunately I am not
aware of a way to differentiate between memory that is still in use, and
memory that just hasn't been deallocated.

By default in these situations SCIFIO will look at the available memory,
see that there's not enough to open the dataset and open it virtually. Note
that it actually would be possible to open the full dataset - as the memory
allocation approached the JVM's limit, it would automatically be released.

Right now this can be worked around programmatically by specifying a PLANAR
or ARRAY[1] ImgMode in the SCIFIOConfig[2]. The Bio-Formats importer lets
you specify whether the image should be opened virtually or not. The only
other option is to click the status bar to force garbage collection.

We also have plans to improve the SCIFIO importer[3] to expose these
settings in the UI. Ultimately the hope is that virtual (cell) images will
incur minimal performance penalties though[4], making this issue moot.

Thanks!
- Mark

[1]
https://github.com/scifio/scifio/blob/scifio-0.21.1/src/main/java/io/scif/config/SCIFIOConfig.java#L124
[2]
https://github.com/scifio/scifio/blob/scifio-0.21.1/src/main/java/io/scif/config/SCIFIOConfig.java#L370-373
[3] https://github.com/imagej/imagej-plugins-commands/issues/4
[4] https://github.com/scifio/scifio/issues/201


On Mon, Jan 26, 2015 at 5:57 AM, Hadrien Mary <hadrien.mary at gmail.com>
wrote:

> Hi,
>
> New SCIFIO I/O system in Fiji/IJ2 automatically detects wether virtual
> stack is needed according to system memory available and image size
> (if I understood correctly).
>
> Virtual stack is fine but it always better to work without when you
> can. For example working with timelapse stacks, sliding time is much
> faster when image is not a virtual stack (and it makes sense since RAM
> memory access is much faster than hard drive memory access...).
>
> Saying that I noticed something: when I close a big image (no virtual
> stack because moving time slider is fast), java does not release
> memory to system (and it's a normal behaviour according to this
>
> http://stackoverflow.com/questions/22912063/automatically-release-unused-memory-in-imagej-fiji
> ).
> Now when I load the same kind of image (same size), the JVM should use
> the free memory released by the previously closed image isn't it ? I
> have the feeling that it does not because, when I move the time slider
> on the second image, it's very slow....
>
> Some numbers:
>
> - image size : 4.6GB
> - Fiji memory after start : 0.3GB
> - Fiji memory after loading first image: 6.9GiB
> - Fiji memory after closing first image: 6.9GiB
> - Fiji memory after loading second image: 6.9GiB
>
> My hypothesis is that when loading an image, Fiji/SCIFIO look at free
> memory in the system but not in the JVM. Is that possible ? Did you
> already noticed that ?
>
> PS: I have attached to this mail some properties of my system.
>
> Best,
>
> --
> Hadrien Mary
>
> Ph.D student in Biology
> Tournier-Gachet Team
> CNRS - LBCMCP - UMR 5088
>
> Université de Toulouse - Bât. 4R3B1
> 118, route de Narbonne - 31062 Toulouse
>
> _______________________________________________
> ImageJ-devel mailing list
> ImageJ-devel at imagej.net
> http://imagej.net/mailman/listinfo/imagej-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://imagej.net/pipermail/imagej-devel/attachments/20150313/c629d59d/attachment.html>