[ImageJ-devel] About memory usage with Fiji/JVM

[Mark Hiner]

>Sorry if it is something obvious but I often see these modes without
knowing how they work.

No problem at all.

This option decides which underlying ImgLib2 data structure is preferred to
back the image data: Array[1], Planar[2], or Cell[3].

Cell is the virtual option that reads planes on demand from disk (or
cache). Array is faster but historically some operations could only operate
on PlanarImgs.

You might find the ImgLib2 examples[4] helpful, as well.

[1]
https://github.com/imglib/imglib2/blob/imglib2-2.2.1/src/main/java/net/imglib2/img/array/ArrayImg.java
[2]
https://github.com/imglib/imglib2/blob/imglib2-2.2.1/src/main/java/net/imglib2/img/planar/PlanarImg.java
[3]
https://github.com/scifio/scifio/blob/scifio-0.21.1/src/main/java/io/scif/img/cell/SCIFIOCellImg.java
[4] http://fiji.sc/ImgLib2_Examples

On Fri, Mar 13, 2015 at 1:54 PM, Hadrien Mary <hadrien.mary at gmail.com>
wrote:

> Thank you Mark for the answer.
>
> About ImgMode I wonder what is the difference between these three modes :
>
> - ArrayImgFactory
> - CellImgFactory
> - PlanarImgFactory
>
> Sorry if it is something obvious but I often see these modes without
> knowing how they work.
>
> Best,
>
> --
> Hadrien Mary
>
> Ph.D student in Biology
> Tournier-Gachet Team
> CNRS - LBCMCP - UMR 5088
>
> Université de Toulouse - Bât. 4R3B1
> 118, route de Narbonne - 31062 Toulouse
>
>
> On Fri, Mar 13, 2015 at 7:43 PM, Mark Hiner <hiner at wisc.edu> wrote:
> > Hi Hadrien,
> >
> >>My hypothesis is that when loading an image, Fiji/SCIFIO look at free
> >>memory in the system but not in the JVM. Is that possible ?
> >
> > Yes, this is almost certainly what's happening. Unfortunately I am not
> aware
> > of a way to differentiate between memory that is still in use, and memory
> > that just hasn't been deallocated.
> >
> > By default in these situations SCIFIO will look at the available memory,
> see
> > that there's not enough to open the dataset and open it virtually. Note
> that
> > it actually would be possible to open the full dataset - as the memory
> > allocation approached the JVM's limit, it would automatically be
> released.
> >
> > Right now this can be worked around programmatically by specifying a
> PLANAR
> > or ARRAY[1] ImgMode in the SCIFIOConfig[2]. The Bio-Formats importer lets
> > you specify whether the image should be opened virtually or not. The only
> > other option is to click the status bar to force garbage collection.
> >
> > We also have plans to improve the SCIFIO importer[3] to expose these
> > settings in the UI. Ultimately the hope is that virtual (cell) images
> will
> > incur minimal performance penalties though[4], making this issue moot.
> >
> > Thanks!
> > - Mark
> >
> > [1]
> >
> https://github.com/scifio/scifio/blob/scifio-0.21.1/src/main/java/io/scif/config/SCIFIOConfig.java#L124
> > [2]
> >
> https://github.com/scifio/scifio/blob/scifio-0.21.1/src/main/java/io/scif/config/SCIFIOConfig.java#L370-373
> > [3] https://github.com/imagej/imagej-plugins-commands/issues/4
> > [4] https://github.com/scifio/scifio/issues/201
> >
> >
> > On Mon, Jan 26, 2015 at 5:57 AM, Hadrien Mary <hadrien.mary at gmail.com>
> > wrote:
> >>
> >> Hi,
> >>
> >> New SCIFIO I/O system in Fiji/IJ2 automatically detects wether virtual
> >> stack is needed according to system memory available and image size
> >> (if I understood correctly).
> >>
> >> Virtual stack is fine but it always better to work without when you
> >> can. For example working with timelapse stacks, sliding time is much
> >> faster when image is not a virtual stack (and it makes sense since RAM
> >> memory access is much faster than hard drive memory access...).
> >>
> >> Saying that I noticed something: when I close a big image (no virtual
> >> stack because moving time slider is fast), java does not release
> >> memory to system (and it's a normal behaviour according to this
> >>
> >>
> http://stackoverflow.com/questions/22912063/automatically-release-unused-memory-in-imagej-fiji
> ).
> >> Now when I load the same kind of image (same size), the JVM should use
> >> the free memory released by the previously closed image isn't it ? I
> >> have the feeling that it does not because, when I move the time slider
> >> on the second image, it's very slow....
> >>
> >> Some numbers:
> >>
> >> - image size : 4.6GB
> >> - Fiji memory after start : 0.3GB
> >> - Fiji memory after loading first image: 6.9GiB
> >> - Fiji memory after closing first image: 6.9GiB
> >> - Fiji memory after loading second image: 6.9GiB
> >>
> >> My hypothesis is that when loading an image, Fiji/SCIFIO look at free
> >> memory in the system but not in the JVM. Is that possible ? Did you
> >> already noticed that ?
> >>
> >> PS: I have attached to this mail some properties of my system.
> >>
> >> Best,
> >>
> >> --
> >> Hadrien Mary
> >>
> >> Ph.D student in Biology
> >> Tournier-Gachet Team
> >> CNRS - LBCMCP - UMR 5088
> >>
> >> Université de Toulouse - Bât. 4R3B1
> >> 118, route de Narbonne - 31062 Toulouse
> >>
> >> _______________________________________________
> >> ImageJ-devel mailing list
> >> ImageJ-devel at imagej.net
> >> http://imagej.net/mailman/listinfo/imagej-devel
> >>
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://imagej.net/pipermail/imagej-devel/attachments/20150313/c4b5740f/attachment-0001.html>