<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=ISO-8859-15">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px;
margin-left: 0px;"><span style="font-family:'Helvetica';
font-size:medium; color:rgba(0, 0, 0, 1.0);">Hi Aivar,<br>
<br>
<font face="sans-serif">some months ago we had a similar
discussion with Barry and Curtis (see below). Sorry for not
posting this on the mailing list. <br>
From our (KNIME) point of view, an algorithm, which is
implemented in ImgLib2, shouldn't know about any Axes / Axes
Names.</font></span><br>
The algorithm just knows on which number of dimensions (2-d, 3-d,
... n-D) it could run. ImgLib2 Views can be used to pre-process
the algorithm input (e.g. 2D Slices of an 3-D Img).<br>
For Example: You have a 4-dimensional Image and you want to
process it plane-wise (2D, also possible in 3-D ... n-D just in
the example: 2D). In KNIME we use a Component called "Dim
Selection" in the GUI where a user selects e.g. X and Y to define
the planes of interest. All other dimensions are used to iterate
over the x-y-planes (e.g. if t=2, c=3 then we iterate over 6
x-y-planes). After we determined the <span class="st">indices</span>
of the X-Y Axes of our incoming Image (using the Axes in ImgPlus)
we internally create several (nested) ImgLib2 Views using
Views.interval, Views.translate etc pp. The resulting views are
passed to the algorithm as a RandomAccessible,
RandomAccessibleInterval or (if necessary) an ImgView/ImgPlusView
(wrapping a RandomAccessibleInterval).<br>
<br>
Using Views we don't restrict the algorithms such that they only
work on dedicated dimensions. In fact, any user-selection of
dimensions can be used to create a View (as long as the algorithms
can handle the number of dimensions of the resulting view). We
suggested a general "Dimension Selection" Service for ImageJ2, but
unfortunately we didn't find the time yet to implement it. <br>
<br>
Looking forward for replies ;)<br>
<br>
Martin, Michael Z. and Christian<br>
<div class="moz-text-html" lang="x-western"><br>
<div class="moz-forward-container"> <br>
-------- Original Message --------
<table class="moz-email-headers-table" border="0"
cellpadding="0" cellspacing="0">
<tbody>
<tr>
<th align="RIGHT" nowrap="nowrap" valign="BASELINE">Subject:
</th>
<td>Re: Processing sub-intervals in ImageJ-Plugins</td>
</tr>
<tr>
<th align="RIGHT" nowrap="nowrap" valign="BASELINE">Date:
</th>
<td>Thu, 31 May 2012 09:06:30 +0200</td>
</tr>
<tr>
<th align="RIGHT" nowrap="nowrap" valign="BASELINE">From:
</th>
<td>Martin Horn <a class="moz-txt-link-rfc2396E"
href="mailto:Martin.Horn@uni-konstanz.de"><Martin.Horn@uni-konstanz.de></a></td>
</tr>
<tr>
<th align="RIGHT" nowrap="nowrap" valign="BASELINE">To:
</th>
<td><a class="moz-txt-link-abbreviated"
href="mailto:bdezonia@gmail.com">bdezonia@gmail.com</a></td>
</tr>
<tr>
<th align="RIGHT" nowrap="nowrap" valign="BASELINE">CC:
</th>
<td>Christian Dietz <a class="moz-txt-link-rfc2396E"
href="mailto:christian.dietz@uni-konstanz.de"><christian.dietz@uni-konstanz.de></a>,
<a class="moz-txt-link-abbreviated"
href="mailto:michael.zinsmaier@gmx.de">michael.zinsmaier@gmx.de</a>,
Curtis Rueden <a class="moz-txt-link-rfc2396E"
href="mailto:ctrueden@wisc.edu"><ctrueden@wisc.edu></a></td>
</tr>
</tbody>
</table>
<br>
<br>
<pre>Hi Barry,
maybe my last Mail was a bit hasty and we reordered our thoughts a bit
;o) Sorry for that. So forget about the part of "processing very large
image objects". But the comments on the Normalize plugin (or rather the
Auto Contrast plugins) are still true. To be clear, let me explain
again, what we wanted to address by means of the "AutoContrast"-plugin:
The AutoContrast plugin first calculates a histogram and then applies a
normalization function determined from the histogram to each pixel. Now,
this histogram, e.g. can be determined on various pixel-(sub)sets of the
image in a n-dimensional image object: one can calculate the histogram
for all pixels, one do it for each XY-plane individually, one can to it
for each XT-plane (if T existent), one can do it for each XYC-cube, if
we consider a 5-dim image with XYCT, and so on (the same is true, e.g.
for Thresholding, CCA, ... . Hence, there are various way how to handle
n-dim data properly (depending on the algorithm itself, whether it can
handle only 2D, 3D or nD data). Certainly, one can define that each
plugin has to care about how to handle nD data in a appropriate way by
itself. Thats fine, actually. But what would be desirable is maybe the
existence of some convenience-tools to simplify the n-D data handling
(e.g. think of a service which ask the user to select the dimensions to
operate on and whose to iterate; or maybe a component in a dialog and
some helper classes, ...). This would for example make the
AutoContrast-implementation more general and hardcoded things like
"axes.getLabel()=="Channel" (line207) (the user decides, what a channel
is) or the question "how to handle RGB images" (line 156) would be
lapsed. Currently the AutoContrast-plugin only processes a nD-image on
the whole.
Maybe we can give you an example how we handle it (which is certainly
not perfect, but illustrates the idea somehow):
Given: Image in 5D (XYCZT).
Let's assume an algorithm is applicable only in 2D (Maybe find edges) or
the results differ if we use 2D or 3D representation of an image
(AutoContrast, Thresholding etc), as mentioned above.
In any case the user may select what his process planes are by selecting
the labels of this planes (e.g. XY, XT, XC, CT, etc., could also be a
N-Dimensional selection, depending on the algorithm). The algorithm then
is applied to any selected plane separately. This is possible as we use
the Operation concept which we defined together with you guys in
Dresden. For the given 5d input we create a proper output (e.g. a 5D
BitType image in the case of Thresholding) and then we create several
views depending on the dimension selection of the user on the input and
the output. The algorithm is called as we provide it as well the input
as the output container before hands. Remember we do not duplicate
memory or something as we use Views on the input and output data (e.g.
one plane).
We hope, we could express ourself more clear.
Thanks a lot!
Best,
Christian and Martin
On 05/30/2012 05:07 PM, Barry DeZonia wrote:
> Hi Martin,
>
> PointSets might help you with this problem. The net.imglib2.ops.pointset
> package defines a set of PointSets that can be used to represent a
> subset of an n-dimensional dataset. For instance you can define a point
> set as a plane in 3d space (XYT). You iterate the PointSet to pull out
> each coordinate within it and use that coordinate for your computations.
> Then you can relocate the plane of interest via pointset.setAnchor() and
> changing the time component of the plane origin. This would allow code
> to iterate planes of a bigger space.
>
> PointSets are a general construct that allow arbitrarily defined regions
> to be iterated and "slid" over a n-dimensional space. Views might work
> in this instance too. I'm not sure. But in your example of a big image
> the PointSet could be an arbitrarily small region within it that is
> repeatedly slid between calculation steps. In OPS the sliding is done by
> the various classes in net.imglib2.ops.input. For instance a
> PointSetInputIterator.
>
> I may not be understanding your problem here though. I'm not sure how a
> normalizer plugin or contrast adjuster plugin exceeds the problem you
> mentioned of arbitrary subregion iteration. Can you be more specific?
> What general operation or general plugin do you envision? Hopefully my
> response here has not been off target. Let me know if there is more to
> the problem that I am missing.
>
> Regards,
>
> Barry
>
> On Wed, May 30, 2012 at 8:03 AM, Martin Horn
> <<a class="moz-txt-link-abbreviated" href="mailto:Martin.Horn@uni-konstanz.de">Martin.Horn@uni-konstanz.de</a> <a class="moz-txt-link-rfc2396E" href="mailto:Martin.Horn@uni-konstanz.de"><mailto:Martin.Horn@uni-konstanz.de></a>> wrote:
>
> Hi Curtis and Barry,
>
> we, Christian, Michael and I, are currently discussing how to better
> realize something we call "dimension selection" especially in
> conjunction with the integration of ImageJ plugins. Imagine a video
> which you want to normalize. You can either do it for the whole
> image object or you can normalize for instance each plane
> individually (even without copying the planes). So far, in some of
> our KNIME nodes we introduced a special dialog component ("dimension
> selection") allowing the user to specify how to proceed in a
> particular case.
>
> Do you already thought of this kind of things? How are you planing
> to handle this issue for instance in a "Normalizer/Contrast Enhancer
> plugin"?
>
> This issue is probably also related to the problem of processing
> very large image objects, which don't fit into the memory. In those
> cases only sub-intervals of an image can be processed at once but
> the result should still be regarded as one image object.
>
> Certainly, each plugin can take care of this by its own. But one may
> also think of a dedicated kind of ImageJPlugin which is e.g.
> similarly defined as a net.imglib2.ops.__UnaryOperation.
>
> What are your ideas??
>
> Thanks a lot!
>
> Best,
>
> Martin
>
>
>
>
> --
> Martin Horn
> Nycomed Chair for Bioinformatics
> and Information Mining
> University of Konstanz
>
> fax: +49 (0)7531 88-5132 <tel:%2B49%20%280%297531%2088-5132>
> phone: +49 (0)7531 88-5017 <tel:%2B49%20%280%297531%2088-5017>
>
>
>
--
Martin Horn
Nycomed Chair for Bioinformatics
and Information Mining
University of Konstanz
fax: +49 (0)7531 88-5132
phone: +49 (0)7531 88-5017
</pre>
<br>
</div>
<br>
</div>
<span style="font-family:'Helvetica'; font-size:medium;
color:rgba(0, 0, 0, 1.0);"><b>From: </b></span><span
style="font-family:'Helvetica'; font-size:medium;">Aivar Grislis
<<a href="mailto:grislis@wisc.edu">grislis@wisc.edu</a>><br>
</span></div>
<div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px;
margin-left: 0px;"><span style="font-family:'Helvetica';
font-size:medium; color:rgba(0, 0, 0, 1.0);"><b>Subject: </b></span><span
style="font-family:'Helvetica'; font-size:medium;"><b>[ImageJ-devel]
ImgLib2 dimensionality</b><br>
</span></div>
<div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px;
margin-left: 0px;"><span style="font-family:'Helvetica';
font-size:medium; color:rgba(0, 0, 0, 1.0);"><b>Date: </b></span><span
style="font-family:'Helvetica'; font-size:medium;">January 10,
2013 10:26:06 PM GMT+01:00<br>
</span></div>
<div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px;
margin-left: 0px;"><span style="font-family:'Helvetica';
font-size:medium; color:rgba(0, 0, 0, 1.0);"><b>To: </b></span><span
style="font-family:'Helvetica'; font-size:medium;"><a
href="mailto:imagej-devel@imagej.net">imagej-devel@imagej.net</a><br>
</span></div>
<br>
I was looking at the dimensional code in net.imglib2.meta.Axes. This
is an enum of known, named AxisTypes such as X, Y, Z..., has the
capacity for CustomAxisTypes, and some sort of characterization of
these dimensions with 'isXY()' and 'isSpatial()'. (Perhaps we could
replace these latter with some kind of EnumSet of DimensionTypes,
such as XY, SPATIAL, etc. for the discussion that follows.)<br>
<br>
It might be useful to have a concept of a COINCIDENTAL type. This
sort of dimension represents different aspects of the same
pixel/voxel/doxel. It's as if you look at it with red glasses, blue
glasses, etc. Effectively this is the same as non-spatial/-temporal
since all of the default AxisTypes besides XYZT would be
coincidental. But with this you can specify if a new CustomAxisType
should be coincidental or not. For example, if you wrote a plugin
that works on an image in 5D that is a series of 4D (XYZT) images,
the dimension that represents the series of images is not
coincidental (and not spatial or temporal either).<br>
<br>
Another useful concept would be whether a dimension is DISCRETE (or
NON_INTERPOLABLE?). This sort of image would be analogous to a
table of height/weight/shoe size per individual (pixel). This works
only because it happens you can express all three measurements as
floating point. Anyway, the point is these measurements are totally
independent, the order is arbitrary, and it never would make any
sense to combine them somehow or interpolate between them. I've
been working on a FLIM fitting plugin and using such images as my
output, where the measurements are a set of fitted lifetime
parameters, for example A1, T1, A2, T2, Z, all doubles. It's the
equivalent of five separate images in one and could just be
refactored as such. [*]<br>
<br>
My motivating concern was that if were to introduce such a DISCRETE
DimensionType most existing plugins or algorithms will not do
anything useful (other than simple things like copying, etc.). In
fact they will most likely clobber my discrete dimension.<br>
<br>
What if an algorithm could somehow declare via annotation what sorts
of dimensions it was interested in? Without such an annotation the
default would be to get all of the non-discrete dimensions, and the
caller splits the image into sub images across discrete dimensions
and processes them successively. Then existing algorithms could do
useful work on my discrete image. Other algorithms that understand
the meaning of this discrete dimension information could declare an
interest in it and get the whole thing.<br>
<br>
This could be a very useful mechanism in other instances. In
general lets say an algorithm could specify dimension types or
specific dimensions of interest and the caller splits the image into
sub images across the remaining dimensions. Then the algorithm gets
a cursor that iterates only on the dimensions of interest. They
could set the cursor position within these dimensions or just
iterate through the whole thing.<br>
<br>
Another use case might be to split up arbitrary images into XY
slices for 2D-processing algorithms. On the input side, my FLIM
fitting algorithm could declare an interest in the LIFETIME AxisType
only and get called pixel by pixel with a lifetime dimension cursor
to process a whole image that has unknown dimensions.<br>
<br>
For another example, the early ImgLib1 sample Floyd-Steinberg
dithering algorithm could process only non-COINCIDENTAL dimensions.
This would avoid distributing errors from the red channel to the
blue channel, for instance; they would be dithered one by one.<br>
<br>
Hope this makes sense! There's a lot of hand-waving here at
present, particularly how this dimensional interest would be
specified.<br>
<br>
Thanks for reading,<br>
<br>
Aivar Grislis<br>
<br>
([*] Another alternative would be my creating a custom
FittedLifetimeType or some kind of generic TupleType)<br>
</body>
</html>