<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hi all,<div><br><div>My suspicion is that there is no One True Solution™. So from my point of view it would be nice to have a way to support different options.</div><div><br></div><div>The recent projectors pull request <a href="https://github.com/imagej/imglib/pull/23">https://github.com/imagej/imglib/pull/23</a> by Michael Zinsmaier (KNIME) has potential to provide this extensibility.</div><div>Their <span style="color: rgb(51, 51, 51); font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 13px; line-height: 18px; white-space: nowrap; background-color: rgb(248, 238, 199); ">DimProjector2D</span> which is a possible replacement for net.imglib2.display.CompositeXYProjector uses a <span style="background-color: rgb(248, 238, 199); color: rgb(51, 51, 51); font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 13px; line-height: 18px; white-space: nowrap; ">final Converter< ProjectedDimSampler< A >, B ></span> to convert from a set of A-values in the "composite dimension" to the output B-value. There could be different converters for different alpha-compositing algorithms and it would be easy to add new options for imglib2 users.</div><div><br></div><div>The projectors branch / pull request requires some work to make it a replacement for the current projectors instead of opening up a parallel hierarchy. If someone wants to work on the compositing issues I think that would be a good place to direct efforts to.</div><div><br></div><div>best regards,</div><div>Tobias</div><div><br></div><div><br></div><div><div><div>On Jul 16, 2013, at 12:28 AM, Aivar Grislis <<a href="mailto:grislis@wisc.edu">grislis@wisc.edu</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">
<meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type">
<div bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">
<blockquote type="cite">I believe ImageJ1 treats it [RGBCMY] as
additive. Look at the sample "Organ of Corti" -- the current
behavior of ImageJ2 causes that sample to appear the same as it
does in IJ1. Before we added the bounds-checking code, it
erroneously wrapped pixel values.</blockquote>
By not being additive I meant C is a secondary color composed of
primaries G & B, etc. In the sense of
<a class="moz-txt-link-freetext" href="http://en.wikipedia.org/wiki/Additive_color">http://en.wikipedia.org/wiki/Additive_color</a> .<br>
<br>
Okay, "Organ of Corti" uses RGBK (and K is even worse than my
example of C since it has all three RGB components not just G
& B) and yet it works as an image. It's useful because the
areas lit up in each channel are fairly distinct. If these areas
overlapped the bounds-checking code would come into play in the
overlapping pixels and some highlights would get squashed and some
colors distorted (when one component is squashed but not the
others). But even if the code did a better job of combining the
colors of overlapping areas you'd still have visual ambiguity in
these areas (since eyes can't distinguish C from G + B). So now
I'm thinking the code works well as is.<br>
<blockquote type="cite">It was intended to be more general than
only the cases Aivar mentioned, and instead provided additive
support for *any* color table per channel you throw at it, the
same as ImageJ1's CompositeImages do.</blockquote>
Sure, it shouldn't crash and burn if you put Fire on one channel
and Ice on another but that's not usable visually unless the areas
lit up in each channel are distinct. If you have a lot of overlap
and you want the colors to add up meaningfully you're better off
sticking with primary additive colors for your channel LUTs.<br>
<br>
On 7/15/13 3:53 PM, Curtis Rueden wrote:<br>
</div>
<blockquote cite="mid:CADN69yk7Dh6esy+AfEUeyYtySbETtw0rJUUMho9a3q8DH5V-VA@mail.gmail.com" type="cite">
<div dir="ltr">Hi all,
<div><br>
</div>
<div>
<div>> the bigger issue is RGBCMY is not an additive color
system.</div>
</div>
<div><br>
</div>
<div style="">I believe ImageJ1 treats it as additive. Look at
the sample "Organ of Corti" -- the current behavior of ImageJ2
causes that sample to appear the same as it does in IJ1.
Before we added the bounds-checking code, it erroneously
wrapped pixel values.</div>
<div style=""><br>
</div>
<div style="">As for the alpha stuff, I will try to digest and
reply soon but I am way too tired at this moment. I just
wanted to clarify why the code is the way it is. It was
intended to be more general than only the cases Aivar
mentioned, and instead provided additive support for *any*
color table per channel you throw at it, the same as ImageJ1's
CompositeImages do.</div>
<div style=""><br>
</div>
<div style="">Regards,</div>
<div style="">Curtis</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Mon, Jul 15, 2013 at 3:46 PM, Aivar
Grislis <span dir="ltr"><<a moz-do-not-send="true" href="mailto:grislis@wisc.edu" target="_blank">grislis@wisc.edu</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex; position: static; z-index: auto; ">
<div bgcolor="#FFFFFF" text="#000000">
<div>I think CompositeXYProjector is meant to handle the
following cases:<br>
<br>
1) Rendering LUT images, a single converter is used.
Grayscale images are included here.<br>
<br>
2) Rendering RGB images, three converters are used.
These use red-only, green-only, and blue-only LUTs.<br>
<br>
3) I believe it's also intended to work with images with
> 3 channels, using C, M, and Y for the excess
channels.<br>
<br>
The existing code works well for cases 1 & 2. Case
3 adds the possibility of overflow, if your red
converter gives you a value of 255 for the red component
but your magenta converter adds another 255. Currently
the code just limits the value to 255 in that case.
Some sort of blending might work better here, but the
bigger issue is RGBCMY is not an additive color system.
If you see a cyan blotch you don't know if its in both
the G & B channels or just the C channel.<span class="HOEnZb"><font color="#888888"><br>
<br>
Aivar</font></span>
<div>
<div class="h5"><br>
<br>
<br>
On 7/15/13 2:40 PM, Lee Kamentsky wrote:<br>
</div>
</div>
</div>
<div>
<div class="h5">
<blockquote type="cite">
<div dir="ltr">Thanks for answering Aivar,
<div>
<div class="gmail_extra"><br>
</div>
<div class="gmail_extra">I think what your reply
did for me is to have me take a step back and
consider what we're modeling. If you look at
my replies below, I think that the best
solution is to use a model where the
background is white and each successive layer
filters out some of that background, like a
gel. A layer attenuates the underlying layer
by a fraction of (1 - alpha/255 * (1 -
red/255)), resulting in no attenuation for 255
and attenuation of alpha/255 for zero. We can
then use a red converter that returns a value
of 255 for the blue and green channels and the
model and math work correctly.</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon, Jul 15, 2013
at 1:59 PM, Aivar Grislis <span dir="ltr"><<a moz-do-not-send="true" href="mailto:grislis@wisc.edu" target="_blank">grislis@wisc.edu</a>></span>
wrote:<br>
<blockquote class="gmail_quote">
<div>
<div>
<blockquote type="cite">I have an
ImgPlus backed by an RGB PlanarImg
of UnsignedByteType and
ARGBType.alpha(value) is 255 for all
of them, so aSum is 765. It would
appear that the correct solution
would be to divide aSum by 3.</blockquote>
Isn't it unusual to define an alpha
for each color component, generally
you have a single A associated with a
combined RGB? So averaging the three
alphas might make sense here, because
I think they should all be the same
value.<br>
</div>
</div>
</blockquote>
<div>I think you're right, the model always
is that each pixel has an alpha value that
applies to R, G and B. The image I was
using was the Clown example image.
DefaultDatasetView.initializeView
constructs three RealLUTConverters for the
projector, one for red, one for green and
one for blue which sends you down this
rabbit hole.</div>
<blockquote class="gmail_quote">
<div>
<div>
<blockquote type="cite">In addition,
there's no scaling of the individual
red, green and blue values by their
channel's alpha. If the input were
two index-color images, each of
which had different alphas, the code
should multiply the r, g and b
values by the alphas before summing
and then divide by the total alpha
in the end. The alpha in this case
*should* be the sum of alphas
divided by the number of channels.</blockquote>
I think alpha processing is more
cumulative, done layer by layer in
some defined layer order. For a given
pixel say the current output pixel
value is ARGB1 and you are compositing
a second image with value ARGB2 on top
of it: For the red channel the output
color should be ((255 - alpha(ARGB2))
* red(ARGB1) + alpha(ARGB2) *
red(ARGB2)) / 255. The alpha of ARGB1
is not involved.<br>
</div>
</div>
</blockquote>
<div>I think that's a valid interpretation.
I've always used (alpha(ARGB1) *
red(ARGB1) + alpha(ARGB2) * red(ARGB2)) /
(alpha(ARGB1) + alpha(ARGB2)) because I
assumed the alpha indicated the</div>
<div>strength of the blending of each
source. In any case, the code as it stands
doesn't do either of these.</div>
<blockquote class="gmail_quote">
<div>
<div> <br>
In other words, if you add a layer
that is completely opaque you no
longer have to consider any of the
colors or alpha values underneath it. <br>
</div>
</div>
</blockquote>
<blockquote class="gmail_quote">
<div>
<div> <br>
I think the bigger issue here is this
code is specifically designed to
composite red, green and blue image
layers. It's a special case since for
a given pixel the red comes from the
red layer, blue from blue layer, and
green from green layer. These layers
shouldn't be completely opaque, since
the colors wouldn't combine at all
then or completely transparent since
then they wouldn't contribute any
color. I don't think transparency is
useful here.<br>
</div>
</div>
</blockquote>
<div>So this is an argument for blending
instead of layering - transparency would
be useful if the images were blended and
treated as if on a par with each other,
allowing the user to emphasize one channel
or the other. </div>
<blockquote class="gmail_quote">
<div>
<div> <br>
It's also possible that a multichannel
image with > 3 channels is being
displayed with more color channels,
namely cyan, magenta, and yellow. The
code here is designed to stop
overflow, but I'm not convinced those
extended color channels would combine
meaningfully.<br>
<br>
Aivar<br>
<br>
<blockquote type="cite">In addition,
there's no scaling of the individual
red, green and blue values by their
channel's alpha. If the input were
two index-color images, each of
which had different alphas, the code
should multiply the r, g and b
values by the alphas before summing
and then divide by the total alpha
in the end. The alpha in this case
*should* be the sum of alphas
divided by the number of channels.</blockquote>
I think alpha processing is cumulative
layer by layer. <br>
<br>
This brings up some interesting
questions:<br>
<br>
1) If the first, bottom-most layer is
transparent, what color should show
through? Black, white? Or perhaps
it's best to ignore this base layer
transparency.<br>
</div>
</div>
</blockquote>
<div>Maybe the model should be that the
background is white and successive layers
are like gel filters on top. In that case,
you'd have:</div>
<div> red = (255 - alpha(ARGB2) *(255 -
red(ARGB2))/255) * red(ARGB1) </div>
<div><br>
</div>
<div>And maybe that points to what the true
solution is. For the default, we could
change things so that red channel would
have blue = 255 and green = 255 and the
first composition would change only the
red channel.</div>
<blockquote class="gmail_quote">
<div>
<div> <br>
2) If you wanted to composite several
transparent images, how do you
calculate the transparency of the
composite? I'm not sure this is
something we need to do.<br>
<br>
Aivar<br>
<br>
<br>
On 7/15/13 10:31 AM, Lee Kamentsky
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>Hi all, </div>
<div>I'm looking at the code for
net.imglib2.display.CompositeXYProjector
and as I step through it, it's
clear that the alpha calculation
isn't being handled correctly.
Here's the code as it stands now,
line 190 roughly:</div>
<div><br>
</div>
<div><span> </span>for ( int i = 0;
i < size; i++ )</div>
<div><span> </span>{</div>
<div><span> </span>sourceRandomAccess.setPosition(
currentPositions[ i ], dimIndex );</div>
<div><span> </span>currentConverters[
i ].convert(
sourceRandomAccess.get(), bi );</div>
<div><span> </span>// accumulate
converted result</div>
<div><span> </span>final int value
= bi.get();</div>
<div><span> </span>final int a =
ARGBType.alpha( value );</div>
<div><span> </span>final int r =
ARGBType.red( value );</div>
<div><span> </span>final int g =
ARGBType.green( value );</div>
<div><span> </span>final int b =
ARGBType.blue( value );</div>
<div><span> </span>aSum += a;</div>
<div><span> </span>rSum += r;</div>
<div><span> </span>gSum += g;</div>
<div><span> </span>bSum += b;</div>
<div><span> </span>}</div>
<div><span> </span>if ( aSum >
255 )</div>
<div><span> </span>aSum = 255;</div>
<div><span> </span>if ( rSum >
255 )</div>
<div><span> </span>rSum = 255;</div>
<div> <span> </span>if ( gSum >
255 )</div>
<div><span> </span>gSum = 255;</div>
<div><span> </span>if ( bSum >
255 )</div>
<div><span> </span>bSum = 255;</div>
<div><span> </span>targetCursor.get().set(
ARGBType.rgba( rSum, gSum, bSum,
aSum ) );</div>
<div><br>
</div>
<div>I have an ImgPlus backed by an
RGB PlanarImg of UnsignedByteType
and ARGBType.alpha(value) is 255
for all of them, so aSum is 765.
It would appear that the correct
solution would be to divide aSum
by 3. In addition, there's no
scaling of the individual red,
green and blue values by their
channel's alpha. If the input were
two index-color images, each of
which had different alphas, the
code should multiply the r, g and
b values by the alphas before
summing and then divide by the
total alpha in the end. The alpha
in this case *should* be the sum
of alphas divided by the number of
channels.</div>
<div><br>
</div>
<div>However, I think the problem is
deeper than that. For an RGB
ImgPlus, there are three LUTs and
each of them has an alpha of 255,
but that alpha only applies to one
of the colors in the LUT. When
you're compositing images and
weighing them equally, if two are
black and one is white, then the
result is 1/3 of the white
intensity - if you translate that
to red, green and blue images, the
resulting intensity will be 1/3 of
that desired. This might sound
weird, but the only solution that
works out mathematically is for
the defaultLUTs in the
DefaultDatasetView to use color
tables that return values that are
3x those of ColorTables.RED, GREEN
and BLUE. Thinking about it, I'm
afraid this *is* the correct model
and each channel really is 3x
brighter than possible.</div>
<div><br>
</div>
<div>It took me quite a bit of back
and forth to come up with the
above... I hope you all understand
what I'm saying and understand the
problem and counter-intuitive
solution and have the patience to
follow it. Dscho, if you made it
this far - you're the
mathematician, what's your take?</div>
<div><br>
</div>
<div>--Lee</div>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
ImageJ-devel mailing list
<a moz-do-not-send="true" href="mailto:ImageJ-devel@imagej.net" target="_blank">ImageJ-devel@imagej.net</a>
<a moz-do-not-send="true" href="http://imagej.net/mailman/listinfo/imagej-devel" target="_blank">http://imagej.net/mailman/listinfo/imagej-devel</a>
</pre>
</blockquote>
<br>
</div>
<br>
_______________________________________________<br>
ImageJ-devel mailing list<br>
<a moz-do-not-send="true" href="mailto:ImageJ-devel@imagej.net" target="_blank">ImageJ-devel@imagej.net</a><br>
<a moz-do-not-send="true" href="http://imagej.net/mailman/listinfo/imagej-devel" target="_blank">http://imagej.net/mailman/listinfo/imagej-devel</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</blockquote>
<br>
</div>
</div>
</div>
<br>
_______________________________________________<br>
ImageJ-devel mailing list<br>
<a moz-do-not-send="true" href="mailto:ImageJ-devel@imagej.net">ImageJ-devel@imagej.net</a><br>
<a moz-do-not-send="true" href="http://imagej.net/mailman/listinfo/imagej-devel" target="_blank">http://imagej.net/mailman/listinfo/imagej-devel</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div>
_______________________________________________<br>ImageJ-devel mailing list<br><a href="mailto:ImageJ-devel@imagej.net">ImageJ-devel@imagej.net</a><br>http://imagej.net/mailman/listinfo/imagej-devel<br></blockquote></div><br></div></div></body></html>