[ImageJ-devel] Bug in creation of CompositeXYProjectors in DefaultDatasetView

Tue Jul 16 06:57:36 CDT 2013

OK - it's up to you all. If it's doing what you think it should, I'm fine
with it.

On Mon, Jul 15, 2013 at 6:28 PM, Aivar Grislis <grislis at wisc.edu> wrote:

>  I believe ImageJ1 treats it [RGBCMY] as additive. Look at the sample
> "Organ of Corti" -- the current behavior of ImageJ2 causes that sample to
> appear the same as it does in IJ1. Before we added the bounds-checking
> code, it erroneously wrapped pixel values.
>
> By not being additive I meant C is a secondary color composed of primaries
> G & B, etc.  In the sense of http://en.wikipedia.org/wiki/Additive_color .
>
> Okay, "Organ of Corti" uses RGBK (and K is even worse than my example of C
> since it has all three RGB components not just G & B) and yet it works as
> an image.  It's useful because the areas lit up in each channel are fairly
> distinct.  If these areas overlapped the bounds-checking code would come
> into play in the overlapping pixels and some highlights would get squashed
> and some colors distorted (when one component is squashed but not the
> others).  But even if the code did a better job of combining the colors of
> overlapping areas you'd still have visual ambiguity in these areas (since
> eyes can't distinguish C from G + B).  So now I'm thinking the code works
> well as is.
>
> It was intended to be more general than only the cases Aivar mentioned,
> and instead provided additive support for *any* color table per channel you
> throw at it, the same as ImageJ1's CompositeImages do.
>
> Sure, it shouldn't crash and burn if you put Fire on one channel and Ice
> on another but that's not usable visually unless the areas lit up in each
> channel are distinct.  If you have a lot of overlap and you want the colors
> to add up meaningfully you're better off sticking with primary additive
> colors for your channel LUTs.
>
> On 7/15/13 3:53 PM, Curtis Rueden wrote:
>
> Hi all,
>
>  > the bigger issue is RGBCMY is not an additive color system.
>
>  I believe ImageJ1 treats it as additive. Look at the sample "Organ of
> Corti" -- the current behavior of ImageJ2 causes that sample to appear the
> same as it does in IJ1. Before we added the bounds-checking code, it
> erroneously wrapped pixel values.
>
>  As for the alpha stuff, I will try to digest and reply soon but I am way
> too tired at this moment. I just wanted to clarify why the code is the way
> it is. It was intended to be more general than only the cases Aivar
> mentioned, and instead provided additive support for *any* color table per
> channel you throw at it, the same as ImageJ1's CompositeImages do.
>
>  Regards,
> Curtis
>
>
> On Mon, Jul 15, 2013 at 3:46 PM, Aivar Grislis <grislis at wisc.edu> wrote:
>
>>  I think CompositeXYProjector is meant to handle the following cases:
>>
>> 1) Rendering LUT images, a single converter is used.  Grayscale images
>> are included here.
>>
>> 2) Rendering RGB images, three converters are used.  These use red-only,
>> green-only, and blue-only LUTs.
>>
>> 3) I believe it's also intended to work with images with > 3 channels,
>> using C, M, and Y for the excess channels.
>>
>> The existing code works well for cases 1 & 2.  Case 3 adds the
>> possibility of overflow, if your red converter gives you a value of 255 for
>> the red component but your magenta converter adds another 255.  Currently
>> the code just limits the value to 255 in that case.  Some sort of blending
>> might work better here, but the bigger issue is RGBCMY is not an additive
>> color system.  If you see a cyan blotch you don't know if its in both the G
>> & B channels or just the C channel.
>>
>> Aivar
>>
>>
>>
>> On 7/15/13 2:40 PM, Lee Kamentsky wrote:
>>
>> Thanks for answering Aivar,
>>
>>  I think what your reply did for me is to have me take a step back and
>> consider what we're modeling. If you look at my replies below, I think that
>> the best solution is to use a model where the background is white and each
>> successive layer filters out some of that background, like a gel. A layer
>> attenuates the underlying layer by a fraction of (1 - alpha/255 * (1 -
>> red/255)), resulting in no attenuation for 255 and attenuation of alpha/255
>> for zero. We can then use a red converter that returns a value of 255 for
>> the blue and green channels and the model and math work correctly.
>>
>> On Mon, Jul 15, 2013 at 1:59 PM, Aivar Grislis <grislis at wisc.edu> wrote:
>>
>>>  I have an ImgPlus backed by an RGB PlanarImg of UnsignedByteType and
>>> ARGBType.alpha(value) is 255 for all of them, so aSum is 765. It would
>>> appear that the correct solution would be to divide aSum by 3.
>>>
>>> Isn't it unusual to define an alpha for each color component, generally
>>> you have a single A associated with a combined RGB?  So averaging the three
>>> alphas might make sense here, because I think they should all be the same
>>> value.
>>>
>> I think you're right, the model always is that each pixel has an alpha
>> value that applies to R, G and B. The image I was using was the Clown
>> example image. DefaultDatasetView.initializeView constructs three
>> RealLUTConverters for the projector, one for red, one for green and one for
>> blue which sends you down this rabbit hole.
>>
>>>  In addition, there's no scaling of the individual red, green and blue
>>> values by their channel's alpha. If the input were two index-color images,
>>> each of which had different alphas, the code should multiply the r, g and b
>>> values by the alphas before summing and then divide by the total alpha in
>>> the end. The alpha in this case *should* be the sum of alphas divided by
>>> the number of channels.
>>>
>>> I think alpha processing is more cumulative, done layer by layer in some
>>> defined layer order.  For a given pixel say the current output pixel value
>>> is ARGB1 and you are compositing a second image with value ARGB2 on top of
>>> it:  For the red channel the output color should be ((255 - alpha(ARGB2)) *
>>> red(ARGB1) + alpha(ARGB2) * red(ARGB2)) / 255.  The alpha of ARGB1 is not
>>> involved.
>>>
>> I think that's a valid interpretation. I've always used (alpha(ARGB1) *
>> red(ARGB1) + alpha(ARGB2) * red(ARGB2)) / (alpha(ARGB1) + alpha(ARGB2))
>> because I assumed the alpha indicated the
>> strength of the blending of each source. In any case, the code as it
>> stands doesn't do either of these.
>>
>>>
>>> In other words, if you add a layer that is completely opaque you no
>>> longer have to consider any of the colors or alpha values underneath it.
>>>
>>
>>> I think the bigger issue here is this code is specifically designed to
>>> composite red, green and blue image layers.  It's a special case since for
>>> a given pixel the red comes from the red layer, blue from blue layer, and
>>> green from green layer.  These layers shouldn't be completely opaque, since
>>> the colors wouldn't combine at all then or completely transparent since
>>> then they wouldn't contribute any color.  I don't think transparency is
>>> useful here.
>>>
>> So this is an argument for blending instead of layering - transparency
>> would be useful if the images were blended and treated as if on a par with
>> each other, allowing the user to emphasize one channel or the other.
>>
>>>
>>> It's also possible that a multichannel image with > 3 channels is being
>>> displayed with more color channels, namely cyan, magenta, and yellow.  The
>>> code here is designed to stop overflow, but I'm not convinced those
>>> extended color channels would combine meaningfully.
>>>
>>> Aivar
>>>
>>> In addition, there's no scaling of the individual red, green and blue
>>> values by their channel's alpha. If the input were two index-color images,
>>> each of which had different alphas, the code should multiply the r, g and b
>>> values by the alphas before summing and then divide by the total alpha in
>>> the end. The alpha in this case *should* be the sum of alphas divided by
>>> the number of channels.
>>>
>>> I think alpha processing is cumulative layer by layer.
>>>
>>> This brings up some interesting questions:
>>>
>>> 1) If the first, bottom-most layer is transparent, what color should
>>> show through?  Black, white?  Or perhaps it's best to ignore this base
>>> layer transparency.
>>>
>> Maybe the model should be that the background is white and successive
>> layers are like gel filters on top. In that case, you'd have:
>>  red = (255 - alpha(ARGB2) *(255 - red(ARGB2))/255) * red(ARGB1)
>>
>>  And maybe that points to what the true solution is. For the default, we
>> could change things so that red channel would have blue = 255 and green =
>> 255 and the first composition would change only the red channel.
>>
>>>
>>> 2) If you wanted to composite several transparent images, how do you
>>> calculate the transparency of the composite?  I'm not sure this is
>>> something we need to do.
>>>
>>> Aivar
>>>
>>>
>>> On 7/15/13 10:31 AM, Lee Kamentsky wrote:
>>>
>>>  Hi all,
>>> I'm looking at the code for net.imglib2.display.CompositeXYProjector and
>>> as I step through it, it's clear that the alpha calculation isn't being
>>> handled correctly. Here's the code as it stands now, line 190 roughly:
>>>
>>>  for ( int i = 0; i < size; i++ )
>>>  {
>>>  sourceRandomAccess.setPosition( currentPositions[ i ], dimIndex );
>>>  currentConverters[ i ].convert( sourceRandomAccess.get(), bi );
>>>  // accumulate converted result
>>>  final int value = bi.get();
>>>  final int a = ARGBType.alpha( value );
>>>  final int r = ARGBType.red( value );
>>>  final int g = ARGBType.green( value );
>>>  final int b = ARGBType.blue( value );
>>>  aSum += a;
>>>  rSum += r;
>>>  gSum += g;
>>>  bSum += b;
>>>  }
>>>  if ( aSum > 255 )
>>>  aSum = 255;
>>>  if ( rSum > 255 )
>>>  rSum = 255;
>>>  if ( gSum > 255 )
>>>  gSum = 255;
>>>  if ( bSum > 255 )
>>>  bSum = 255;
>>>  targetCursor.get().set( ARGBType.rgba( rSum, gSum, bSum, aSum ) );
>>>
>>>  I have an ImgPlus backed by an RGB PlanarImg of UnsignedByteType and
>>> ARGBType.alpha(value) is 255 for all of them, so aSum is 765. It would
>>> appear that the correct solution would be to divide aSum by 3. In addition,
>>> there's no scaling of the individual red, green and blue values by their
>>> channel's alpha. If the input were two index-color images, each of which
>>> had different alphas, the code should multiply the r, g and b values by the
>>> alphas before summing and then divide by the total alpha in the end. The
>>> alpha in this case *should* be the sum of alphas divided by the number of
>>> channels.
>>>
>>>  However, I think the problem is deeper than that. For an RGB ImgPlus,
>>> there are three LUTs and each of them has an alpha of 255, but that alpha
>>> only applies to one of the colors in the LUT. When you're compositing
>>> images and weighing them equally, if two are black and one is white, then
>>> the result is 1/3 of the white intensity - if you translate that to red,
>>> green and blue images, the resulting intensity will be 1/3 of that desired.
>>> This might sound weird, but the only solution that works out mathematically
>>> is for the defaultLUTs in the DefaultDatasetView to use color tables that
>>> return values that are 3x those of ColorTables.RED, GREEN and BLUE.
>>> Thinking about it, I'm afraid this *is* the correct model and each channel
>>> really is 3x brighter than possible.
>>>
>>>  It took me quite a bit of back and forth to come up with the above...
>>> I hope you all understand what I'm saying and understand the problem and
>>> counter-intuitive solution and have the patience to follow it. Dscho, if
>>> you made it this far - you're the mathematician, what's your take?
>>>
>>>  --Lee
>>>
>>>
>>> _______________________________________________
>>> ImageJ-devel mailing listImageJ-devel at imagej.nethttp://imagej.net/mailman/listinfo/imagej-devel
>>>
>>>
>>>
>>> _______________________________________________
>>> ImageJ-devel mailing list
>>> ImageJ-devel at imagej.net
>>> http://imagej.net/mailman/listinfo/imagej-devel
>>>
>>>
>>
>>
>> _______________________________________________
>> ImageJ-devel mailing list
>> ImageJ-devel at imagej.net
>> http://imagej.net/mailman/listinfo/imagej-devel
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://imagej.net/pipermail/imagej-devel/attachments/20130716/8f3460b6/attachment-0001.html>