Regression in bit depth reduction?

[Drarakel]

If one has a 16 bit (channel bit depth) image and saves that to a 8 bit quality file format, an implicit bit depth reduction is done. That implicit bit depth reduction should be same as the explicit "-depth 8" operation.. Is that correct?
(At least the documentation seems to imply that.)

If yes, then the implicit bit depth reduction has been changed or broken.
Try this:

Code: Select all

convert xc:"rgb(220,221,220)" xc:"rgb(220,222,220)" xc:"rgb(220,220,219)" -append -colorspace gray test1.bmp
convert test1.bmp test1.txt

# ImageMagick pixel enumeration: 1,3,255,rgb
0,0: (220,220,220) #DCDCDC gainsboro
0,1: (221,221,221) #DDDDDD rgb(221,221,221)
0,2: (219,219,219) #DBDBDB grey86

The first and third pixel values are wrong (according to the 'Gray' formula here).
But I get the correct values with an explicit '-depth 8':

Code: Select all

convert xc:"rgb(220,221,220)" xc:"rgb(220,222,220)" xc:"rgb(220,220,219)" -append -colorspace gray -depth 8 test2.bmp
convert test2.bmp test2.txt

# ImageMagick pixel enumeration: 1,3,255,rgb
0,0: (221,221,221) #DDDDDD rgb(221,221,221)
0,1: (221,221,221) #DDDDDD rgb(221,221,221)
0,2: (220,220,220) #DCDCDC gainsboro

I'm using IM v6.6.1-10 Q16, on Windows XP.

So, without the explicit '-depth 8' operation, a value of '219.886' (third pixel here; 16 bit representation: 0xDCBF) gets reduced to 219. Of course this was only an example. The slight errors occur with all sorts of operations. I just found out after some greyscale conversions when the values were lower than expected. All 8bit file formats (as output) like JPG, PNG8, GIF, etc. are affected. You only get 'correct' values by adding '-depth 8'.

I think this was introduced in version 6.6.0-6. Older versions produce a correct output - with and without '-depth 8'.

[magick]

The resulting gray values @ 16-bit are 56691, 56842, 56511. If we divide by 257 (16 => 8-bit) we get 220, 221, and 219. Suggesting ImageMagick is returning correct values. We have two different methods for computing gray values. One of them has an optimization that is buggy. The problem is fixed in ImageMagick 6.6.2-0 Beta available by sometime tomorrow. Thanks.

[Drarakel]

The problem is fixed in ImageMagick 6.6.2-0 Beta available by sometime tomorrow.

OK, thanks.
But with "the problem", did you mean something with the bit depth reduction - or something with the grayscale conversion? Because my posting was not about the actual grayscale conversion - that was just a random example to create some 16 bit values.

I will explain it a little more.. (But if you meant that with "the problem", just ignore the following text. )
Take again the grayscale example (now first stored with the 16 bit values):
convert xc:"rgb(220,221,220)" xc:"rgb(220,222,220)" xc:"rgb(220,220,219)" -append -colorspace gray gray16.png

The resulting gray values @ 16-bit are 56691, 56842, 56511.

Yes. And divided by 257, it would be approx. 220.59, 221.18 and 219.89. Conversion to 8 bit with '-depth 8':
convert gray16.png -depth 8 test2.bmp
The 8 bit values get rounded:
# ImageMagick pixel enumeration: 1,3,255,rgb
0,0: (221,221,221) #DDDDDD rgb(221,221,221)
0,1: (221,221,221) #DDDDDD rgb(221,221,221)
0,2: (220,220,220) #DCDCDC gainsboro

Now the conversion to 8 bit without '-depth 8':
convert gray16.png test1.bmp
No rounding in 'test1.bmp':
# ImageMagick pixel enumeration: 1,3,255,rgb
0,0: (220,220,220) #DCDCDC gainsboro
0,1: (221,221,221) #DDDDDD rgb(221,221,221)
0,2: (219,219,219) #DBDBDB grey86

That different behaviour seemed strange to me. As I said, especially since with older IM versions (up to v6.6.0-5), that difference is not there. (With these versions it always gets rounded to the 8 bit values 221, 221 and 220 in the above example - even without '-depth 8'.)

[rnbc]

Well, bit-depth (or shall I say quantum?) conversion has been changed lately, exactly because of a slight bias it was introducing. But that was only in the HDRI version, I think. At least that's were I detected the bug.

Apparently the explicit conversion is now working fine as far as I can tell, but not the implicit conversion... btw, are you testing the integer or the HDRI (float) version?

Of course all versions should deliver the same consistent and correct results for such simple operations, be they implicit or explicit.

For more info check: viewtopic.php?f=3&t=16157

[Drarakel]

For more info check: viewtopic.php?f=3&t=16157

Interesting thread!
I think you wrote about the same issue (a bias towards half a sample darker). It's just appearing at different places/versions...

I'm using the integer versions of ImageMagick (the regular Q16 binaries). And for that versions, I think the behaviour of the implicit conversions changed with version 6.6.0-6. Does anyone know why it was changed back then? (I didn't find a hint in the changelog.)

And for the current Q16 version (6.6.2-0), it got worse. Now, even the explicit conversions (with "-depth 8") don't round anymore!?

An example that magnifies the error:

Code: Select all

convert granite: granite.png
convert granite: gr2.png
FOR /L %%i IN (1, 1, 10) DO (
	mogrify -gamma 0.8 gr2.png
	mogrify -gamma 1.25 gr2.png
)

That script (for Windows) takes the granite image and makes some gamma conversions. Every step is saved with 8-bit channel bit depth. After every loop, the image should have only a very small discrepancy to the original image (or no discrepancy at all). With v.6.6.2-0, after every loop the image gets darker. After 10 such loops, the difference is as follows:

granite.png:
...
Channel statistics:
Red:
min: 160 (0.627451)
max: 187 (0.733333)
mean: 176.928 (0.693834)
Green:
min: 152 (0.596078)
max: 187 (0.733333)
mean: 176.861 (0.693574)
Blue:
min: 160 (0.627451)
max: 195 (0.764706)
mean: 179.532 (0.704047)

gr2.png:
...
Channel statistics:
Red:
min: 150 (0.588235)
max: 177 (0.694118)
mean: 166.928 (0.654619)
Green:
min: 146 (0.572549)
max: 177 (0.694118)
mean: 166.868 (0.654384)
Blue:
min: 150 (0.588235)
max: 185 (0.72549)
mean: 169.532 (0.664831)

So, the differences get clearly visible after repeated conversions.
You will get the same difference with this command:

Code: Select all

convert granite: -gamma 0.8 -depth 8 -gamma 1.25 -depth 8 -gamma 0.8 -depth 8 -gamma 1.25 -depth 8 -gamma 0.8 -depth 8 -gamma 1.25 -depth 8 -gamma 0.8 -depth 8 -gamma 1.25 -depth 8 -gamma 0.8 -depth 8 -gamma 1.25 -depth 8 -gamma 0.8 -depth 8 -gamma 1.25 -depth 8 -gamma 0.8 -depth 8 -gamma 1.25 -depth 8 -gamma 0.8 -depth 8 -gamma 1.25 -depth 8 -gamma 0.8 -depth 8 -gamma 1.25 -depth 8 -gamma 0.8 -depth 8 -gamma 1.25 -depth 8 gr2b.png
compare -metric mae granite: gr2b.png null:

That's for 10 such 'loops'. "Compare" shows a mean absolute error of 4% at this step.

If you omit the 8 bit reductions, the error is of course gone:

Code: Select all

convert granite: -gamma 0.8 -gamma 1.25 -gamma 0.8 -gamma 1.25 -gamma 0.8 -gamma 1.25 -gamma 0.8 -gamma 1.25 -gamma 0.8 -gamma 1.25 -gamma 0.8 -gamma 1.25 -gamma 0.8 -gamma 1.25 -gamma 0.8 -gamma 1.25 -gamma 0.8 -gamma 1.25 -gamma 0.8 -gamma 1.25 gr0.png
compare -metric mae granite: gr0.png null:

If I use IM versions up to 6.6.0-5 - or up to 6.6.1-10 (with explicit "-depth 8") - I can make as many 'back and forth'-conversions as I want: There, the 8-bit values get correctly rounded and in my example, the final output is even identical to the original image.
And if I use the Q8 version of 6.6.2-0, I also get an identical image after the processing! So, with simple conversions, the Q8 version is now more precise than the Q16 version.
Is it possible that the rounding of 8 bit values is re-enabled?


Edit:
After 49 loops of my script at the top of the post, I now have a complete uni-color, dark grayscale image.

Original granite.png:

49 gamma/8bit reduction loops with ImageMagick v6.5.8-5:

49 gamma/8bit reduction loops with ImageMagick v6.6.2-0:

[magick]

We can reproduce the problem you posted and have a patch. Look for a new point release of ImageMagick within a day or two. Thanks.

[Drarakel]

Thanks.
The explicit 8 bit reductions are fine again now, but the implicit ones (all the examples without "-depth 8") still produce the differences.

Code: Select all

convert xc:"rgb(220,221,220)" xc:"rgb(220,222,220)" xc:"rgb(220,220,219)" -append -colorspace gray gray16.png
convert gray16.png test1.bmp
convert gray16.png -depth 8 test2.bmp
convert test1.bmp test1.txt
convert test2.bmp test2.txt

test2.bmp (explicit bit depth reduction):
# ImageMagick pixel enumeration: 1,3,255,rgb
0,0: (221,221,221) #DDDDDD rgb(221,221,221)
0,1: (221,221,221) #DDDDDD rgb(221,221,221)
0,2: (220,220,220) #DCDCDC gainsboro

test1.bmp (implicit bit depth reduction):
# ImageMagick pixel enumeration: 1,3,255,rgb
0,0: (220,220,220) #DCDCDC gainsboro
0,1: (221,221,221) #DDDDDD rgb(221,221,221)
0,2: (219,219,219) #DBDBDB grey86

Edit: Corrected the pixel values.. (I had accidentally swapped them.)
The higher values (rounded) are the correct ones. The lower values are getting truncated (at the 16bit->8bit conversion).

[magick]

In general we use integer division to go from 16-bit to 8-bit because it is more efficient ( divide by 257). We were using floating point which is more precise for the -depth option. However, the results should be the same for -depth and when saving 16-bit pixels to 8-bits. We have a patch in ImageMagick 6.6.2-2 Beta to fix the problem. It will be available by sometime tomorrow. Note, it returns

• # ImageMagick pixel enumeration: 1,3,255,rgb
  0,0: (221,221,221) #DDDDDD rgb(221,221,221)
  0,1: (221,221,221) #DDDDDD rgb(221,221,221)
  0,2: (220,220,220) #DCDCDC gainsboro

Which is the correct results for 16-bit pixel / 257 => 8-bit.

[Drarakel]

Let's put it that way: This is the situation when you have a Q8 version:
All conversion results get some sort of rounding, when they are stored at 8bit (so the values are not getting darker after every conversion).
This is the situation with the 16bit->8bit reductions when you have a Q16 version:
- IM up to v6.6.0-5: all fine (there's always a proper rounding)
- IM v6.6.0-6 till v6.6.1-10: explicit reduction - rounding, implicit reduction - no rounding
- IM v6.6.2-0: explicit and implicit reductions - no rounding
- IM v6.6.2-1: explicit reduction - rounding, implicit reduction - no rounding
- IM v6.6.2-2 and v6.6.2-3: explicit and implicit reductions - no rounding

Note, it returns

• # ImageMagick pixel enumeration: 1,3,255,rgb
  0,0: (221,221,221) #DDDDDD rgb(221,221,221)
  0,1: (221,221,221) #DDDDDD rgb(221,221,221)
  0,2: (220,220,220) #DCDCDC gainsboro

Which is the correct results for 16-bit pixel / 257 => 8-bit.

Yes, I would like to get that result. With the regular IM Q16 versions 6.6.2-2 and newer, it's again not possible.

Somehow the IM versions up to v6.6.0-5 were able to do this. I don't know how.. (Perhaps some conversions were done with a division by 257.0 and an addition of 0.5 or something like that.) I just see the results.
If my understanding of proper bit depth/quantum conversions is wrong or can't be done, then please say so. I mean, these changes since v6.6.1-10 must have been some work for you.

(Right now, implicit and explicit bit depth reductions produce the same result, yes. But it's the slightly wrong result. So for me, the results are now again worse - compared to the situation before I started the thread. That's.. a bit frustrating. )

[magick]

Download ImageMagick 6.6.2-4 and let us know if that fixes the problem you reported.

[Drarakel]

I've downloaded IM v6.6.2-4 Q16 for Windows, and I think all problems are gone now. I'm getting the same precise results as in versions up to 6.6.0-5. Sweet!
Thank you!