Problems with PNG8, color reduction, etc.

[Drarakel]

New cases.
(Caution: Long posts coming...)

I'm using IM v6.6.7-10 Q16, on Windows XP.


Works (and is written as palette PNG):

Code: Select all

convert logo: -transparent white logo.png
identify -format "%[channels]" logo.png

rgba

Still works:

Code: Select all

convert logo: -transparent white PNG8:logo.png
identify -format "%[channels]" logo.png

rgba

Now alpha channel disappearing (the tRNS chunk is missing):

Code: Select all

convert logo: -transparent white -define png:color-type=3 logo.png
identify -format "%[channels]" logo.png

rgb

Also fails:

Code: Select all

convert logo: -transparent white -type PaletteMatte logo.png
identify -format "%[channels]" logo.png

rgb

But this works again (but the image already has only 256 colors, so this shouldn't be needed):

Code: Select all

convert logo: -transparent white -colors 256 -type PaletteMatte logo.png
identify -format "%[channels]" logo.png

rgba

When combining that with color-type=3 (palette), it still fails:

Code: Select all

convert logo: -transparent white -colors 256 -type PaletteMatte -define png:color-type=3 logo.png
identify -format "%[channels]" logo.png

rgb

[Drarakel]

To more severe errors:

This works (it's written as grayscale+alpha):

Code: Select all

convert logo: -colorspace gray -transparent white logog.png

Trying to write that image (256 colors) as palette PNG:

Code: Select all

convert logo: -colorspace gray -transparent white PNG8:logog8.png

I'm getting a crash.
(Looking at the debug messages before the crash, libpng reports the "Valid palette required for paletted images" and there's an exception.)

No crash here, but generates the same corrupt image as above (with only a header and a color profile chunk):

Code: Select all

convert logo: -colorspace gray -transparent white -define png:color-type=3 logog8b.png
identify logog8b.png

Magick: Expected 8 bytes; found 0 bytes `logog8b.png' @ warning/png.c/MagickPNGWarningHandler/1306.
Magick: Read Exception `logog8b.png' @ error/png.c/MagickPNGErrorHandler/1280.
Magick: corrupt image `logog8b.png' @ error/png.c/ReadPNGImage/3102.

Working file, but the alpha channel (tRNS chunk) is missing:

Code: Select all

convert logo: -colorspace gray -transparent white -type PaletteMatte logog8c.png
identify -format "%[channels]" logog8c.png

rgb
(One can also add 'PNG8:' or 'color-type=3' now, but the tRNS chunk of course is again missing.)

But with adding the "-colors 256" (although the transparent, grayscale logo has only 256 colors in 16bit), it works (and is written as palette+tRNS automatically):

Code: Select all

convert logo: -colorspace gray -transparent white -colors 256 logog8d.png
identify -format "%[channels]" logog8d.png

rgba

Perhaps some of the errors with no 'valid palette' are already fixed in IM v6.6.8-0? But I just wanted you to know that there can be still crashes with the current version. I would understand some of the errors if these would be images with more than 256 colors, but this isn't even the case here. And the transparency is always binary transparency (and with my examples, all white pixels are fully transparent; though IM sometimes changes the transparent areas to a transparent black - but even in these cases, all black pixels are fully transparent then).
Of course, as I'm using Q16, I have 16bit images in between (again: with only 256 colors). But that means that IM or the PNG module only have to do a bit depth reduction, no color reduction (when explicitly saving with "-depth 8" - or when IM detects that the input image is only 8bits per channel, so that it does an implicit bit depth reduction) - right?


Looking at the above (grayscale) examples a bit closer:

Code: Select all

convert logo: -colorspace gray -transparent white -depth 8 PNG8:logog8_2.png
identify -format "%[channels]\n%k" logog8_2.png

rgba
160

Without "-depth 8", IM crashed. Now, it works (I've also added a color count now). But remember: Without "-depth 8" and without "PNG8:", it worked, too (and an automatic bit depth reduction was made). It's just that some combinations don't work.

But this still doesn't work completely? (at least no corrupt file):

Code: Select all

convert logo: -colorspace gray -transparent white -depth 8 -define png:color-type=3 logog8b_2.png
identify -format "%[channels]\n%k" logog8b_2.png

rgb
160

With the next example, "-depth 8" doesn't help. "-type PaletteMatte" just always loses the alpha channel - and additionally, the colors are reduced (from 160 to 58 or 60, depending where the type option is added).

Code: Select all

convert logo: -colorspace gray -transparent white -depth 8 -type PaletteMatte logog8c_2.png
identify -format "%[channels]\n%k" logog8c_2.png

rgb
60

This already 'worked' above - but look at the colors:

Code: Select all

convert logo: -colorspace gray -transparent white -colors 256 logog8d.png
identify -format "%[channels]\n%k" logog8d.png

rgba
59

Just adding "-depth 8" has the same color-reduced result here.
I have to disable dithering for that to work:

Code: Select all

convert logo: -colorspace gray -transparent white -depth 8 +dither -colors 256 logog8d_2.png
identify -format "%[channels]\n%k" logog8d_2.png

rgba
160

Now, I can also improve the "-type PaletteMatte" result - even here I have to disable dithering (but the alpha channel is still missing):

Code: Select all

convert logo: -colorspace gray -transparent white -depth 8 +dither -type PaletteMatte logog8c_3.png
identify -format "%[channels]\n%k" logog8c_3.png

rgb
160

For "-type PaletteMatte" to work here, I have to add the complete "+dither -colors 256":

Code: Select all

convert logo: -colorspace gray -transparent white -depth 8 +dither -colors 256 -type PaletteMatte logog8c_4.png
identify -format "%[channels]\n%k" logog8c_4.png

rgba
160


Okay, the thing with "-colors" (with dithering) is probably a separate issue (there are other threads about that).
And the trouble with "-type PaletteMatte": That first fooled me in thinking that the problem is there only in conjunction with the PNG format. Again - example:

Code: Select all

convert logo: -transparent white -type PaletteMatte PNG:- | identify -format "%[channels]" -

rgb
With MIFF:

Code: Select all

convert logo: -transparent white -type PaletteMatte MIFF:- | identify -format "%[channels]" -

rgba

But: The alpha channel is also 'destroyed' - even with MIFF. It's just that the PNG encoder sees that the alpha channel is fully opaque at the end and so it doesn't write the channel anymore in the current versions. I could even use "PNG32:" instead of "PNG:". With that, the file would be written as truecolor+alpha of course. But the alpha channel would be again fully opaque (and even there, IM reports only "rgb"/doesn't recognize any alpha channel, as the PNG decoder now leaves that away when it's fully opaque.. or something along that line).
So that single problem seems to be the fault of "-type PaletteMatte" alone.

And another thing:
In some of the examples (with "-depth 8 +dither -colors 256" and "-depth 8 +dither [-colors 256] -type PaletteMatte"), I can leave away the "-depth 8" and get the same (good) color count. But: a few pixels have shifted color values with that. Seems that there is also an automatic bit depth reduction in "-colors" and "-type PaletteMatte" that is less precise. Strange.


Regarding the problems with the PNG encoder: Some of these are not that dramatic.. and there were already large improvements with the PNG format in ImageMagick lately. I don't know how much work it is to fix some of the remaining issues. But maybe these cases help a bit.

[glennrp]

Some of these problems are addressed in ImageMagick-6.6.8-0 (beta).
The "palettematte" problem seems to be occurring before the PNG encoder
is called.

[Drarakel]

The "palettematte" problem seems to be occurring before the PNG encoder
is called.

Yeah, that's probably true. It's just sometimes difficult to separate these issues completely.

Some of these problems are addressed in ImageMagick-6.6.8-0 (beta).

I'll wait for the next release version and then try again. Thanks!

[Drarakel]

@Glenn: I'll try to list only the PNG encoder problems. Unfortunately, the errors stayed the same in IM v6.6.8-2 (Q16, Win32).

Now alpha channel disappearing (the tRNS chunk is missing):

Code: Select all

convert logo: -transparent white -define png:color-type=3 logo.png
identify -format "%[channels]" logo.png

rgb
...
When combining that with color-type=3 (palette), it still fails:

Code: Select all

convert logo: -transparent white -colors 256 -type PaletteMatte -define png:color-type=3 logo.png
identify -format "%[channels]" logo.png

rgb

Trying to write that image (256 colors) as palette PNG:

Code: Select all

convert logo: -colorspace gray -transparent white PNG8:logog8.png

I'm getting a crash.
(Looking at the debug messages before the crash, libpng reports the "Valid palette required for paletted images" and there's an exception.)

No crash here, but generates the same corrupt image as above (with only a header and a color profile chunk):

Code: Select all

convert logo: -colorspace gray -transparent white -define png:color-type=3 logog8b.png
identify logog8b.png

Magick: Expected 8 bytes; found 0 bytes `logog8b.png' @ warning/png.c/MagickPNGWarningHandler/1306.
Magick: Read Exception `logog8b.png' @ error/png.c/MagickPNGErrorHandler/1280.
Magick: corrupt image `logog8b.png' @ error/png.c/ReadPNGImage/3102.

...

Looking at the above (grayscale) examples a bit closer:

Code: Select all

convert logo: -colorspace gray -transparent white -depth 8 PNG8:logog8_2.png
identify -format "%[channels]\n%k" logog8_2.png

rgba
160

Without "-depth 8", IM crashed. Now, it works (I've also added a color count now). But remember: Without "-depth 8" and without "PNG8:", it worked, too (and an automatic bit depth reduction was made). It's just that some combinations don't work.

But this still doesn't work completely? (at least no corrupt file):

Code: Select all

convert logo: -colorspace gray -transparent white -depth 8 -define png:color-type=3 logog8b_2.png
identify -format "%[channels]\n%k" logog8b_2.png

rgb
160

[glennrp]

Interestingly, the -colorspace gray examples work with the Q8 build but not with the Q16.

The root cause of the failure is that when image depth is 16, there are 257 colors
including the background. The example succeeds with "-define PNG:exclude-chunk=bkgd"

The PNG8 failure is fixed in ImageMagick-6.6.8-4, SVN r3844.

[Drarakel]

Thanks! I didn't think of the background color.
I wonder why IM sets such a random, non-grayscale background color in the first place. I already encountered other cases where the background color automatically changed to a strange value. Of course only a minor problem.

The root cause of the failure is that when image depth is 16, there are 257 colors
including the background.

Perhaps IM should do an automatic bit depth reduction to 8 first - before checking if the color count is <=256, to be able to save it as palette PNG? Well, I don't want to.. push the button.

[glennrp]

I wonder why IM sets such a random, non-grayscale background color in the first place.

In this particular case it was not random. It was opaque white, which happens to be the only opaque level not in the palette because
the "-transparent white" option made them transparent white. So, adding the background to the already
present 1 transparent and 255 opaque colors makes 257.