Issues while writing to stdout

[Drarakel]

These are only small things - but perhaps someone wants to look into that.

1) Pixel cache files are sometimes not deleted. Try:

Code: Select all

convert rose: TIFF:- | identify -

I have looked into the debug messages - while using commands with "-limit area 0". It seems that when writing to file (small images), only one pixel cache file is created - which gets deleted afterwards. When writing to stdout in certain formats, e.g. GIF, there's also only one pixel cache file - which gets deleted afterwards. But when writing to stdout in other formats (like TIFF or BMP), two pixel cache files are created - and the second one isn't deleted afterwards.(?)

2) When writing image sequences to stdout, IM can crash:

Code: Select all

convert rose: logo: BMP:- | identify -

Such a command doesn't make much sense, but perhaps a crash can be avoided.
(When I'm using 'PNG:-' for example, only the rose image gets identified - but there's no crash.)

3) Writing to stdout while using "-verbose" doesn't seem to work:

Code: Select all

convert -verbose xc:white GIF:- | identify -

Magick: no decode delegate for this image format `R:/Temp/magick-_5_l-w3p' @ error/constitute.c/ReadImage/532.

When writing it as file, there's no problem of course:

Code: Select all

convert -verbose xc:white GIF:test.gif

xc:white=>white XC 1x1 1x1+0+0 16-bit DirectClass 0.000u 0:00.000
xc:white=>test.gif XC 1x1 1x1+0+0 16-bit Palette PseudoClass 1c 0.016u 0:00.016

Part of that originated in that thread:
viewtopic.php?f=1&t=16615

I'm using ImageMagick v6.6.3-0 Q16, on Windows XP.

[Drarakel]

From the other thread:

Code: Select all

convert rose: TIFF:- | identify -

No junk /tmp files were left, and I get a good identify.

Maybe the reported behaviour shows only in Windows. (Snibgo could reproduce it in the other thread.)
Another example - with bigger images:

Code: Select all

convert -limit area 0 -debug resource -log "%m/%f/%l\n  %e" -size 5000x10000 xc:blue TIFF:- | convert - null:

My debug output is:
resource.c/AcquireMagickResource/262
Area: 400MB/400MB/0B
resource.c/AcquireMagickResource/262
Disk: 400MB/400MB/18.446744EB
resource.c/AcquireUniqueFileResource/441

resource.c/AcquireUniqueFileResource/472
R:/Temp/magick-R3FJpZYE
resource.c/AcquireMagickResource/262
File: 1B/1B/1.54KB
resource.c/AcquireMagickResource/262
Area: 400MB/400MB/0B
resource.c/AcquireUniqueFileResource/441
-
resource.c/AcquireUniqueFileResource/472
R:/Temp/magick-eNsPKxhE
resource.c/RelinquishUniqueFileResource/834
R:/Temp/magick-eNsPKxhE
resource.c/RelinquishMagickResource/801
File: 1B/0B/1.54KB
resource.c/RelinquishMagickResource/801
Disk: 400MB/0B/18.446744EB
resource.c/RelinquishUniqueFileResource/834
R:/Temp/magick-R3FJpZYE

I've highlighted the temporary files - there are two of them in this example. (Well, there's a third one, probably for the metadata - that doesn't show up in the resource messages. But that one always gets deleted afterwards.) The second file ('magick-eNsPKxhE') holds the output data from the first convert command (here, a 5000x10000 TIFF file) and doesn't get deleted in the end. It's different if I write directly to file:

Code: Select all

convert -limit area 0 -debug resource -log "%m/%f/%l\n  %e" -size 5000x10000 xc:blue TIFF:test.tif

resource.c/AcquireMagickResource/262
Area: 400MB/400MB/0B
resource.c/AcquireMagickResource/262
Disk: 400MB/400MB/18.446744EB
resource.c/AcquireUniqueFileResource/441

resource.c/AcquireUniqueFileResource/472
R:/Temp/magick-NfP9EInU
resource.c/AcquireMagickResource/262
File: 1B/1B/1.54KB
resource.c/AcquireMagickResource/262
Area: 400MB/400MB/0B
resource.c/RelinquishMagickResource/801
File: 1B/0B/1.54KB
resource.c/RelinquishMagickResource/801
Disk: 400MB/0B/18.446744EB
resource.c/RelinquishUniqueFileResource/834
R:/Temp/magick-NfP9EInU

No remaining temporary files. The same if I write to stdout again - this time as PNG:

Code: Select all

convert -limit area 0 -debug resource -log "%m/%f/%l\n  %e" -size 5000x10000 xc:blue PNG:- | convert - null:

resource.c/AcquireMagickResource/262
Area: 400MB/400MB/0B
resource.c/AcquireMagickResource/262
Disk: 400MB/400MB/18.446744EB
resource.c/AcquireUniqueFileResource/441

resource.c/AcquireUniqueFileResource/472
R:/Temp/magick-R-iarSkt
resource.c/AcquireMagickResource/262
File: 1B/1B/1.54KB
resource.c/AcquireMagickResource/262
Area: 400MB/400MB/0B
resource.c/RelinquishMagickResource/801
File: 1B/0B/1.54KB
resource.c/RelinquishMagickResource/801
Disk: 400MB/0B/18.446744EB
resource.c/RelinquishUniqueFileResource/834
R:/Temp/magick-R-iarSkt

No problem with that command. (But I'm not completely sure if the number of temporary files in the debug output hints to the problem. When I write JNG to stdout, there are also two temporary files in the debut output - but here, they get deleted.)
Anyway, the problem of the remaining junk files while writing to stdout appears with formats TIFF, BMP, JP2, PCX, MNG.. At least on my system (Windows).

Code: Select all

convert rose: logo: BMP:- | identify -

Produces an error (convert: unable to write blob `-') so it may be the BMP coder can not handle stdout. It does leave 2 /tmp junk files.

The last message of "convert -debug all rose: logo: BMP:- | convert - null:":

Code: Select all

2010-07-14T19:21:34+02:00 0:00.234 0.063u 6.6.3 Policy Magick[1132]: policy.c/IsRightsAuthorized/485/Policy
  Domain: Path; rights=Write; pattern="-" ...

That's probably the attempt of writing the second image to stdout. After that point, IM crashes on Windows.
But it's not BMP alone. On my system, other formats crash, too. The same with TIFF - when used with '+adjoin':

Code: Select all

convert rose: logo: +adjoin TIFF:- | identify -

Do you need the complete debug output?

[magick]

We can reproduce the problems you reported and have a patch in ImageMagick 6.6.3-1 Beta available by sometime tomorrow. Thanks.

[Drarakel]

Thanks!

[Drarakel]

3) Writing to stdout while using "-verbose" doesn't seem to work:
...

OK, that's solved in version 6.6.3-1.

2) When writing image sequences to stdout, IM can crash:

Code: Select all

convert rose: logo: BMP:- | identify -

Such a command doesn't make much sense, but perhaps a crash can be avoided.
(When I'm using 'PNG:-' for example, only the rose image gets identified - but there's no crash.)

The crashes are gone. Thanks! There can be still small issues - for example the above command gives this error now (though the images are fine):

Code: Select all

Magick: length and filesize do not match `R:/Temp/magick-DSgS2dk8' @ error/bmp.c/ReadBMPImage/790.

But as I said, writing image sequences to stdout doesn't make much sense anyway. So, it's ok now.

1) Pixel cache files are sometimes not deleted. Try:

Code: Select all

convert rose: TIFF:- | identify -

I have looked into the debug messages - while using commands with "-limit area 0". It seems that when writing to file (small images), only one pixel cache file is created - which gets deleted afterwards. When writing to stdout in certain formats, e.g. GIF, there's also only one pixel cache file - which gets deleted afterwards. But when writing to stdout in other formats (like TIFF or BMP), two pixel cache files are created - and the second one isn't deleted afterwards.(?)

But this problem persists. At least on Windows.

[fmw42]

convert rose: TIFF:- | identify -

FYI: This command works on my Mac OSX TIGER IM 6.6.3.1 Q16. So looks like it is a Windows issue.

[Drarakel]

Can you try this command, too, Fred? I would be interested in your debug messages..

Maybe the reported behaviour shows only in Windows. (Snibgo could reproduce it in the other thread.)
Another example - with bigger images:

Code: Select all

convert -limit area 0 -debug resource -log "%m/%f/%l\n  %e" -size 5000x10000 xc:blue TIFF:- | convert - null:

On my system, 3 temporary files are created, and one of them isn't deleted in the end. In this example, it's a 286 MB TIFF file (the output from the first convert).
If I had to speculate, I would say that ImageMagick tries to delete the temporary file, but the file is not released from the system (Windows) yet, or something like that...

[fmw42]

Thomas,

I get

convert -limit area 0 -debug resource -log "%m/%f/%l\n %e" -size 5000x10000 xc:blue TIFF:- | convert - null:
resource.c/AcquireMagickResource/262
Area: 400MB/400MB/0B
resource.c/AcquireMagickResource/262
Disk: 400MB/400MB/18.446744EB
resource.c/AcquireUniqueFileResource/441

resource.c/AcquireUniqueFileResource/472
/var/tmp/magick-to4xrFUz
resource.c/AcquireMagickResource/262
File: 1B/1B/192B
resource.c/AcquireMagickResource/262
Area: 400MB/400MB/0B
resource.c/AcquireUniqueFileResource/441
-
resource.c/AcquireUniqueFileResource/472
/var/tmp/magick-N76n1aay
resource.c/RelinquishUniqueFileResource/834
/var/tmp/magick-N76n1aay
resource.c/RelinquishMagickResource/801
File: 1B/0B/192B
resource.c/RelinquishMagickResource/801
Disk: 400MB/0B/18.446744EB
resource.c/RelinquishUniqueFileResource/834
/var/tmp/magick-to4xrFUz

I only see two distinct files above.

Left in my /var/tmp directory, I have a whole bunch of files (perhaps they can be deleted manually, but I don't know enough about that). Any way, the magick files left are:


magick-6ePNb6cP
magick-6sYU3PiX
magick-FHAuUB33
magick-GrT2KSFe
magick-KnaTEpfC
magick-WMjwW4Ng
magick-ZfE99MZG
magick-aJJb1ZGe
magick-cBCX4YPZ
magick-jig3vcnO
magick-jlkB8Jew
magick-zueH9uZr


None of which seem to match the files above.

Fred

[Drarakel]

Thanks for your tests. When looking at the debug messages, it seems that ImageMagick essentially tries to do the same things on our different operating systems. But on Windows, one deletion is not successful.

Left in my /var/tmp directory, I have a whole bunch of files

At least delete the 'magick' files.

I only see two distinct files above.

Yeah, it's basically the same as in my output:

I've highlighted the temporary files - there are two of them in this example. (Well, there's a third one, probably for the metadata - that doesn't show up in the resource messages. But that one always gets deleted afterwards.) The second file ('magick-eNsPKxhE') holds the output data from the first convert command (here, a 5000x10000 TIFF file) [...]

There was one error in this statement: The 'third' temporary file here is simply from the second convert command (the one that reads from stdin) - no wonder that this one doesn't show up in the debug messages from the first convert command.
So... when looking at your temporary directory, you perhaps can see three new temporary files while the convert command is processing the image (should take a while). And on Windows, it seems that one of the two temporary files that show up in the debug messages can't get deleted by IM.

Perhaps it would be different if IM waited a bit before trying to delete that file on Windows. But that's only speculation.

[Drarakel]

I don't know how, but the issue with the remaining pixel cache files when writing to stdout got solved - with IM v6.6.3-2. At least on my system (Windows XP), these files now get deleted with this (or a newer) version.
Thanks!