File Handling -- IM v6 Examples

Index

ImageMagick Examples Preface and Index

Image Formats Summary

Saving Images

Reading Images

Read Modifiers

Importing Images from special sources

Special Output File Formats (Specific to IM)

miff: info: null: txt: histogram: mpr: mpc: x:, & win:

Compressing Images

Delegating Image Coding

Writing a Multi-Image Sequence

Starting Sequence (Scene) Number

Writing an Image, Multiple Times

Really Massive Image Handling

To process an image you not only need operators work on the images but you also need ways to read in and write out the image in as many different file formats as posible. In this section we look at IM file formats in general.

Image Formats Summary

One of the most common uses of ImageMagick is not to modify images at all, but only to convert an image from one image format to another. In fact this was the original reason for IM's creation was this sort of image format conversion. This is why the primary IM command is call "convert".

To this end, ImageMagick can handle a bewildering array of image and file formats. Added to this array, are a large number of special input and output formats for built-in test images, simple image creation, and image formats specific for programming shell scripts, and programs. For a complete list see IM Image Formats Page, on the IM web site.

All this can be daunting for a new user of ImageMagick. My best advise is, ignore most of the file formats, as you will probably never need them. Instead concentrate on what you want to do, and try to do it. If you don't know how, try to look for an example in these pages and across the web.

For image formats demonstrated in IM Examples see Reference Index, File Formats.

Saving Images

Processing images is well and good but it can be just as important to save the results in the right way.

The last argument of the "convert", "montage" and "composite" defines the save filename, and image format for that image. Though you can also save an image in the middle of an image sequence using "-write".

To specify what file format you want to save your image, or images, you can either, use a filename suffix, such as I use in just about all these examples, or prefix the filename with the string "{format}:". For example...


    convert tree.gif    GIF:tree_image

If you check the resulting image you will find that the GIF image created does not have a ".gif" filename suffix. The case of the format is not sensitive, so you can use either lowercase or uppercase.

This image format specification becomes particularly important when you want to save the image to the standard output of the command (using a "-" filename). This special filename does not have a suffix, so you must tell ImageMagick what format to use. If you don't, the image will default to the original image format that the image came from (if known) or produce a 'delegation error'.

For example, here we write a IM pixel enumeration to the screen using a "-" to output the result to the standard output.


    convert tree.gif  -resize 1x3\!  txt:-

Or if we like to pass the image, on to another command such as "identify" though a shell 'pipeline', without saving it to a temporary file.


    convert tree.gif -resize 200% miff:- | identify -

In this case you can also see that the special "-" filename is also used to denote reading an image from standard input by the "identify" command.

The save filename can contain a few special percent escape (%) sequences. Specifically '%d', '%x', and '%o'. These inserts the images 'scene number' into the file name using the C 'printf()' formats. For more information see Writing a Multi-Image Sequence below.

Of course this means that if you want to insert a percent character into the filename you will need to double it ('%%').

Automatic GZip Suffix
IM will also automatically "gzip" images if a ".gz" suffix is given.

For example here I save the built-in "rose:" image as a "gzip"ed, uncompressed GIF file. I turn off the normal LWZ compression of GIF, as it would prevent "gzip" compression from achieving its best compression.


    convert rose: -compress none  rose.gif.gz

How browsers handle a gzipped image depends on the file type returned by the web server and how your browser handled compressed images. Because of this I did not directly display the above image. Click on the 'art' icon to see what your browser does, with such an image from this web server.

Compare the size of this to a normal saved LWZ compressed GIF image...


    convert rose: rose.gif

The "gzip"ed rose is

bytes in size, while a normal LWZ compressed rose is

bytes. As you can see ZIP compression is actually better than the LWZ compression the GIF format uses, so may be better for archiving purposes.

Other Settings specific to image writing....
    -depth  -quality  -compress -type  -loop
Also see Image Depth,
Image Type,
JPEG Quailty,
PNG Quailty.
GIF loop.

IM also allows you save sensitive images with encrypted with a pass phrase using the options "-encipher" and "-decipher". See Encrypting Images

Reading Images

It may look weird to talk about image reading after image saving, but reading images is a much more complex matter than saving, with more options and settings being applied to the image as it is being read.

IM by default will attempt to determine the image format type by the 'magic' file identification codes within the file itself. If this fails however you will need to specify the images file format using with the files suffix, or by adding a prefix format.

Some formats will not read any files and ignore any given filename. These are some of the common builtin images...


    logo:      granite:     rose:

Some of them will generate images based on arguments given as a filename and prehaps an extra "-size" controlling the final image size...


    -size 30x30  xc:red
    -size 30x30  gradient:yellow-limegreen
    -size 30x30  pattern:fishscales
    import:

In some cases you can even use a format inside the filename argument of another format...


    -size 30x30  tile:pattern:gray95

When a prefix file format is given, any suffix given as part of the filename does not have any bearing on the way the file is read. This is in fact vital when reading some file formats such as the "text:" verses the "txt:" file format handling. Of course if a image generator actually reads in a image file to process it in a special way (for example "tile:") then the suffix (or prefix) file formats will again become important, as it was in the last example.

If the filename is the special string '-' IM reads the image from standard input.

Filename can have the special 'file meta-characters', such as '*' and '?' embedded in them. IM will expand these characters to generate a list of filenames to be read in, avoiding the need for an external shell to do this, or problems with command line length limits. For example...


    montage  '*.jpg' -geometry 50x50+2+2  image_index.gif

will produce a single montage index image of all the JPEG files in the current directory. Note however that I needed to quote the argument to prevent my UNIX shell from expanding the file names rather than ImageMagick. See below for a more complete "montage" specification.

IM can also use a image that is published on the 'world wide web' by specifying that images URL. You can of course also modify that image before finally saving the result to disk.


    convert http://www.cit.gu.edu.au/~anthony/images/anthony_castle.gif \
            -resize 100x100 castle_logo.png

The special character '@' at the start of a filename, means replace the filename, or component with the contents of the given file.

Read Modifiers

On the end of the filename and format specification you can also add a special square bracket '[...]' modifier to the end of the filename. These are performed after the whole specified image has been read into memory, but before it is added to the current image sequence.

They are generally used to limit the amount of memory needed, by pre-processing images before all the input images are read in.

Here are the special '[...]' modifiers and their effects. The '#' represent some number.

'[#]' '[#-#]' '[#,#,#]' [#,#-#,#]'.

Will select specific sub-frames from a multi-image file format from the image that has been read in. The given number '#' index specifies the frame number to read. Multiple indexes can be specified in either comma order or as a index range.

The image index start at zero for the first image, 1 for the second and so on. If you specify a negative index then the count is from the end of the image sequence, in reverse order, -1 for the last image, -2 for the second last image.

This is exactly the same convention as used for the Image Sequence Operations.

For example


    convert document.pdf'[0]'     first_page_of_pdf.gif
    convert animation.gif'[1-3]'  second_to_fourth_frames.gif
    convert animation.gif'[-1,2]' last_then_the_third_frame.gif

'[#x#]'

From IM version 6.2.6-2 a new modifier was added to help IM users to handle very very large images.

This modifier will resize the image that was just read in, immediately before that image is added to the other images already in memory. For example a small image can be enlarged as part of the image read.


    convert pattern:gray95'[60x60]' enlarged_dots.gif

The modifier is most important when you are attempting to read in lots of very very large images, as each image will be resized before the next image is read, producing a substantial saving in total memory needed to handle those images.

For example instead of...


    montage '*.tiff'  -geometry 100x100+5+5 -frame 4  index.jpg

which reads all the tiff files in first, then resizes them. You can instead do...


    montage '*.tiff[100x100]'  -geometry 100x100+5+5 -frame 4  index.jpg

This will read each image in, and resizes it, before proceeding to the next image. Resulting in far less memory usage, and possibly prevent disk swapping (thrashing), when memory limits are reached.

For JPEG images I recommend you use somethng like...


    montage -size 200x200 '*.jpg[100x100]' -strip \
            -geometry 100x100+5+5 -frame 4  index.png

Note that the "-size" setting has a special meaning for JPEG image format reading. It is passed to the JPEG library and limits the reading in the JPEG image produce an image that size or slightly larger. In other words it further limits the memory usage for large JPEG images, before the input modifier resizes the image down to the actual size wanted. See Reading JPEG Images.
Only the JPEG image file format uses "-size" in this way. You can not use it for TIFF or PNG image file formats, which is why the input resize modifier was developed for general use.

'[#x#+#+#]'

From IM v6.3.1 if you also add a offset the above becomes a crop of the image being read in.

For example, To get a smaller 600x400 pixel tile from a very large image.


    convert 'image.png[600x400+1900+2900]' tileimage.png

This however will read in the entire image into memory then crop it before it is finally added to the current image sequence.

If you want to handle really large images I suggest you look at the "stream" command and pipe you image into the "convert" command for further processing. See Massive Image Handling below.

Note however that '[]' characters are usally also special shell meta-characters, so if you use them it is a good idea to quote the additional modifier, to stop UNIX shells interpreting it.

Also when you use a modifier, you must let IM handle any special file expandsion meta-characters, such as '*' and '?', as a UNIX shell will not 'find' the requested files due to the modifier. What it actually does in that case is shell dependant. As such the whole filename, should quoted when using read modifiers.

FUTURE
Using "-scenes" to read
in a sequence of numbered files,

   convert -scenes 5-7   image  +scenes ...

will read in the filenames  "image.5", "image.6", 'image.7"...

You can specify the number format using a C langauage printf() specification.
For example.

   convert -scenes 5-7  image_%03d.gif +scenes  ...

will read in the images   "image_005", "image_006", 'image_007"...

WARNING: be sure you turn off the setting before attempting to read in
any other un-numbered image.

If the image is "gzip"ed, IM will automatically uncompress it, into a temporary file before attempting to figure out the image format and decode the image file format. As such you can not only save images in gzip compressed format, but use them directly in later IM processing. For large text based images this can result in enormous disk space savings.

The PNG format includes "gzip" compression as part of its format specification. In this case the first digit of the two digit PNG "-quality" setting defines the level of compression. For more detail see PNG Image File Format examples.

The above is only a short summary of the special input options available when reading images into ImageMagick. A full summary is given on the The Anatomy of the Command Line page on the ImageMagick Website.

As shown previously the image input can be modified by some IM settings such as "-size" for image creation and JPEG reading. Other options also effect image input creation, including, "-page", "-type", "-dispose", "-delay".

WARNING: Be very careful when passing a user provided argument to IM in a script, insuring that the argument is what you expect. You do not want to let a web image processing script return a image of the system password file for example.

FUTURE: See Image Attributes, in IM basics.

Importing Images from special sources

To grab an image X windows Display, using the "import"..

import     read images from the on screen display

If a single mouse click is used the whole window clicked in is
grabbed and returned.  Note that it will try to rasie the window to
the top so that no other windows are obscuring the selected window
being grabbed.

A click in the root window will return the whole screen.

If you click and drag the mouse, a smaller section of the visible
screen is returned.

Other options allow you to avoid human interaction with the mouse
by grabbing the whole screen ("-window root"), or a specific
window (given a X window ID, which you can find using the X window
utility "xwininfo". You can also cut down the area
of the selected window ("-extract").

You can also import images by reading from "x:" or a special window selected by ID using "x:{id}". If you write to "x:" you will display an image on script using a very low level "display"-like way.


Windows Clipboard  use the "clipboard:"  read delegate, for example...
        convert clipboard:myimage image.png

Special File Formats (specific to IM)

As you saw above, ImageMagick understands a huge number of well known image file formats. It also includes a good number of special image generators. On top of these there are also some very special file formats specific to IM. or which allow some very special handling of images.

All these special file formats are both read as well as write formats.

miff:

is the ImageMagick File Format. The whole image sequence and all the attributes associated with the images are saved in this file format. Of course only ImageMagick commands will read this format, so it is not suitable for long term or transferring between different image processing packages.

The "miff:" file formats primary purpose is as a intermediate save format during large image processing scripts. Or when 'pipelining' an image from one IM command to another. See the last example for one such pipeline.

For those interested in parsing this format, it is a semi-text format consisting of a ascii header of image attributes, a line containing a single formfeed, followed by the raw image data as a binary. This header is itself a useful way of extracting basic image information in various image processing processing scripts.

For example here are I use a GNU-sed command to show all the attributes (the few that there are), of the built-in "rose:" image.


  convert rose:   miff:-  | sed -n '1,/^\f$/{ /^\f$/d; p; }'

This is actually quite useful as it reveals all the current settings flags and meta data that IM knows about the image. However there is also statistics, as these are generated by either: the "identify" command, the "-identify" operator or the special "info:" format; if when requested with a "-verbose" option. (see next)

The "miff:" format can also handle multiple images, and in fact does so simply by appending the images together.

This means you do not need to process all the images, by a single command. You can put the command in a loop, and just output MIFF format images, one after the other, to the loops standard output. that result can then be piped directly into a single command to generate montages, collages, animations, or something else.

For example the following generates a list of colors starting with the letter 'b', then uses a separate convert command to generate label for each color, one label at a time. These are then montaged together to produce a simple color table.


  convert -list color | egrep '^b' | \
    while read color junk; do \
       convert -label $color -size 70x20 xc:$color  miff:-; \
    done |\
      montage - -frame 5 -tile 6x -geometry +2+2 \
              -background none color_table.png

The above was also converted into a script "show_colors" which you can use to search for and find colors to use in your image.

This looped image output technique also means you can use code like "-write miff:-", to output a miff format image from multiple places in a single command. Each image will be automatically append together in the final output stream.

This stream can not only be feed into "montage", say to debug image processing, but also into "display" (with some timing settings) as a simple slide show. Finally this format (while not compressed) can handle ANY type image IM knowns about at any depth or quality level the IM was compiled with.

It is about the most ideal format to use for temporary images, and pipelined image commands you can use, though IM is the only one that can read it.

A very useful image file format indead.

info:

The "info:" file format (added IM v6.2.4) does NOT output an actual image! This format basically outputs the same information that the ImageMagick "identify" command will output.

Like "identify" this output format is controlled by the "-format" and "-verbose" options allowing you to output just the specific information you are interested in, as defined by the Image Property Escapes page.

For example instead of piping a MIFF image to "identify" as we did above (see Saving Images), we could have used the following, to retrieve the single line identification of the resulting image format.


  convert  granite:  info:-

Of course you can use a "-format" setting to output the desired information in a specific and more parsable way.

What is so useful about "info:" is that you can now produce your image, while extracting extra information about it, at the same time. This is done by using the "-write" operator to save this special image format to a file (or the commands normal standard output).


  convert rose: -shave 12x0 -repage 64x64+9+9 \
          -format '%wx%h %g'  -write info:info_paged.txt    paged.gif

There is also a new "-identify" operator that is equivelent using "-write info:-" to output image identification information to standard output. This make it even easier to monitor what is happening to your images when debugging your IM commands.

For example...


  convert logo:    -identify    -trim             -identify \
          +repage  -identify    -resize 80x80\!   -identify \
          logo_thumbnail.gif

Here you can see how "-trim" reduced the size of the image but preserves the 'crop' information of what part of the image was trimmed, then the "+repage" removing that extra 'canvas' or 'page' information.

Also like the "identify" command, both "info: and "-identify", will become much more verbose, if the "-verbose" setting is turned on. Here I limit the long output to just the first few lines, just so you can get a bit of an idea about it.


  convert  rose:  -verbose  info:-  | head

The "-verbose" setting will also cause extra information about images being read in or out, to be printed to the standard error (with the exception of the "info:" format). It also causes some operators like "-colors" to output additional information. As such you may like to turn it off again after using it with either "-identify" or the "info:" format.
For example "

-verbose -write
      info:image_info.txt +verbose

" or "-verbose -identify +verbose" .

Scripted readers of any form of "identify" command output, should do so in a case in-sensitive way, for backward compatibility between different versions of ImageMagick.

NOTE: "info:" (and "-identify") is only special an output format, producing the same output as the "identify" command. You can not read, or create an image using the "info:" file format.

null:

As an output format, this will just 'junk' the image results. As such if used as the final argument in a "convert", "montage", or "composite" command the final result will not be saved!

Why? Well it may be that you are more interested in specific images, generated during image processing rather than the overall result, especially when debugging.

For example, here we extract and save one image, from an image sequence, then junk all the other images using "null:".


  convert  eye.gif news.gif storm.gif tree.gif rose: logo: cyclops.gif  \
            \( -clone 2 -write write_storm.gif \)   null:

This is a lot simplier than attempting to delete all the other images one at a time.

As a input image format however, "null:" will generate a special placeholder image of a single transparent pixel, with with a special 'null source' flag, in the current image sequence.

This special image is especially important to Leave Gaps in a Montage, and as a list separator for multi-image Layer Composition. It is closely related to another special image format known as a 'missed image', that can be generated for operations like "-crop". This image format is produced when an operation produces an empty or non-sensible result. Both images are a single transparent pixel, and as such "null:" images will also be treated as if it is a 'missed image'.

At this time there is no method to remove any "null:" or even 'missed image', from the current image sequence. However such a method has been proposed. Mail me if you find you need such a method.

txt:

This is a simple ASCII text file, which basically lists each pixel in the image, one per line. It is not a general text to image converter, for that see Multi-line Text Files Examples.

For example here is a "netscape:" image scale to a 2x2 pixel image, then listed using a "txt:" image format.


  convert  netscape: -scale 2x2\! txt_netscape.txt

Identifier Magic: The image header defines this file as a the special IM text image format (EG a "ImageMagick pixel enumeration" file), this is known in computing circles as the files 'magic' or the code string which identifies this file as being this specific file format.

Image Size: The next two numbers define the size of the image contained in this file. Multiplying these numbers together will also tell you how many lines should follow the header to fully define the image.

MaxValue: The last number in the header defines the 'maximum value' of the image data, that is posible. In the above examples this was '255' which is a result of using a 8 bit depth.

The reason it output the built-in "rose:" image at this depth is because it was defined internally using 8-bit values, and as such IM preserved this depth level for the image. See the section on the depth setting for more information.

But you can override the depth setting (up to the limit of your IM's Q or Compile-time Quality setting, by changing the images "-depth". For example here I output the color values as 16 bit (or values from 0 to 65535)...


  convert -depth 16 netscape: -scale 2x2\! txt_netscape_16.txt

At this time you can not set a specic 'Maximum Value' to use in the output file format. You can only define a different value in terms of the current "-depth" setting, making the maximum value equal to 2^depth-1.

Colorspace: The last item in the header defines the colorspace of the image being output. If the image contained any transparency, a final letter 'a' (for alpha) is also appended to the colorspace name, and an extra column of numbers added between parenthesis. Grayscale images will use the 'rgb' format, though all three numbers are the same value.

For example here is the same image using a colorspace of 'LAB' with an alpha channel added!


  convert  netscape: -scale 2x2\! -colorspace LAB -matte txt_cspace_lab.txt

Pixel Data: The pixel lines themselves are reasonably self explanatory. The first two numbers up to the colon ':' is the pixel position, starting from 0. After this the color values fo rthe pixel is given in parenthesis, with anywhere from 3 to 5 numbers depending on the current colorspace for the image. The numbers themselves range from zero to the 'maxvalue' also defined in the header. Spaces are optional so caution is advised when parsing the numbers in parenthesis.

Color Comments: Anything that follows the numbers in parenthesis is regarded as comment. IM will fill in extra information on the pixel color using formats that it can parse as a color argument (See "-fill" manual entry for details of these color specifications).

The color comments are however variable, and mat output RGB() or color names depending of the pixel data given. Also exactly what color comment is provided is highly dependant on the IM version you are using, especially in early IM v6 versions and before. There is no gurantee that this comment area will not change again in the future, so it is best not to rely on it. IM doesn't.

Reading from a "txt:" format file, is also valid. You do not need to define ALL the pixels in the image. In fact you do not even need to have the pixels in the correct order. ImageMagick will just read each of the pixel in turn and 'draw' it onto a blank image canvas. Only the numbers in the first parenthesis is used for this.

The initial blank canvas, is normally cleared and zero'ed image data memory, which results in a opaque-black color for any undrawn pixels. Their is however no gurantees about this on all systems.

Using the "txt:" format is especially useful with the "-unique-colors" operator, which replaces each image in the current image sequence with a new image containing one pixel for each unique color found. When this is output to a "txt:" format file, you get a basic summery of the colors contained in an image (though not their counts, or histogram).

For example here are the colors used by the tree image. As GIF can only use 8 bit numbers, I also chose to output the file the same "-depth".


  convert tree.gif -unique-colors -depth 8 txt:-

There is another alturnative to using the IM "txt:" format using the various NetPBM image file formats. IM by default outputs this format as binary, but you can turn off "-compress" to output a ASCII text version of the NetPBM format. For example.


    convert tree.gif -unique-colors -compress None -depth 8 tree_netpbm.ppm

You may notice that the numbers in the above matches the number in the IM's Enumerated Pixel ("txt:") format. See Resized Gradient for some examples of reading a text NetPBM format.

If you just want the color of a specific pixel you can crop the image down to one pixel, and output it as a "txt:" image.


  convert rose: -crop 1x1+12+26 txt:

Or you can use a special FX Escape Format to output the color in a form directly usable by IM.


  convert rose: -format '%[pixel:u.p{12,26}]' info:

Compressing Images

Under Construction

Talk about file compressions which are part of various image formats.

Different compressions for different image formats.

Especially the JPEG to TIFF compression change needed.

Using or "-compres None" and "+compress" NetPBM text/binary format selection.

The GIF compression and the copyright patent.


Other than using IM to reduce -quality or changing the format to something else
the -compression option is rarely used.  Often it is only used internally by
IM to save images using the same compression the image was read with.


You can of course also compress images, especially text images such as ".txt"
".xpm", ".xbm" images further by adding a ".Z" or ".gz" suffix to the output
filename, and IM will make the appropriate calls to "compress" or "gzip" as
appropriate.

IM will also read these compressed files via the appropriate comamnds.

After that you can refer to the image in the browser as most web servers and
browsers will support sending the compressed file, compressed and uncompress it
for use at the other end.

Compressed XPixmap (xpm) images is also automatically handled by the Xpixmap
library, so you can use that format directly in many X window programs.

Coders and Delegating Image Coding

Under Construction

Coders are dynamic library modules that handle the "format:" aspect of image input and output. They can also be used by users to create special purpose filters.

Delegates are used when no 'coder' module has been provided, and basically asks external programs to convert from the given unknown image format to some other format that IM does know, or is converting directly to.

They are of special importance for very complex 'image' formats such as postscript and PDF.

Input

NOTE: There is a "http:" coder but a "https:" wget delegate.

For example, while Postscript (PS:), and Encapsulated Postscript (EPS:) can be written directly by ImageMagick. These file formats can not be read by IM. Postscript is a full computer language and requires a very complex interpreter to create images from it. As such it is far beyond the scope of IM to handle the reading of this file format.

To solve this IM looks for a external program called "ghostscript" to do the work of converting an PS or EPS format file to some other image format that IM can read easily. Using an external program in this way is called 'delegation' and the program used a 'delegate'.

Basically if you get an error like...

convert: no decode delegate for this image format `...'

Basically means that IM was unable to find the appropriate external program to convert your known image file into a format that IM itself can handle. For Postscript images, that usally means "ghostscript" is not installed, mis-configured, or in an unknown location on your system.

Sometimes it actually will string together multiple delegate programs to read in an image. For example to read a 'HTML' page as an image, it first calls the delegate "html2ps" to convert it to postscript. Then it converts the generated postscript file into a set of mulitple images using "ghostscript".

Of course using two delegates like his can produce other problems due to the complex interactions and bugs that may be present. But in general it works. See below for more detail and specific alturnatives.

Output

Simular things are done when saving to specific highly complex image file formats. For example the AVI video format is far to complex, and has its own quickly evolving development team, that is imposible to replicate in IM. As such a delegate is used to convert a multi-image file format into an AVI video format.

Direct Delegate only Conversion

The delegate system also allows IM to call a external program to convert a image from one format to another format without processing the image itself.

For example if you try convert a 'Adobe Illistrator' file (".ai") (which is a type of Postscript), to EPS (encuplated postscript)...


  convert -density 300   map.ai  map.eps

Then IM will first read the image into memory, and then find that it does not actually modify it.

Because no change was made to the image, and a direct delegate conversion is available, it will call the special "ghostscript" utility "ps2eps" to do the conversion directly. The result however is that the generated EPS is a vector postscript image, rather than a postscript wrapped raster (pixel array) image at the density requested.

You can however force IM to actually read-in and write-out the image, by using the special "-taint" operator, to mark it as being modified, without actually modifying it. As such this will force IM to output a rasterized image data, in the encapulated postscript file.


  convert -density 300 map.ai  -taint  map.eps

Delegate Listings

A full list of external delegates' that IM can use for converting image formats is read from a special system file called "delegates.xml" as well as a personal "delegates.xml" file (see below). If you can find this file it makes interesting reading.

The format of this file hwoever is too complex to do into here, though it is explained in both the system file and the manuals provided both online and with your ImageMagick installation (docs area).

A simplified summery of the delegates and conversions that IM is reading from these files can be printed using...


  convert -list delegate

All delegates are optional, and more than one can be created for a specific conversion. If one delegate is not available, then IM will try the next delegate, until one is found that does work, or it runs out of delegates to try, at which point an error will be produced.

The PDF/PS "ghostscript" delegates are in a special format used internally. IM internally examines postscript format images to attempt to determine exactly how to rasterize the file via the given delegates.
In fact, multiple delegates are present and selected by IM depending on the situation. For example the ghostscript device used ('bmpsep8' verses 'pngalpha') is selected depeding on if "-colorspace RGB" had previously been set or not. For PDF we use the 'ps:color' delagate rather than 'ps:alpha' because the 'pngalpha' ghostscript device only supports a one page/one image and PDF's generally are multi-page.
If you use pdf2png you would need to concatenate all the PNG images into a multi-image format such as MNG, MIFF, TIFF, etc. For an example, see the 'pov' delagate. You might be better off using Ghostscript directly and bypassing ImageMagick if you are attempting to get transparent PDF images.

Creating your own Delegates

You can also add to this list by creating a private "delegates.xml". For UNIX users this file is saved in the ".magick" subdirectory of your home.

If you do create a useful (or better) delegate, then please mail it the development team so that it can be included in the next release of IM.

For security reasons a personal "delegates.mgk" will not override the delegates defined in the system installed "delegates.mgk" file. You can only add unique delegates in ".magick/delegates.mgk" in your home directory, duplicate delegates will be ignored.
Of course if the input format is known internally then of course system delegates are not looked at.

You can however still create your own custom 'input and output' formats to provide delegates for specific purposes, by specifying your own special 'image formats'.

A better video decoder delegate

For example here is a delegate published by Mikko Koppanen, on his Mikko’s blog site. Add this to your personal "delegate.xml" file in ".imagick" directory of your home...

IM can now use the "ffmpeg" program to decode the frames from an MPEG video image. For example.


  convert  "ffmpeg:test1.mpg[40]"  frame_40.png

Creating CR2 Delegate

Use delegates to spawn an external process to convert an image format not natively supported by ImageMagick to one that is or from an image format that is natively supported to one that is not. As an example, we use dcraw to convert the CR2 image format to TIFF. The XML looks like this: <delegate decode="cr2" thread-support="False" command='"/usr/local/bin/dcraw" -T -w -c "%i" > "%o"' /> When you issue this command: convert cr2:image.cr2 image.png ImageMagick determines there is no native support for CR2 so it checks the various "delegate.xml" configuration files and finds a match as presented above. The "dcraw" program is spawned and it converts the CR2 image file to TIFF. ImageMagick understands TIFF so it reads the resulting TIFF image and subsequently converts it to PNG. If you do not fully define the filepath of the "dcraw" executable, IM will search the for the program along the current PATH environment variable, however allowing this could represent a security problem.

Modifying Postscript Delegate for CMYK postscript

See Blig of John for details.

A HTML alturnative

The "html2ps" is not known for its image handling abilities. It has been reported that it basically ignores image size tags, so that it just does not preform very well placing images and text together. There is a technique of running a web browser in a virtual X server to get a more advanced image of the top of a web page, but this is not throughly tested (mail me for more info if interested).

Writing a Multi-Image Sequence

A major problem with saving images, is that ImageMagick works with a ordered sequence (list) of images, not just one image at a time. Because of this IM will attempt to write ALL the images in the current image sequence into the filename given.

If the file format allows multiple images IM will by default save all the images in the current image sequence into that image file. For example if you look at the GIF Animation Basics examples page you will see that it will save multiple image frames into a single image file format to produce a animation.

If the output format does not allow you to save multiple images into the one file, IM will instead generate multiple files. For example, when saving to image formats like JPEG and PNG and so on.

You can also force this behavior on image formats that do allow multiple images per file, such as GIF and PS by using the "+adjoin" output file handling setting.


  convert eye.gif news.gif storm.gif  +adjoin  image.gif

If you look closely at the filenames of the three images generated above, you will see that IM generated images named "image-0.gif" to "image-2.gif".

Previous to ImageMagick version 6.2.0 the output filename of the above would have been "image.gif.0" to "image.gif.2". This resulted in many problems due to the loss of the filename suffix, so was changed to add the image number, before the filename suffix.

An alternative is to add a 'C language printf()' construct "%d" to the output filename. This special string will be replaced by the current image number of each image in sequence.


  convert eye.gif news.gif storm.gif  +adjoin  image_%d.gif

Here we generated the images "image_0.gif" to "image_2.gif", using an underscore rather that the IM default of a dash.

Not only can you use '%d' for a decimal number, but you can use '%x' for a heximedical number (lowercase), '%X' for a heximedical number (uppercase), or '%o' for an octal number.

If you really want a percent character which is followed by one of these letters, then you will need to double the percent character to escape its meaning. That is you will need to use '%%' to ensure you actually generate a percent symbol.

The '%d' in the output filename actually enables the "+adjoin" setting of ImageMagick, automatically.
However while I don't actually need the "+adjoin" in the above, it is probably a good idea to provide it anyway, just so it is clear that you are generating separate images.

This works well for a small number of images, but if you have more than ten images you will get a mix of image with one digit and two digit numbers. And if you have more than a hundred, you get three digit numbers too. When that happens, directory listings will no longer list the saved images in sequence as "image_15.gif" would alphabetically appear before "image_5.gif".

Of course there are ways to fix this. For example using a command line shell expressions like..


  convert  image_[0-9].gif  image_[1-9][0-9].gif  animation.gif
  convert  image_?.gif  image_??.gif  image_???.gif  animation.gif
  convert  image_(?|??|???|????).gif  animation.gif
  convert -scenes 0-122 image_%d.gif  animation.gif

The last (using "-scenes") is the proper IM way of handling a sequence of files, though you need to know the range of number you want to use.

In any case, this is awkward and prone to mistakes, can produce errors if files are missing, and can be dependant on what type of computer system you are using. Better to avoid this problem altogether.

If you are familiar with the 'C' language (look up the UNIX system manpage for 'printf') then you will probably know that if you use something like "%03d" you will always get 3 digit numbers (with leading zeros) for the image sequence frame number. The image names would in that case be "images_000.gif", "images_001.gif" and so on.


  convert  eye.gif news.gif storm.gif  +adjoin  image_%03d.gif

Using this method, the images will not only number correctly, but will also list alphabetically correctly, making image file handling a whole lot easier.

I thus recommended you add a '%03d' or whatever is appropriate, to the output filename whenever you plan on writing multiple images, as separate image files.

Written Scene Numbers

If you want the image sequence to start at '1', instead of '0', and don't want to rename all the resultant image files, the simplest solution is to prepend a 'junk' image on the front of the sequence to be written.


  convert  null:  eye.gif news.gif storm.gif  +adjoin  image_%01d_of_3.gif
  rm image_0_of_3.gif

You can, of course, use "+insert" to do this after your image processing. This is not a particularly nice solution, but works, and is simple, and backward compatible with the older major versions of IM.

As of IM version 6.2 you can use the "-scene" setting to set the starting number for the current image sequence.


  convert  eye.gif news.gif storm.gif  +adjoin -scene 101 image_%03d.gif

Which produced the image files "image_101.gif" to "image_103.gif".

Writing an Image, Multiple Times

While on the subject of writing images, it is possible to write an image from the middle of a sequence of image operations, using the special "-write" image operator.

This is very useful when you like to output an image multiple times with variations in image processing, such as thumbnail size. It can also be used for grabbing an image for debugging your IM command.


  convert cyclops.gif  -fill lightsteelblue  -draw 'color 0,0 floodfill' \
          \( +clone -resize x128  -write  cyclops_lrg.jpg +delete \) \
          \( +clone -resize x96   -write  cyclops_big.jpg +delete \) \
          \( +clone -resize x64   -write  cyclops_med.jpg +delete \) \
                    -resize x32           cyclops_sml.jpg

As you can see we can use the Image Sequence Operators to process a 'clone' of an image, write out the result, then delete and backtrack back to the original source image, repeating the process as many times as you need.

In this particular case it means I did not end up resizing the same image over and over, but each time re-started from the source image, and thus preventing the image degrading due to processing it multiple times. It also meant I could have just as easily generate the smaller images first, then the larger images after that, without problems.

Note that "+clone" does not actually duplicate the image data! IM uses a reference-counted cloning process which only copies the image pixels when they are updated. As such only enough memory to hold the original image and the new resized image is actually used, in the above process. It also makes "+clone" very fast.

Really Massive Image Handling

For handling any sort of large image it would probably be better for you to use a Q8 version of ImageMagick, whcih has half the memory requirements of the higher quality Q16 version. Check your IM's compiled Q level using "identify -version".

For medium sized images you can attempt to use "-limit" to increase the processing limits (for example processing "

-limit area 8192 -limit memory
8192

"), so as to try to avoid IM caching the image data to disk. However your system may reject large memory requests and still force IM to cache the image to disk (about 1000 times slower).

To see if IM is using disk cache for the image processing you can use "-debug cache" to monitor that action.

However if you are planing to process really large images you can make sure IM does not use up all the computers memory, and slowing down the processing of other programs (by spending all its to shuffling between memory and disk swap) simply by asking it to immediately use temporary swap disk files.

For example this is a nice way of processing a very large image over a long period of time without stopping you from using your computer for other things. Basically it forces IM to cache everything to disk.


  env MAGICK_TMPDIR=/data nice -5 \
    convert -limit memory 32 -limit map 32 \
            huge_9Gb_file.psd  -scene 1 +adjoin layer_%d.png

Of course this assumes that "/data" has enough file and disk space to handle the images memory requirements.

If you have many operations to perform on the same source image and you have plenty of disk space you can use the MPC image format which is expensive to create but has near zero overhead when loading...


  convert mybigassimage.jpg mybigassimage.mpc
  convert mybigassimage.mpc   -resize 50%  resized.jpg
  convert mybigassimage.mpc   -rotate 90   rotated.jpg
  ...etc...
  rm -f mybigassimage.mpc mybigassimage.cache

This will let you read a very large image multiple times with a minimal cost, and memory usage. Basically the format consists of two files, a informational ".mpc" file, and a direct memory paged copy of the image in a ".cache". Of course you need to clean up two files with this format then finished.

This method is designed so that IM does not have to reparse the image format to use it making it next to instant for making it available to IM in memory, though when processing it still needs to read it from disk rather than memory.

If you plan to process a very large MPC copy of the image, you must extract or crop a smaller section of the image for actual processing. This is because just about any operation performed on an image, will generally result in a new in-memory copy being made of the result, so an initial crop is a very good idea.

If you have the memory you can also try to use a 'memory disk' such as a 'TMPFS' type filesystem commonly used on Solaris systems. Be warned however that filling that type of disk also fills your computers memory.

If you have tried this method, please let me know how it went and how successful you were.

Processing Images in small sections

You can use the MPC method above to crop out various sections from a source image for further processing. However that still means you need to read in and write out the image in MPC format.

IM has also evolved a simplier pipeline processor for images called "stream". This program has a limited set of image operations that are designed to only process images one scan line (row of pixels) at a time. As such only enough memory to hold a single line of pixels is used when processing images in this way.

For example this allows you to extract a smaller area of a very large image for further processing, without needing to read in the whole image into memory first. However the output of "stream" is raw RGB image values, so some post-processing is recommended.


  stream -map rgb -storage-type char -extract 600x400+1900+2900 image.png - |\
    convert -depth 8 -size 600x400 rgb:- tileimage.png

You don't have the save the ouput to a file but can continue processing the smaller image directly. For example...


  stream -map rgb -storage-type char -extract 600x400+1900+2900 image.png - |\
    convert -depth 8 -size 600x400 rgb:-  ...more_processing_here...  tile.png

This will only process the 600x400 pixel image extracted without reading in the whole larger image first.

Note from Peter V <peter.v@pv2c.com>: In my experience is the approach of using "stream" for cutting 800MB PNM files the fastest compared to use of MPC files, or using "convert -crop".

Additional note from Jenny Drake < jennydrake @ lineone.net >

You may also like to look at the non-IM alturnative of "Vips" and "nip", developed by the National Portrait Gallery in London, which is designed to work on very large image files (generally TIFF) with low specification computers. "Vips" is the underlying engine and "nip" is the gui. Works on Linux, Windows and sometimes on Mac.

The older 'NetPBM' image processing suite, generally also processes images in a pipeline programming style, just as "stream" does. It is an inherent part of that library, but with the serious limitation of using a specific file format for the image.
That library is also much more limited in what it is able to do in terms of image processing, making image processing scripts much more complex.
If anyone out there does compare IM with NetPBM, say for 'crop' or 'extract' from a large image, can you please let us know what you found.

ImageMagick v6 Examples -- Image File Handling

Image Formats Summary

Saving Images

Reading Images

Read Modifiers

Importing Images from special sources

Special File Formats (specific to IM)

Compressing Images

Coders and Delegating Image Coding

Input

Output

Direct Delegate only Conversion

Delegate Listings

Creating your own Delegates

A better video decoder delegate

Creating CR2 Delegate

Modifying Postscript Delegate for CMYK postscript

A HTML alturnative

Writing a Multi-Image Sequence

Written Scene Numbers

Writing an Image, Multiple Times

Really Massive Image Handling

Processing Images in small sections

ImageMagick v6 Examples --
Image File Handling