- Index
ImageMagick Examples Preface and Index
API and other IM usage methods
Security Issues
Hints for Better ImageMagick Shell/PHP Scripts
Why do you use multiple "convert" commands
Making IM faster (in general)
Creating linux RPMs from SRPMs
The Command Line Interface (CLI) of ImageMgaick which this examples deals with
is only one method by which you can use, modify and control images with the
core library of ImageMagick functions (MagickCore). It is basically the
'shell' API interface. There are lots of other Application Programming
Interfaces (API's) which you can use more directly from many programming
languages, see
ImageMagick APIs.
Here I look at ways of improving your IM scripting and programming,
differences between Windows and Unix scripting, and look at basics of using IM
from other API's and programming languages.
APIs and other IM usage methods
Windows Batch Scripts
Window users should specifically note that...
- Backslashes '
\' appearing on the end of the shown example
lines represents a 'line continuation' which appends the next line to the
same command line sequence. Just append all the lines onto one line and
remove those backshashes.
- Backslashes '
\' are not needed to escape parenthesis
'()' or exclamation marks '!'.
- Double quotes '
"' may need to be used (carefully) in place
of single quotes ''' so that IM commands to work correctly.
Watch out for quotes within quotes such as in -draw commands. You can
however use single quotes within the double quotes as IM itself
understands them.
- The special character '
>' (used for resize geometry)
needs to be escaped using '^'. For example
"-resize 100x100^>"
- Simularly the 'internal fit resize' flag '
^' needs to be
doubled to become '^^'.
- Also in DOS batch files any percent '
%' characters may need
to be doubled '%%' to work correctly.
- Finally "
convert" is a windows command, it is suggested you
rename the IM convert command to "imconvert" to distinguish
the two commands. You can't rename the system command, as a service pack
would just restore it.
Thorsten Röllich pointed out that you can use a
FOR ... DO
sequence in a windows batch script to pass text output of one
"
convert" command to another. For example...
FOR /F "tokens=*" %i IN ('convert ...') DO convert ... %i ...
|
Basically the output of the single quoted "
convert" command
in parenthesis is saved into the special batch script "
%i"
variable and substitued at the "
%i" location in the second
"
convert" command. Normally percent characters need to be
doubled in window batch scripts, this is the exception.
This could for example take the output of the convert options "
-format
%wx%h info:" to a "
-crop" operator in the second convert
command.
PHP
(IM commands from "system()" functions)
PHP users have three ways of using ImageMagick, The "
imagick
PECL interface, the "
MagicWand" interface, and a seeming common
method of making system calls directly to the command line interface. As IM
Examples is mostly about using the command line, that is the form I will look
at here.
PHP using Shell Commands
The best source of information on using this technique is the IM forum user
'
Bonzo' and his web site
RubbleWebs.
The following is the recommend procedure for initial tests of an ISP command
line IM interface, assuming you do not have direct command line 'shell' access
on the remote system. That is you can only upload web files for execution.
So the first thing we need to do is try and find and run the 'convert' command.
<?php
header('Content-Type: text/plain');
system("exec 2>&1; type convert");
system("exec 2>&1; convert -version");
system("exec 2>&1; convert -list type"); <!-- before IM v6.3.5-7 -->
system("exec 2>&1; convert -list font");
?>
|
Upload the above PHP and look at it via the web.
This will run three commands to see what is present. If convert on the command
PATH, and if so where is it. What version is it. And what font does IM thing
it has access to.
If you only see errors, then the "
convert" is not on the command
line path, and your ISP provider did NOT initialise the web server PATH
properly.
If this is the case you will need to find out exactly where it is located and
use something like this for you PHP scripts, which makes your script less
portable.
<?php
$im_path="/opt/php5extras/ImageMagick/bin"
header('Content-Type: text/plain');
system("exec 2>&1; $im_path/convert -version");
system("exec 2>&1; $im_path/convert -list type");
system("exec 2>&1; $im_path/convert -list font");
?>
|
If you get "ldd" library errors, the LD_LIBRARY_PATH is wrong, and the ISP has
definitely fallen down on the job, and you need to report the error, and
have them fix the web servers LD_LIBRARY_PATH environment variable.
After that try some of the simpler examples from IM Examples and try to get
them working. EG: output the IM 'rose' image as a JPEG.
<?php
header( 'Content-Type: image/jpeg' );
system("convert rose: jpg:-");
?>
|
Or try one of the fonts listed from the first test page. On my Solaris Test
Server I noticed that the '
Utopia' font set was available so I
can try to create a label with it.
<?php
header('Content-Type: image/gif');
system("convert -pointsize 72 -font Utopia-Italic label:'Font Test' gif:-");
?>
|
Watch the extra quotes
Note that typically the IM commands are wrapped by an extra set of quotes
(usually double quotes), as such care must be taken to remove
'line-continuation' backslashes, and to double up other backslashes.
For example... This command
convert -background none -fill red -gravity center \
-font Candice -size 70x46 caption:"A Rose by any Name" \
\( rose: -negate -flip \) +swap -composite \
output.gif
|
Will become something like this PHP equivalent...
<?php
header('Content-Type: image/gif');
$color="red";
$image="rose:";
$size="70x46";
$string="A Rose by any Name";
passthru("convert -background none -fill '$color' -gravity center" .
" -font Candice -size '$size' caption:'$string'" .
" \\( '$image' -negate -flip \\) +swap -composite" .
" gif:-" );
?>
|
Note how I still split up the lines to make the image processing sequence
easier to follow, but using appended strings rather than a shell line
continuation. Especially note the extra space at the start of later lines.
And double up the other backslashes that was present in the original command.
Alternatively you can protect those options by using single quotes.
I also use PHP variables to better adjust the PHP script, and control the
results, but insert those options using single quotes to protect them from
further modification by the shell. Watchout for single quotes within those
inserted strings.
Also you can perform multiple shell commands within the same system call
string. In fact a single system call can contain a complete shell script if
you want! So you can do shell loops and multiple commands (with cleanups) all
in the one system call. Something not may people realise.
Basically if you are careful you can make good use of the mathematics provided
by PHP, and the scripting abilities of the shell. All at the same time.
Just watch the quotes.
I recomend you at least read the PHP manuals on
PHP exec(),
system(), and
passthru() and understand there
differences between them.
For various examples of calling ImageMagick commands from PHP see
Rubble
Web, Writing IM code in PHP which describes about four different
techniques.
The more secure method...
Also if you what to avoid the shell parsing the arguments and the need for
most of the quoting requirements, (you separate the arguments yourself), such
as for security reasions, or specific user input, then use the
pcntl_exec() PHP function.
This basically avoids all the quoting or other shell escaping, and is thus
much more secure that using a 'all in one string' type command. Note you will
also
However you will also have to fork a sub-process for the funtion to run in as
well, ir it replaces the PHP being executed. This can thus get fairly complex.
It is just a shame that PHP has not provided any other 'shell-less' command
execution function, such as provided in other languages like perl.
PHP PECL 'IMagick'
To test if the
PHP PECL
imagick module is actually working upload a simple test
"
image.jpg" image and this PHP script to the same web assessable
directory.
<?php
$handle = imagick_readimage( getcwd() . "image.jpg" );
if ( imagick_iserror( $handle ) ) {
$reason = imagick_failedreason( $handle ) ;
$description = imagick_faileddescription( $handle ) ;
print "Handle Read failed!<BR>\n";
print "Reason: $reason<BR>\n";
print "Description: $description<BR>\n";
exit ;
}
header( "Content-type: " . imagick_getmimetype( $handle ) );
print imagick_image2blob( $handle );
?>
|
PHP 'MagickWand'
You can check if the
PHP
MagickWand module is part of the PHP installation (and switch to
command line or other fallbacks) using...
<?php
if (extension_loaded('magickwand')) {
echo "PHP MagickWand is available!";
}
?>
|
But to check that it is actually working properly, upload some test
"
image.png" and this script...
<?php
$image = NewMagickWand();
if( MagickReadImage( $image, 'image.png' ) ) {
header( 'Content-Type: image/jpeg' );
MagickSetImageFormat( $image, 'JPEG' );
MagickEchoImageBlob( $image );
} else {
echo "Error in MagickReadImage()";
echo MagickGetExceptionString($image);
}
?>
|
No guarantees with the above, though more feedback welcome. I do not generally
program in PHP, but used the above for testing a SunONE-PHP5 test installation
(with all three methods: command-line, magick, MagickWand).
Complex PHP scripts...
If you need to generate and output both HTML and IMAGES, consider designing
your PHP script so that separate HTML requests or input options, generate the
various parts you need on your web document, from either the same, or
different PHP scripts.
That is a top level PHP script can output HTML with appropriate <IMG>
tags, that call itself (or another PHP script) with appropriate options, to
create or modify the images displayed on the first top level PHP script. This
is in what a lot of photo album, and graphing PHP scripts do. All controlled
by the GET, and PATH_INFO extensions to the URL calls. Note that you can not
use POST within an IMG tag.
By doing things in this way you should be able to completely avoid the need to
both generate, save, and clean-up, temporary images for PHP generated web
pages. A solution that is full of problems, such as resource limitations and
garbage collection, making it a very bad programming technique.
Security Warnings
When writing a script for public use, especially a web-based PHP script where
ANYONE in the world could be running it, it is vitally important to check
everything that comes from a unkown (or even a known) user. And I mean
EVERYTHING, from arguments, filenames, URLs, and images too.
Until you verify some input argument, that argument it could contains letters,
numbers, spaces, puncutation, or even 'null' and control characters.
It does not matter that you are using some web controled input form. A
slightly knowledgeable person can easilly call your PHP with his own arguments
without using that input form at all. An don't believe they won't do it,
robots are out there, reading input forms and creating there own 'hacked'
argments to try an break into random scripts.
Meta-characters in file names
As a security issue, you should watch out for filenames that contain
spaces, quotes, punctuation, control-characters, or other meta-characters as
both IM and Shells may try to expand them.
The problem is that a file called '
*?@${&) .jpg' is actually a
perfectly legal filename under UNIX, but a LOT of programs will have trouble
handling it if that program (like shell and IM) also do filename expansion.
Even when you quote the file name correctly some filenames can still break
your quoting.
As a security measure it is often a good idea to error and abort if a filename
has some unknown or unusual characters in it (like white space, control, or
other meta-chars). Before passing such a filename to a shell command, or IM.
It is better to be lightly to restrictive when security is involved than
allowing something bad through.
Hints for Better ImageMagick Shell/PHP Scripts
These were some basic script programming points I made about a contributed
shell script that was sent to the IM mail list for others to use. I originally
sent these to the author privately (and who will remain anonymous), for which
he was grateful.
They are not all IM specific, but should be applied anyway, as standard
programming practice. Especially if you plan to have someone else, use, look
at, and/or bugfix your program or script. It will in turn make your script
more useful.
- Place the 'help' or 'doc' at the top of scripts and programs.
This makes it lot easier for other people to figure out what a particular
program does without needing to install or run the script. I myself will
often junk a program that has no such clear comment on what it is for
rather than try to compile or run such an unknwon script.
In fact I have seen huge projects where the first README file does not
even say what the huge and complex project does! The programmer just
assumes that if you downloaded it you must know what it does!
Also make sure that a 'bad option' such as '-?' will print a synopsis of
not just the options, but also a quick summery of what the program does,
or where to find such help. Do not just point to a remote web site, that
may disappear 10 years down the track!
For an example of a script that prints its own 'opening comments' see the
"jigsaw" script in the IM scripts area.
Perl can use POD for self documentation (See the perl
"
Pod::Usage" module). For example see the contributed dpx_timecode.pl script.
Having the help as a 'here file' in the first sub-routine is also
acceptable, and works for most languages. But have that subrountine AT THE
TOP, not at the bottom or middle of the script or program.
- Make sure you clean up your code, remove old obsolete code and comments,
making things as neat an tidy as you can. Be consise, with simply
commented stages (if possible).
Again see the "jigsaw" script.
- Make sure temporary files are cleaned up at the end.
Use a "trap" shell command to remove them on file exit or interupt.
You can of course re-use a single temporary files multiple times, so you
don't need very many, especially with IM v6 convert commands.
Again see the "jigsaw" script, and search
for "trap".
- Also not everyone uses system XXXX, though it may seem like that to you.
Don't refer to specific system requirements, or methods on rectifying the
problem. What works for you may be completely in-appropriate for their
system and setup. They may not even have internet access!
Just say that the "
convert" command from ImageMagick, was not
found.
If you want add installation requirements, or suggestions, add it as
part of separate more extensive documentation.
- Do check that the IM being used is a high enough version, or add backward
compatible changes. I purposefully give 'version warning' notes throught
IM examples as to when various special features were added for this
purpose. It makes it easier when creating scripts with a single minimal
version check.
Here is one simple way you can get a single version number for testing
purposes in a shell script. It extracts the 4 version numbers, and
inserts the right number of zeros to make each number 2 digits, producing
a simple 8 digit number.
IM_VERSION=`convert -list configure | \
sed '/^LIB_VERSION_NUMBER /!d;
s//,/; s/,/,0/g;
s/,0*\([0-9][0-9]\)/\1/g'`
|
For example IM v6.3.5-10 will generate "06030510" while
the very next release IM v6.3.6.0 produces "06030600".
A PHP version of the above is available from RubbleWeb, Font List examples page.
The resulting string can be tested, using either a simple numeric or
string test, to discover if the avialble ImageMagick version is modern
enough for what your script is tring to do. For example...
if [ "$IM_VERSION" -lt '06030600' ]; then
echo >&2 "The perspective distortion operator is not available."
echo >&2 "Sorry your installed ImageMagick is too old -- ABORTING"
exit 10
fi
|
Also note how I added an extra note, either as output to the user, or as a
programming comment, about what special feature(s) the version check was
for. Otherwise you may later forget why that specific version (or higher)
was needed.
You can also modify the behaviour of IM for specific versions.
For example say I want to get a list of available fonts. Before IM
version v6.3.5-7 the "-list" setting "type" returned the 'known font'
list. With later versions, you need to use "font" setting
instead. So here I can do a version check to use the right setting to
get the information needed.
if [ "$IM_VERSION" -lt '06030507' ];
then font_list="type";
else font_list="font";
fi
avail_fonts=`convert -list $font_list | cut -d\ -f1 |\
egrep -v '^($|----|Path:|Name$)'`
|
WARNING: a string starting with '0' in PERL could be interpreted as being
an octal number!!! However compareing two octal numbers will still come
out correct. Caution, and checking of the version test is advised.
- You can also make use of the "
-list" information output to check if some special feature
has been added to the currently installed ImageMagick.
convert -list distort | grep 'Arc' >/dev/null 2>&1
if [ "$?" -ne 0 ]; then
echo >&2 "Arc distortion method not available."
echo >&2 "Sorry your installed ImageMagick is too old -- ABORTING"
exit 10
fi
|
However be warned that often a new method, like "-distort Arc" could appear
during IM development before it is properly ready for general use. As
such a version check may still be the better idea.
This is why in IM Examples I try to note (look for
symbols) the IM version
when a new feature have become stable enough for general use.
- Try not to rely on too many external programs, or only use them only if
they are available (with posible alturnatives). Other people probably
will not have that program, or prefer to use something else. If a program
use can be optional, make it optional, either under user control, or
automatic use if found. Do not make it a forced requirement, if at all
posible.
For example you could make use of "
pngcrush",
"optipng", "pngnq" to compress PNG better than
IM would normally provide (IM is designed to be general not specific).
Also "gifsicle", "intergif", or other for LZW
compression optimizers for GIF animations, have good and bad points. Just
don't make it a definate requirement for a scripts working.
As a practial example, an older version of the "gif2anim" did not use the ImageMagick
"identify" to lookup GIF specific meta-data, but relied on a
patched version of "giftrans". This requirement was later not needed due to
improvements in the ImageMagick "identify", so I removed
that requirement to make the script more widely usable.
IM itself has a LOT of optional requirements, such as
"ghostscript" for postscript and PDF document reading, or the
"librsvg" for correct handling of SVG vector images. IM
itself is quite happy for them not to be available, and does not make
these a definate requirement. It only needs them to be present if you
want to process images of those formats.
- Don't use very very long single lines. Especially for complex 'convert'
commands. Split them up using '
\' at end of lines at the
appropriate processing stages. This makes it a lot easier to read, print
and follow the more complex commands.
For example, do one major operation or stage per line, create new image,
modify image, merge with other images, etc. Place all operational
settings needed for a specific operator just before that operator.
Think of the command line as a sequence of setting/operation stages.
Indenting various stages or add (near) empty lines, can make it clearer
the major stages of operations in the processing. See the montage font table as an example of this.
Using parenthesis can help, even if
you don't actually need them. See Appended Array example.
The clearer the separation of operators, the easier a complex image
process is to follow.
I use these techniques all over IM examples, so just look around!
- Let the user decide on the image formats.
ImageMagick is primarilly a image converter and able to use lots of
different formats. It can output to the screen, postscript, printers, or
pipe the image into another command for further processing. Don't limit
the user to a specific format.
For example the "jigsaw" and "gif_anim_montage" scripts allow
the user to specify any input or output image. That way users can pipeline
images into or out of that script for further processing to other programs
and scripts.
For example, I often use a command like...
gif_anim_montage animation.miff x:
|
to display the result of the script on my display screen, rather than save
it to a file. I also did not limit it to input from a GIF animation file,
or limit output to a GIF, PNG, or JPEG image format.
In fact IM can read from either files, pipelines, the current display, or
even from the World Wide Web using a "URL:" or
"HTTP:" input format.
- Read input images only ONCE!
If the user provides a pipeline file name or a URL, you do should not try
to read those images more than once, or things can go bad. Use a
temporary file, cloned images, or "
MPC:" save, if you have
to refer to the image multiple times. If you can handle a multiple image
sequence from the pipeline, even better.
Again see the "jigsaw" script, for an
example of saving input images into a temporary file, when you are forced
to process the input image multiple times, or in a different order than
what the program arguments imply.
These things basically gives the user using your program more freedom to do
what THEY want rather than what YOU want. Don't limit them or yourself by
making assumptions on what the script will be used for.
PS: One of my main expertise is in UNIX script writing, over lots of different
architures and 'flavors' of UNIX, LINUX, and other UNIX-like systems, with
more than 15 years experience behind me. I should know what I am talking
about with regard to the above.
Why do you use multiple "convert" commands
- Willem on Wed, 25 Oct 2006 wrote...
-
I was wondering; sometimes I see in your examples you're invoking Convert
more then once to obtain the desired result. In general, I would expect
that invoking convert more then once isn't needed; it should all be
possible in 1 invocation (but the command would be more complex then). Do
you agree with this statement?
I agree totally. You should be able to do all your processing in one single
step.
I use multiple commands for a number possible reasons. Typically in the
example pages, I do it so I can get and display the intermediate image result,
so as to better demonstrate the intermediate processing stages that are
involved. Often later in the same example area I repeat the process but using
a single command. As such in principle, yes a single command can do all image
processing.
The exception to this is in cases where I need to extract information and
later insert that info into another command. An example of this is the
Fuzzy Trim technique which requires you to
extract the results of a trim on a blurred copy of the image. This result is
then used to crop the original image. I also did this in the update to
Thumbnail Rounded Corners example, where I
used IM itself to generate a draw command using an images size for the next
command.
However there is a
proposal that will
allow options to be generated from image that have been previously read into
memory.
You are most welcome to combine the image processing techniques all into a
single command. I do this all the time myself.
In scripts, such as the '
jigsaw' script (see
Advanced Techniques,
Jigsaw Pieces) I
commonly end up using multiple commands for a different reason -- optional
processing. That is the user may or may not what a particular step, so I used
a separate command for each stage of processing which is executed based on the
options user provided the script.
In such a case a temporary file is basically unavoidable. However I typically
only need at most one or two temporary images, and each step processes the
image back into the same temporary filename, for the next optional processing
step to continue with. In this can a
MPC file
can speed up the reading of intermediate files to a near instant, though this
file format actually needs two files to save the image.
EG: convert /tmp/image1.png ..operations.. /tmp/image1.png
And finally you may need to change your processing style based on the results
of previous processing steps.
For example in image comparisons, I often need to change techniques based on
the images being compared. Comparing a diagram or cartoon can require vary
different techniques to say a real image.
If multiple commands is becoming a problem, perhaps it is time to go to an
API interface such as
PerlMagick, where multiple image sequences can
all be held in memory so as to avoid unnecessary disk IO.
Making IM Faster (in general)
There are many ways of making IM work faster. Here are the most important
aspects to keep in mind.
- Use a single "
convert" command if possible. This saves on
having to create temporary files, or pipelines between commands. meaning
IM does not have to encode/decode and do slow disk I/O on the images.
Of course sometimes you still need to use multiple commands, to allow
for optional image processing steps.
- Shell scripts are inherentally slow. It is interpreted, require multiple
scrips and extra file handling to disk. This is of course better thanks
to the new IM v6 option handling, allowing you to do a large number of
image processing operations in a single command. Even so you can rarely
can do everything in a single "
convert" command, so you often
have to use multiple commands to achieve what you want.
As such an API like perl, ruby, or a PHP magick module is faster as it
removes the all the interpretation aspects of both the shell, and the IM
command line API, as multiple setups of the dynamic library initialization
IM uses.
- An API also can hold hold all the images, or even multiple lists of images,
for the lifetime of the program (as long as you have enough memory for
it). This means you can chop and change what images are being worked on,
without the need to shuffle and juggle the images on the command line.
- A lower level API can use multiple threads of processes. That means you
can have the computer do two or more separate threads of image processing.
This however requires some very advanced programming skills.
- IM Q8 (or 8 bits per color value, 3 to 4 bytes per pixel) is a lot faster
(3 to 4 times faster) than the IM Q16 default, with its higher color
resolution. If you don't need a Q16 for your images, perhaps you should
replace your IM with a Q8 version. Be warned however that only using 8 bit
internal quality can effect the overall image processing as intermediate
images loose information. See HDRI for the
opposite of this.
- GIF images is slow. IM has to work hard to color reduce (quantize)
the colors of an image to fit into the limited colors of the file format.
Even then you often need to do extra work to get it right, especially for
GIF animations. PNG and JPEG are faster but at a cost of size for PNG,
and loss of quality for JPEG.
- Pre-preparing and caching images such as backgrounds, overlays, frames,
masks, or pre-generating color lookup tables, distortion maps, templates,
masks, etc. all can make large differences in your processing time. Think
about what can be done before hand. A large library of pre-generated
images can be a lot faster that trying to create the images as needed.
See also the use of "MPC:" for intermediate
and cached images. These are memory mapped images on disk, and
essentially have zero read time, but are useless for anything else.
- Avoid using FX, The Special Effects Image
Operator, if you can use Alpha
Composition, the simpler Evaluate,
Simple Math Operations, or other techniques instead.
- Use smaller sub-images, or regions, for complex processing of small areas.
For example, find and extract the eyes of an person, before masking and
recoloring, can result in a huge speed increase than processing the whole
image and only using a small masked small area for changes.
- Avoid using large sized Image Blurs
when a few smaller ones may be faster. Same goes for other Convolution Operators such as Shadow.
- For Really Massively Large Images that
have to be processed from disk, it may be better to process them in
smaller chunks.
- When reading images, it is better to use a Read Modifier to resize, or crop
them, as it uses a smaller memory requirement.
This in turn prevents 'disk
thrashing' (which makes computers VERY slow), when lots of large
images are involved, such as when generating Montaged Directory Indexed, or other multi-image collages.
Building ImageMagick RPMs for linux from SRPMs
You do NOT need root to actually build the RPM's though you do need root
to install the RPMs.
First get the latest source RPM release from
Linux Source
RPMs.
Create a working temporary directory in which to build IM
tmp=/tmp/im_build; rm -rf $tmp; mkdir $tmp
|
Now build the RPMs... All the defines restrict the build to the temporary
directory. If you remove all the defines the RPMs will be built in the system
/usr/src area which is usually root owned.
nice rpmbuild --define="_sourcedir $tmp" --define="_specdir $tmp" \
--define="_rpmdir $tmp" --define="_builddir $tmp" \
--nodeps --rebuild ImageMagick*.src.rpm
|
Get the just built RPMs for the magick core and PerlMagick...
cp -p $tmp/*/ImageMagick-[6p]*.i386.rpm
|
Clean up the build areas...
rm -rf $tmp /var/tmp/rpm-tmp.*
|
Now you can install the RPM packages you build specifically for your linux
system... You will need to be root for this...
rpm -ihv --force --nodeps ImageMagick-*.i386.rpm
|
To upgrade an existing one do this
rpm -Uhv --force --nodeps ImageMagick-*.i386.rpm
|
To later remove IM, do this
rpm -e --force --nodeps ImageMagick\*
|
Warning, other packages may need an IM installed, so if you remove it update
your computer system to install the, default old version supplied by your
linux system. This generally involved using a GUI Update package or "yum".
Enjoy.