v7.1.2-34 Latest Stable Release

Distributed Pixel Cache

Learn how ImageMagick leverages a distributed pixel cache to offload memory and disk requirements across remote servers for processing large images and sequences.

quickstart.sh
# Install via your favorite Package Manager
$ brew install imagemagick
# Convert PNG to WebP with compression
$ magick input.png -quality 85 output.webp
# Resize image maintaining aspect ratio
$ magick input.jpg -resize 800x600^ output.jpg

A distributed pixel cache extends the traditional single-host pixel cache across multiple remote servers. This allows storage and transactional capacity to scale significantly, making it ideal for processing massive images or large image sequences when local resources are constrained.

When local resource limits are reached during image read or processing operations, ImageMagick automatically contacts one or more designated remote pixel cache servers to store or retrieve pixel data seamlessly.

Security & Setup

To establish communication between client and server nodes, first configure a shared secret in your policy.xml security policy configuration file:

<policy domain="cache" name="shared-secret" value="passphrase" stealth="true"/>

When the distributed pixel cache transmits image pixels, it secures the data through two primary mechanisms. First, it scrambles the pixel data, rendering it as random digital noise to anyone attempting to intercept or spy on the network. Second, it applies a tamper-proof cryptographic signature, ensuring the data originated from a trusted source and has not been altered or forged in transit. However, these built-in protection mechanisms are not cryptographically strong, meaning they remain vulnerable to sophisticated attacks and should not be relied upon for high-security environments. For truly secure communication across networks, you may need to implement dedicated cryptographic security measures such as encrypted tunnels or secure loopback configurations:

  • Restrict to Loopback: If distributed caching is only used across processes on the same machine, bind the server strictly to 127.0.0.1 to eliminate adjacent network visibility.
  • Enforce Encrypted Tunnels: For multi-host deployments, never expose the plaintext TCP port directly. Route traffic through an encrypted, authenticated tunnel (such as WireGuard, IPsec, or an SSH port forward) to provide confidentiality and integrity at the network layer.

Usage Examples

Single Remote / Localhost Cache

Start a single distributed pixel cache server and process an image utilizing defined resource limits:

$ magick -distribute-cache 6668 &  # start on localhost
$ magick -limit memory 1GiB -limit map 2GiB -limit disk 4GiB \
  -define registry:cache:hosts=localhost:6668 \
  myhugeimage.jpg -sharpen 5x2 myhugeimage.png

Restrict to Loopback (127.0.0.1) Example

Bind the server strictly to 127.0.0.1 to eliminate adjacent network visibility when processes run on the same machine:

$ magick -distribute-cache 127.0.0.1:6668 &  # bind strictly to loopback
$ magick -limit memory 1GiB -limit map 2GiB -limit disk 4GiB \
  -define registry:cache:hosts=127.0.0.1:6668 \
  myhugeimage.jpg -sharpen 5x2 myhugeimage.png

Multi-Server Cluster Cache

Deploy pixel cache servers on multiple hosts to distribute storage load:

$ magick -distribute-cache 6668 &  # start on 192.168.100.50
$ magick -distribute-cache 6668 &  # start on 192.168.100.51
$ magick -limit memory 1GiB -limit map 2GiB -limit disk 4GiB \
  -define registry:cache:hosts=192.168.100.50:6668,192.168.100.51:6668 \
  myhugeimage.jpg -sharpen 5x2 myhugeimage.png

When multiple hosts are listed, ImageMagick contacts servers in a round-robin fashion to distribute load across the cluster. Smaller images or operations below local -limit thresholds are allocated directly on the host desktop.

Performance & Compatibility Considerations

Network Impact on Performance

Shuffling pixel data over network sockets introduces latency. Algorithms accessing virtual pixels (such as -sharpen) may perform up to 3× slower than algorithms operating solely on authentic pixels (such as -negate) due to increased traffic.

Strict Compatibility Checks

Clients can only connect to matching cache servers. Compatibility requires identical library interface versions, quantum depth, HDRI status, OS word size, endianness, and shared secret passphrases. An exception is thrown if attributes mismatch.