Concurrent Processes Executed on the Same File Lead to Corrupt Images
Posted: 2017-08-10T01:31:22-07:00
Foreword: Because we are using TYPO3, here is the problem I posted on Forge, the TYPO3 Issue Tracker: https://forge.typo3.org/issues/81989
To begin, I do not know much about the inner workings of ImageMagick. I found out about it through using TYPO3 to process images. Thus, I am not sure if this is considered a bug.
Problem: When large images (>= 30MB) are converted simultaneously by multiple processes, we are often left with broken JPEGs. We can emulate this behavior if page (edited by the backend containing a fresh image that has yet to be converted into various sizes, usually at least 5 different versions for various viewports) is accessed by two browsers at the same time. If the file has not been processed yet, this will happen at runtime, which causes both instances to process the image concurrently. The conversion process then breaks and the file is incomplete. After doing a little reading, I thought ImageMagick employs some sort of mutex to prevent concurrent access to the file, but perhaps I don't quite understand how that works.
I would first like to know if this behavior could be due to an improperly configured installation of ImageMagick. Secondly, if ImageMagick is properly configured, but steps could be taken to prevent concurrent processing of a file without touching the TYPO3 Core, I would like to know what these are.
Current environment:
Server: Apache 2.4.23
PHP: PHP 7.0.20 Nonthread safe
CMS: TYPO3 7.6.21
ImageMagick 6.7.8-9
If you need any further information regarding the server configuration, etc. please let me know. I may need to contact our hoster.
To begin, I do not know much about the inner workings of ImageMagick. I found out about it through using TYPO3 to process images. Thus, I am not sure if this is considered a bug.
Problem: When large images (>= 30MB) are converted simultaneously by multiple processes, we are often left with broken JPEGs. We can emulate this behavior if page (edited by the backend containing a fresh image that has yet to be converted into various sizes, usually at least 5 different versions for various viewports) is accessed by two browsers at the same time. If the file has not been processed yet, this will happen at runtime, which causes both instances to process the image concurrently. The conversion process then breaks and the file is incomplete. After doing a little reading, I thought ImageMagick employs some sort of mutex to prevent concurrent access to the file, but perhaps I don't quite understand how that works.
I would first like to know if this behavior could be due to an improperly configured installation of ImageMagick. Secondly, if ImageMagick is properly configured, but steps could be taken to prevent concurrent processing of a file without touching the TYPO3 Core, I would like to know what these are.
Current environment:
Server: Apache 2.4.23
PHP: PHP 7.0.20 Nonthread safe
CMS: TYPO3 7.6.21
ImageMagick 6.7.8-9
If you need any further information regarding the server configuration, etc. please let me know. I may need to contact our hoster.