deskew

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
Locked
jaffamuffin
Posts: 60
Joined: 2009-01-30T03:46:08-07:00

deskew

Post by jaffamuffin »

Hi all
I'm trying to deskew scanned images but weird things are happening. https://gofile.io/?c=ZgXVTY

here is my command

Code: Select all

FOR /F %%B IN ('convert %1 -gravity center -crop 1000x1000+0+0 -deskew 40%% -format %%[deskew:angle] info:') DO (
    SET angle=%%B
    echo angle %%B
)
convert %1 -rotate "%angle%" +repage %2
The main thing is the first command convert %1 -gravity center -crop 1000x1000+0+0 -deskew 40%% -format %%[deskew:angle] info: to detect the angle required.

Now I know my test image is skewed by 1.5 degrees. it's a yellow A4 page on black background. 300 dpi :
300_01.tif TIFF 2742x3756 2742x3756+0+0 8-bit sRGB 29.4659MiB 0.000u 0:00.000

1. If I run this i get a value of 0.111 degrees
2. If I crop the image to 1000x1000 and then run it on the newly cropped image I get 1.45 degree, correct.
3. If I remove the crop part of the command and run it on the original image i get 0.27 degrees.
4. if i remove the crop part of the command and run it on the crop i get also 1.45

So I can't inline crop ? , but when I do I get a different result to supplying an already cropped image. I don't want expensive temp files and read.write ops ? I thought I could do it this way with imagemagick ?

Also have tried a +repage after the crop but makes no difference.

snibgo
Posts: 13034
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: deskew

Post by snibgo »

I don't understand your points 3 and 4. Showing the actual commands would help.

Pre-processing before "-deskew" usually helps, eg:

Code: Select all

magick 300_01.tif ( +clone -fuzz 10% -fill White +opaque Black -deskew 40% -set option:DESK %[deskew:angle] +delete ) -rotate %[DESK] out.png
This needs IM v7. It processes a copy to find the angle, then rotates the original by that angle.
snibgo's IM pages: im.snibgo.com

jaffamuffin
Posts: 60
Joined: 2009-01-30T03:46:08-07:00

Re: deskew

Post by jaffamuffin »

i'm using version
Version: ImageMagick 7.0.7-22 Q8 x64 2018-01-22 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2015 ImageMagick Studio LLC
License: http://www.imagemagick.org/script/license.php
Visual C++: 180040629
Features: Cipher DPC Modules OpenMP
Delegates (built-in): bzlib cairo flif freetype gslib jng jp2 jpeg lcms lqr open
exr pangocairo png ps raw rsvg tiff webp xml zlib


as for each command :
1. convert ORIG.TIF -gravity center -crop 1000x1000+0+0 -deskew 40%% -format %%[deskew:angle] info: gives 0.11 degrees
2. if i use crop it i.e. convert %1 -gravity center -crop 1000x1000+0+0 1000pxcrop.tif. Then Feed 1000pxcrop.tif into my command above, I get 1.45 degrees, OK
3. if i do this : convert ORIG.TIF -deskew 40%% -format %%[deskew:angle] info: I get 0.27 degrees
4. convert "1000pxcrop.tif" -deskew 40%% -format %%[deskew:angle] info: same 1.45 degrees

jaffamuffin
Posts: 60
Joined: 2009-01-30T03:46:08-07:00

Re: deskew

Post by jaffamuffin »

another way to look at it :
file as in first post is 300_01.tif


This code supposedly takes a 1000x1000 section from the top middle and bottom of the supplied image and prints the detected skew angle. (test.bat)

Code: Select all

FOR /F %%B IN ('convert %1 -contrast-stretch 10%%x10%% -gravity center -crop 1000x1000+0+0 +repage  +deskew  -format %%[deskew:angle] info:') DO (
    SET angle=%%B
    echo angle %%B
)
FOR /F %%B IN ('convert %1 -contrast-stretch 10%%x10%% -gravity north -crop 1000x1000+0+0 +repage  +deskew  -format %%[deskew:angle] info:') DO (
    SET angle=%%B
    echo angle %%B
)
FOR /F %%B IN ('convert %1 -contrast-stretch 10%%x10%%  -gravity south -crop 1000x1000+0+0 +repage   +deskew -format %%[deskew:angle] info:') DO (
    SET angle=%%B
    echo angle %%B
)
If I prepare the same 3 sections top middle bottom independently :

D:\samples>convert 300_01.tif -gravity south -crop 1000x1000+0+0 +repage south.tif

D:\samples>convert 300_01.tif -gravity center -crop 1000x1000+0+0 +repage center.tif

D:\samples>convert 300_01.tif -gravity north -crop 1000x1000+0+0 +repage north.tif

north south and center should print the same 3 values as there is only one possible 1000x1000 area to select:

D:\samples>test.bat center.tif
angle 0.1119056770662069
angle 0.1119056770662069
angle 0.1119056770662069

D:\samples>test.bat north.tif
angle -0.1119056770662069
angle -0.1119056770662069
angle -0.1119056770662069

D:\samples>test.bat south.tif
angle 1.2307746699577373
angle 1.2307746699577373
angle 1.2307746699577373


BUT I Would expect this file (the original image) to print same as the values above i,e, expected output should be
angle -0.1119056770662069
angle 1.2307746699577373
angle 0.1119056770662069

BUT I get this instead. Actually, 1.45 is the correct and best value.
D:\samples>test.bat 300_01.tif
angle 1.4544631491195343 (center)
angle 1.3426240265377571 (north)
angle 1.2307746699577373 (south)

What am i doing wrong ?

snibgo
Posts: 13034
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: deskew

Post by snibgo »

jaffamuffin wrote:What am i doing wrong ?
Nothing wrong, as such. You are comparing the results from (a) contrast-stretch and then crop with (b) crop and then contrast-stretch. The images in these two cases are different, so the deskew angles are also different.
snibgo's IM pages: im.snibgo.com

User avatar
fmw42
Posts: 26383
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: deskew

Post by fmw42 »

contrast-stretch looks at the histogram. When you crop, the histogram will differ from the original.

jaffamuffin
Posts: 60
Joined: 2009-01-30T03:46:08-07:00

Re: deskew

Post by jaffamuffin »

got it. good point thanks.

IM.nut
Posts: 13
Joined: 2014-02-24T13:03:49-07:00
Authentication code: 6789

Re: deskew

Post by IM.nut »

It's a bug, you say. So it is. At least, that's what I'm seeing and OP confirms. (Or maybe he sees and I...)

Deskew worked perfectly on all image sizes with IM6. In IM7 (running 7.0.9-2 in Linux), apparently the deskew angle diminishes with increasing image sizes beyond a threshold. In my testing, that's around 144x144. Tried a couple of input images and got same results so it's repeatable. Also found that as long as the image's clone was first resized to one of the 'good' sizes before capturing the rotation angle, the command operated correctly.

It was in the process of trying to help -deskew work right that I tried resizing and found erratic results. (The deskew angle should remain constant.) So used the following to generate data and see what was going on:

Code: Select all

for s in $(seq 96 4 200); do echo -n "$s " >> results.txt; magick in.png -alpha extract +opaque white -negate -resize $sx$s -deskew 80% -format %[deskew:angle] info: >> results.txt; echo >> results.txt; done
(Thanks to those in this thread, here and below.) In the above command line, discreet pixel values separated by a space can be substituted for the '$(seq 96 4 200)'. Ex. 358 393 407 622

Data Summary (Correct angle is approx 0.895)
Image Size (px) - Angle
144x144 or less - 0.89517371021107439155 << correct
larger, up to approx 300x300 - 0.44761417086055305115
larger, up to approx 400x400 - 0.22381050036853808449
greater than approx 500x500 - 0.11190567706620689614

So the workaround I'm using is:

Code: Select all

magick in.png \( +clone -alpha extract +opaque white -negate -resize 144x144 -deskew 80% -set option:DESK %[deskew:angle] +delete \) -background none -rotate %[DESK] out.png
Also happy to know it wasn't the nut behind the wheel.

snibgo
Posts: 13034
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: deskew

Post by snibgo »

What version of IM?
IM.nut wrote:In my testing, that's around 144x144. Tried a couple of input images and got same results so it's repeatable.
If you show your images, we can test your results. Are your images "clean", eg skewed white rectangles with black surround? If not, then the algorithm may be influenced by noise that is blurred away at small resolutions.

For example, making a clean example for testing, IM v7.0.8-64 on Windows 8.1:

Code: Select all

%IMG7%magick ^
  -size 1000x800 xc:White ^
  -virtual-pixel Black ^
  +distort SRT 1,1.5 ^
  +repage ^
  -bordercolor Black -border 50x100 ^
  +write info: ^
  x.png
The image is 1122x1028 pixels. Test at a range of resizes:

Code: Select all

for %%A in (72 144 300 600 1000) do (
  %IMG7%magick ^
    x.png ^
    -resize %%A ^
    -deskew 40%% ^
    -format %%[deskew:angle] info:
)
The results, in degrees, are:

Code: Select all

-0.89517371021107439
-1.3426240265377571
-1.4544631491195343
-1.510378606791748
-1.4544631491195343
Except for the first, they are all close to 1.5 degrees.
snibgo's IM pages: im.snibgo.com

IM.nut
Posts: 13
Joined: 2014-02-24T13:03:49-07:00
Authentication code: 6789

Re: deskew

Post by IM.nut »

My worst nightmare; answering one of the pros. Ha, but at least my entire post wasn't invalidated by something simple. Answers inline...
snibgo wrote:
2019-11-13T18:51:44-07:00
What version of IM? "running 7.0.9-2 in Linux" (first line of second paragraph)

Are your images "clean"...? Images are bona fide clean: At full resolution, GIMP auto-deskews them perfectly, and fast too.

If you show your images, we can test your results. Can't share those, but let me put together one or more that are similar.
...
The results, in degrees, are:

Code: Select all

-0.89517371021107439
-1.3426240265377571
-1.4544631491195343
-1.510378606791748
-1.4544631491195343
Except for the first, they are all close to 1.5 degrees.
Creating and using the same image from your post, there is some difference. (My version is fresh-compiled, latest available as of a couple days ago.)

Code: Select all

72 -0.89517371021107439155
144 -1.1189159305671887434
300 -1.4544631491195343376
600 -1.4544631491195343376
1000 -1.4544631491195343376
And this is the data from a quick run through the same range as the earlier post. Highs at 100 and 184, lowest at 124. Overall, not only less linear, but non-linear. Really would like to understand why. If reason is not clear, let me cobble up that test drawing or two to experiment with.

Code: Select all

96 -1.3426240265377571426
100 -1.7899106082460694012
104 -1.3426240265377571426
108 -1.7899106082460694012
112 -1.7899106082460694012
116 -1.7899106082460694012
120 -1.3426240265377571426
124 -0.89517371021107439155
128 -1.3426240265377571426
132 -1.3426240265377571426
136 -1.7899106082460694012
140 -1.3426240265377571426
144 -1.1189159305671887434
148 -1.3426240265377571426
152 -1.3426240265377571426
156 -1.3426240265377571426
160 -1.1189159305671887434
164 -1.3426240265377571426
168 -1.1189159305671887434
172 -1.3426240265377571426
176 -1.3426240265377571426
180 -1.1189159305671887434
184 -1.7899106082460694012
188 -1.1189159305671887434
192 -1.1189159305671887434
196 -1.1189159305671887434
200 -1.3426240265377571426

snibgo
Posts: 13034
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: deskew

Post by snibgo »

Sorry I missed your IM version number.

True, I don't get perfect results. Note that:

Code: Select all

 tand(1.5)-tand(1.4544) = 0.0007964
So the error is less than 1 part in 1000. When we resize to be within a box 1000x1000 pixels, the error is less than one pixel. We can't hope for much better than this.

If we don't resize at all, the deskew angle is -1.48242 degrees, and the error is 0.3 pixels.

If we shrink my test image to a box 100x100, an error of plus or minus 1 pixel will be more significant: we expect the calculated deskew angle will be 1.5 plus or minus 0.5729 degrees (because artan(1/100) is 0.5729).

I don't know why your results are different to mine. They are not very different for resizing to 300 or above.

My point was to test your assertion that resizing down to 144x144 improves accuracy. I don't think that's true, provided the image is clean, eg pure white rectangle skewed, with black surround, like my image. You say your "Images are bona fide clean", but you can't share them. So I assume they are not clean, and the white rectangles have text or something confidential.

In my limited experience, "-deskew" doesn't work well on unprocessed scans of text. We need to pre-process the image, to remove the text or blur it or whatever.
snibgo's IM pages: im.snibgo.com

IM.nut
Posts: 13
Joined: 2014-02-24T13:03:49-07:00
Authentication code: 6789

Re: deskew

Post by IM.nut »

To tie up this loose end, everything's ok without doing the 'resize' -- which is faster and easier so, a big NM.

The test/setup images were from a batch of approx 200, which caused them to receive excessive scrutiny, I think. On running the full batch, IM7 deskewed everything amazingly well, and all using the full-scale images.

At one point I noticed that a +gravity before an operation caused the top object edge to be aligned true, while not explicitly setting gravity caused the left edge to be trued. (Most of the images portrayed objects that had straight sides across top and left, but with the two slightly out-of-square both mutually and absolutely.)

Bottom line: I would not pursue this as a bug. (And of course, much gratitude for this awesome piece of software. ...And to the pros who constantly prowl this forum to provide help; it is top-tier support and greatly appreciated.)

Locked