How would you go about separating elements from their background?

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
Post Reply
coloring
Posts: 82
Joined: 2015-08-27T10:17:36-07:00
Authentication code: 1151

How would you go about separating elements from their background?

Post by coloring »

I would like to separate "islands" (icons) from a "sea" (background). The difficulty is that the background can have more than one color depending on the area.

In other words, I need to make "islands of content" stand out from the "background sea".

I suppose the technique would be somewhat similar to how scanned documents are post-processed to whiten the page and darken the fonts.

The idea is to later quickly determine the coordinates of those "islands" by scanning for a single color.

I cannot upload any images right now, but I may do so later on when I get better internet.
snibgo
Posts: 12159
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: How would you go about separating elements from their background?

Post by snibgo »

coloring wrote:I suppose the technique would be somewhat similar to how scanned documents are post-processed to whiten the page and darken the fonts.
Yeah, maybe.

The trick is to define what visually distinguishes foreground and background.

For more specific advice, you know what to do.
snibgo's IM pages: im.snibgo.com
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: How would you go about separating elements from their background?

Post by fmw42 »

Background foreground separation can be very hard unless the background is a relatively constant color.
User avatar
anthony
Posts: 8883
Joined: 2004-05-31T19:27:03-07:00
Authentication code: 8675308
Location: Brisbane, Australia

Re: How would you go about separating elements from their background?

Post by anthony »

Also how 'fuzzy' and 'shaped' the border is between background and foreground. For example rotated square images? Or other odd shapes.
What other 'junk' is on the page... Text?

It is all a matter of defining what is foreground and background in a way the computer can understand.
The trouble with computers, of course,
is that they're very sophisticated idiots. -- Doctor Who, "Robot"
Anthony Thyssen -- Webmaster for ImageMagick Example Pages
https://imagemagick.org/Usage/
coloring
Posts: 82
Joined: 2015-08-27T10:17:36-07:00
Authentication code: 1151

Re: How would you go about separating elements from their background?

Post by coloring »

In this case, the borders are not fuzzy at all. I am trying to isolate widgets such as fields and buttons from a screenshot.
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: How would you go about separating elements from their background?

Post by fmw42 »

Post an example image and also identify your IM version, date, and platform.
coloring
Posts: 82
Joined: 2015-08-27T10:17:36-07:00
Authentication code: 1151

Re: How would you go about separating elements from their background?

Post by coloring »

Code: Select all

C:> magick.exe --version
Version: ImageMagick 7.0.8-16 Q16 x64 2018-12-10 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2018 ImageMagick Studio LLC
License: http://www.imagemagick.org/script/license.php
Visual C++: 180040629
Features: Cipher DPC HDRI
Delegates (built-in): bzlib cairo flif freetype gslib heic jng jp2 jpeg lcms lqr
 lzma openexr pangocairo png ps raw rsvg tiff webp xml zlib
Image

I'd like to either overlay a yellow border around those "islands" like in the image, or crop out the widgets inside the yellow border and save them as separate images.
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: How would you go about separating elements from their background?

Post by fmw42 »

Get rid of the background image. Then threshold or use -fill -opaque on the image to make all the text white and the gray into black. Blur the image a little so the text is all connected and re-threhsold. Then use connected components to extract the bounding boxes around those regions. Then use the bounding boxes to crop your original image or draw yellow boxes given he bounding box coordinates. Sorry I do not code in Windows scripting.

See https://imagemagick.org/script/connected-components.php
coloring
Posts: 82
Joined: 2015-08-27T10:17:36-07:00
Authentication code: 1151

Re: How would you go about separating elements from their background?

Post by coloring »

Thanks! Non-GNU Linux or macOS are fine too. From there I can figure out how to translate it for Windows.
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: How would you go about separating elements from their background?

Post by fmw42 »

Here is some code that seems to work for me on my Mac. I have posted the tmp images to show intermediate steps. The -define connected-components:area-threshold=20 is there to remove some small black spots interior to the white, but may not be necessary.

The main idea is to

1) replace white with red
2) replace the darker gray with red
3) extract the Chroma channel from HCL
4) filltoborder with black to remove the image texture outside your gray area
5) threshold at 0 to binarize
6) use connected components to filter out some small areas
7) horizontal only blur 0x1.5 via morphology and threshold 0 to binarize again
8 ) do a morphology open to remove some long horizontal lines from the original white lines
9) do connected components again to get only the gray(255) areas and save only the bounding boxes for them
10) loop over all the white region bounding boxes and convert the boxes from WxH+X+Y to X1,Y1 X2,Y2 for drawing and save as a string
11) draw all the bounding boxes with one draw command

Code: Select all

bboxArr=(`convert image_1.gif \
-fill red -opaque white \
-fill red -opaque "gray(216)" \
-colorspace HCL -channel g -separate +channel +write tmp0.png \
-fill black -bordercolor black -draw "color 20,10 filltoborder" -alpha off \
-threshold 0 \
-define connected-components:area-threshold=6 \
-define connected-components:mean-color=true \
-connected-components 4 \
-morphology convolve blur:0x1.5 -threshold 0 \
-morphology open octagon:1 -type bilevel +write tmp1.png \
-define connected-components:verbose=true \
-define connected-components:area-threshold=20 \
-define connected-components:mean-color=true \
-connected-components 4 tmp2.png | grep "gray(255)" | awk '{print $2}'| tr "x" "+"`)
num="${#bboxArr[*]}"
echo "$num"
draw_args=""
for ((i=0; i<num; i++)); do
bbox="${bboxArr[$i]}"
ww=`echo "$bbox" | cut -d+ -f1`
hh=`echo "$bbox" | cut -d+ -f2`
x1=`echo "$bbox" | cut -d+ -f3`
y1=`echo "$bbox" | cut -d+ -f4`
y1=$((y1-4))
x2=$((ww+x1))
y2=$((hh+y1+6))
draw_args="$draw_args rectangle $x1,$y1 $x2,$y2"
done
convert image_1.gif -fill none -stroke red -strokewidth 2 -draw "$draw_args" -alpha off image_1_proc.gif

tmp00.png
Image

tmp0.png
Image

tmp1.png
Image

tmp2.png
Image


Output
Image
Post Reply