Page 1 of 1

What is IM's strategy for rounding floats?

Posted: 2018-07-20T13:45:14-07:00
by donahchoo
Hi,

I'm trying to understand how IM will round floats when resizing. I've seen a case where I had a resize operation result in an image with a 337.5 height and that got rounded down to 337. I'm not seeing a case where an where a resize results in 119.9877 and that is getting rounded up to 120. I thought IM was basically truncating the floats, but that's not the case.

Any suggestions?

Thanks!

Re: What is IM's strategy for rounding floats?

Posted: 2018-07-20T13:56:36-07:00
by snibgo
I would expect that most operations that convert from float to integer will round, rather than truncate. For example:

Code: Select all

f:\web\im>%IMG7%magick -size 1199877x1 xc: -resize 0.01% info:

xc: XC 120x1 120x1+0+0 16-bit sRGB 0.203u 0:00.202

Re: What is IM's strategy for rounding floats?

Posted: 2018-07-20T14:25:27-07:00
by donahchoo
Yeah, I was expecting it to just use C's round function. I'll test some more...

Re: What is IM's strategy for rounding floats?

Posted: 2018-07-20T14:33:09-07:00
by fmw42
I am answering a different aspect of resizing. I am not sure how it determines the final image size.

But, when resizing it does resampling to get the fractional input position and blends neighboring pixels to get the resampled (interpolated) pixel color for putting in the output at a given integer pixel. See

http://www.imagemagick.org/Usage/filter/
http://www.imagemagick.org/Usage/distorts/#summary

Re: What is IM's strategy for rounding floats?

Posted: 2018-07-23T19:16:15-07:00
by donahchoo
Thanks, I'll study and learn.