Remove White Background from an Image and Make It Transparent

We're trying to do the following in Mathematica - RMagick remove white background from image and make it transparent.

But with actual photos it ends up looking lousy (like having a halo around the image).

Here's what we've tried so far:

unground0[img_] := With[{mask = ChanVeseBinarize[img, TargetColor->{1.,1.,1.}]},
  Rasterize[SetAlphaChannel[img, ImageApply[1-#&, mask]], Background->None]]]

Here's an example of what that does.

Original image:

Image with the white background replaced with no background (or, for demonstration purposes here, a pink background):

Any ideas for getting rid of that halo? Tweaking things like LevelPenalty, I can only get the halo to go away at the expense of losing some of the image.

EDIT: So I can compare solutions for the bounty, please structure your solution like above, namely a self-contained function named unground-something that takes an image and returns an image with transparent background.

This function implements the reverse blend described by Mark Ransom, for an additional small but visible improvement:

reverseBlend[img_Image, alpha_Image, bgcolor_] :=
 With[
  {c = ImageData[img], 
   a = ImageData[alpha] + 0.0001, (* this is to minimize ComplexInfinitys and considerably improve performance *)
   bc = bgcolor},

  ImageClip@
   Image[Quiet[(c - bc (1 - a))/a, {Power::infy, 
       Infinity::indet}] /. {ComplexInfinity -> 0, Indeterminate -> 0}]
  ]

This is the background removal function. The threshold parameter is used for the initial binarization of the image, the minSizeCorrection is for tweaking the size limit of small junk components to be removed after binarization.

removeWhiteBackground[img_, threshold_: 0.05, minSizeCorrection_: 1] :=
  Module[
  {dim, bigmask, mask, edgemask, alpha},
  dim = ImageDimensions[img];
  bigmask = 
   DeleteSmallComponents[
    ColorNegate@
     MorphologicalBinarize[ColorNegate@ImageResize[img, 4 dim], threshold], 
    Round[minSizeCorrection Times @@ dim/5]];
  mask = ColorNegate@
    ImageResize[ColorConvert[bigmask, "GrayScale"], dim];
  edgemask = 
   ImageResize[
    ImageAdjust@DistanceTransform@Dilation[EdgeDetect[bigmask, 2], 6],
     dim];
  alpha = 
   ImageAdd[
    ImageSubtract[
     ImageMultiply[ColorNegate@ColorConvert[img, "GrayScale"], 
      edgemask], ImageMultiply[mask, edgemask]], mask];
  SetAlphaChannel[reverseBlend[img, alpha, 1], alpha]
  ]

Testing the function:

img = Import["http://i.stack.imgur.com/k7E1F.png"];

background = 
  ImageCrop[
   Import["http://cdn.zmescience.com/wp-content/uploads/2011/06/\
forest2.jpg"], ImageDimensions[img]];

result = removeWhiteBackground[img]

ImageCompose[background, result]
Rasterize[result, Background -> Red]
Rasterize[result, Background -> Black]

Brief explanation of how it works:

1. Choose your favourite binarization method that produces relatively precise sharp edges

2. Apply it to an up-scaled image, then downscale the obtained mask to the original size. This gives us antialiasing. Most of the work is done.

3. For a small improvement, blend the image onto the background using the brightness of its negative as alpha, then blend the obtained image over the original in a thin region around the edges (edgemask) to reduce the visibility of white pixels on the edges. The alpha channel corresponding to these operations is calculated (the somewhat cryptic ImageMultiply/Add expression).

4. Now we have an estimate of the alpha channel so we can do a reverse blend.

Steps 3 & 4 don't improve that much, but the difference is visible.

Perhaps, depending on the edge quality you need:

img = Import@"http://i.stack.imgur.com/k7E1F.png";
mask = ChanVeseBinarize[img, TargetColor -> {1., 1., 1.}, "LengthPenalty" -> 10]
mask1 = Blur[Erosion[ColorNegate[mask], 2], 5]
Rasterize[SetAlphaChannel[img, mask1], Background -> None]

Edit

Stealing a bit from @Szabolcs

img2 = Import@"http://i.stack.imgur.com/k7E1F.png";
(*key point:scale up image to smooth the edges*)
img = ImageResize[img2, 4 ImageDimensions[img2]];
mask = ChanVeseBinarize[img, TargetColor -> {1., 1., 1.}, "LengthPenalty" -> 10];
mask1 = Blur[Erosion[ColorNegate[mask], 8], 10];
f[col_] := Rasterize[SetAlphaChannel[img, mask1], Background -> col, 
                     ImageSize -> ImageDimensions@img2]
GraphicsGrid[{{f@Red, f@Blue, f@Green}}]

Click to enlarge

Edit 2

Just to get an idea of the extent of the halo and background imperfections in the image:

img = Import@"http://i.stack.imgur.com/k7E1F.png";
Join[{img}, MapThread[Binarize, {ColorSeparate[img, "HSB"], {.01, .01, .99}}]]

ColorNegate@ImageAdd[EntropyFilter[img, 1] // ImageAdjust, ColorNegate@img]

I'm going to speak generically, not specifically in reference to Mathematica. I have no idea whether these operations are difficult or trivial.

The first step is to estimate an alpha (transparency) level for the pixels on the edge of the image. Right now you're using a strict threshold, so the alpha is either 0% totally transparent or 100% totally opaque. You should define a range between the total white of the background and colors that are indisputably part of the image, and set an appropriate proportion - if it's closer in color to the background it's low alpha, and if it's closer to the darker cutoff then it's a high alpha. After that you can make adjustments based on the surrounding alpha values - the more a pixel is surrounded by transparency, the more likely it is to be transparent itself.

Once you have alpha values you need to do a reverse blend to get the proper color. When an image is displayed over a background it is blended according to the alpha value using the formula c = bc*(1-a)+fc*a where bc is the background color and fc is the foreground color. In your case the background is white (255,255,255) and the foreground color is the unknown, so we reverse the formula: fc = (c - bc*(1-a))/a. When a=0 the formula calls for a divide by zero, but the color doesn't matter anyway so just use black or white.

Here's a try at implementing Mark Ransom's approach, with some help from belisarius's mask generation:

Locate the boundary of the object:

img1 = SetAlphaChannel[img, 1];
erosionamount=2;
mb = ColorNegate@ChanVeseBinarize[img, TargetColor -> {1., 1., 1}, 
      "LengthPenalty" -> 10];
edge = ImageSubtract[Dilation[mb, 2], Erosion[mb, erosionamount]];

ImageApply[{1, 0, 0} &, img, Masking ->edge]

Set the alpha values:

edgealpha = ImageMultiply[ImageFilter[(1 - Mean[Flatten[#]]^5) &, 
   ColorConvert[img, "GrayScale"], 2, Masking -> edge], edge];
imagealpha = ImageAdd[edgealpha, Erosion[mb, erosionamount]];
img2 = SetAlphaChannel[img, imagealpha];

Reverse color blend:

img3 = ImageApply[Module[{c, \[Alpha], bc, fc},
   bc = {1, 1, 1};
   c = {#[[1]], #[[2]], #[[3]]};
   \[Alpha] = #[[4]];
   If[\[Alpha] > 0, Flatten[{(c - bc (1 - \[Alpha]))/\[Alpha], \[Alpha]}], {0., 0., 
   0., 0}]] &, img2];

Show[img3, Background -> Pink]

Notice how some of the edges have white fuzz? Compare that with the red outline in the first image. We need a better edge detector. Increasing the erosion amount helps with the fuzz, but then other sides become too transparent, so there is a tradeoff on the width of the edge mask. It's pretty good, though, considering there is no blur operation, per se.

It would be instructive to run the algorithm on a variety of images to test its robustness, to see how automatic it is.