使用 opencv Python 去除图像的背景

Remove background of the image using opencv Python

提问by muazfaiz

I have two images, one with only background and the other with background + detectable object (in my case its a car). Below are the images

我有两张图片，一张只有背景，另一张有背景+可检测的物体（在我的情况下是一辆车）。下面是图片

I am trying to remove the background such that I only have car in the resulting image. Following is the code that with which I am trying to get the desired results

我正在尝试删除背景，以便在结果图像中只有汽车。以下是我试图获得所需结果的代码

import numpy as np
import cv2


original_image = cv2.imread('IMG1.jpg', cv2.IMREAD_COLOR)
gray_original = cv2.cvtColor(original_image, cv2.COLOR_BGR2GRAY)
background_image = cv2.imread('IMG2.jpg', cv2.IMREAD_COLOR)
gray_background = cv2.cvtColor(background_image, cv2.COLOR_BGR2GRAY)

foreground = np.absolute(gray_original - gray_background)
foreground[foreground > 0] = 255

cv2.imshow('Original Image', foreground)
cv2.waitKey(0)

The resulting image by subtracting the two images is

两幅图像相减得到的图像为

Here is the problem. The expected resulting image should be a car only. Also, If you take a deep look in the two images, you'll see that they are not exactly same that is, the camera moved a little so background had been disturbed a little. My question is that with these two images how can I subtract the background. I do not want to use grabCut or backgroundSubtractorMOG algorithm right now because I do not know right now whats going on inside those algorithms.

这是问题所在。预期的结果图像应该只是一辆汽车。此外，如果您仔细观察这两张图片，您会发现它们并不完全相同，即相机稍微移动了一点，因此背景受到了一点干扰。我的问题是，使用这两个图像如何减去背景。我现在不想使用grabCut或backgroundSubtractorMOG算法，因为我现在不知道这些算法内部发生了什么。

What I am trying to do is to get the following resulting image

我想要做的是获得以下结果图像

Also if possible, please guide me with a general way of doing this not only in this specific case that is, I have a background in one image and background+object in the second image. What could be the best possible way of doing this. Sorry for such a long question.

另外，如果可能的话，请指导我使用一种通用方法来执行此操作，而不仅仅是在这种特定情况下，即我在一个图像中有背景，在第二个图像中有背景 + 对象。这样做的最佳方法是什么。抱歉问了这么长的问题。

采纳答案by TasosGlrs

I solved your problem using the OpenCV's watershedalgorithm. You can find the theory and examples of watershed here.

我使用 OpenCV 的分水岭算法解决了您的问题。您可以在此处找到分水岭的理论和示例。

First I selected several points (markers) to dictate where is the object I want to keep, and where is the background. This step is manual, and can vary a lot from image to image. Also, it requires some repetition until you get the desired result. I suggest using a tool to get the pixel coordinates. Then I created an empty integer array of zeros, with the size of the car image. And then I assigned some values (1:background, [255,192,128,64]:car_parts) to pixels at marker positions.

首先，我选择了几个点（标记）来指示我想要保留的对象在哪里，以及背景在哪里。此步骤是手动的，并且可能因图像而异。此外，它需要一些重复，直到您获得所需的结果。我建议使用工具来获取像素坐标。然后我创建了一个空的零整数数组，具有汽车图像的大小。然后我为标记位置的像素分配了一些值 (1:background, [255,192,128,64]:car_parts)。

NOTE:When I downloaded your image I had to crop it to get the one with the car. After cropping, the image has size of 400x601. This may not be what the size of the image you have, so the markers will be off.

注意：当我下载你的图片时，我必须裁剪它才能得到汽车上的图片。裁剪后，图像大小为 400x601。这可能不是您拥有的图像大小，因此标记将关闭。

Afterwards I used the watershed algorithm. The 1st input is your image and 2nd input is the marker image (zero everywhere except at marker positions). The result is shown in the image below.

后来我使用了分水岭算法。第一个输入是您的图像，第二个输入是标记图像（除标记位置外，所有地方都为零）。结果如下图所示。

I set all pixels with value greater than 1 to 255 (the car), and the rest (background) to zero. Then I dilated the obtained image with a 3x3 kernel to avoid losing information on the outline of the car. Finally, I used the dilated image as a mask for the original image, using the cv2.bitwise_and() function, and the result lies in the following image:

我将值大于 1 的所有像素设置为 255（汽车），其余（背景）设置为零。然后我用 3x3 内核对获得的图像进行了扩张，以避免丢失关于汽车轮廓的信息。最后，我用膨胀的图像作为原始图像的掩码，使用 cv2.bitwise_and() 函数，结果在下图：

Here is my code:

这是我的代码：

import cv2
import numpy as np
import matplotlib.pyplot as plt

# Load the image
img = cv2.imread("/path/to/image.png", 3)

# Create a blank image of zeros (same dimension as img)
# It should be grayscale (1 color channel)
marker = np.zeros_like(img[:,:,0]).astype(np.int32)

# This step is manual. The goal is to find the points
# which create the result we want. I suggest using a
# tool to get the pixel coordinates.

# Dictate the background and set the markers to 1
marker[204][95] = 1
marker[240][137] = 1
marker[245][444] = 1
marker[260][427] = 1
marker[257][378] = 1
marker[217][466] = 1

# Dictate the area of interest
# I used different values for each part of the car (for visibility)
marker[235][370] = 255    # car body
marker[135][294] = 64     # rooftop
marker[190][454] = 64     # rear light
marker[167][458] = 64     # rear wing
marker[205][103] = 128    # front bumper

# rear bumper
marker[225][456] = 128
marker[224][461] = 128
marker[216][461] = 128

# front wheel
marker[225][189] = 192
marker[240][147] = 192

# rear wheel
marker[258][409] = 192
marker[257][391] = 192
marker[254][421] = 192

# Now we have set the markers, we use the watershed
# algorithm to generate a marked image
marked = cv2.watershed(img, marker)

# Plot this one. If it does what we want, proceed;
# otherwise edit your markers and repeat
plt.imshow(marked, cmap='gray')
plt.show()

# Make the background black, and what we want to keep white
marked[marked == 1] = 0
marked[marked > 1] = 255

# Use a kernel to dilate the image, to not lose any detail on the outline
# I used a kernel of 3x3 pixels
kernel = np.ones((3,3),np.uint8)
dilation = cv2.dilate(marked.astype(np.float32), kernel, iterations = 1)

# Plot again to check whether the dilation is according to our needs
# If not, repeat by using a smaller/bigger kernel, or more/less iterations
plt.imshow(dilation, cmap='gray')
plt.show()

# Now apply the mask we created on the initial image
final_img = cv2.bitwise_and(img, img, mask=dilation.astype(np.uint8))

# cv2.imread reads the image as BGR, but matplotlib uses RGB
# BGR to RGB so we can plot the image with accurate colors
b, g, r = cv2.split(final_img)
final_img = cv2.merge([r, g, b])

# Plot the final result
plt.imshow(final_img)
plt.show()

If you have a lot of images you will probably need to create a tool to annotate the markers graphically, or even an algorithm to find markers automatically.

如果您有很多图像，您可能需要创建一个工具来以图形方式注释标记，甚至需要创建一个算法来自动查找标记。

回答by Dan Ma?ek

The problem is that you're subtracting arrays of unsigned8 bit integers. This operation can overflow.

问题是您正在减去无符号8 位整数数组。此操作可能会溢出。

To demonstrate

展示

>>> import numpy as np
>>> a = np.array([[10,10]],dtype=np.uint8)
>>> b = np.array([[11,11]],dtype=np.uint8)
>>> a - b
array([[255, 255]], dtype=uint8)

Since you're using OpenCV, the simplest way to achieve your goal is to use cv2.absdiff().

由于您使用的是 OpenCV，因此实现目标的最简单方法是使用cv2.absdiff().

>>> cv2.absdiff(a,b)
array([[1, 1]], dtype=uint8)

回答by wordsforthewise

I recommend using OpenCV's grabcut algorithm. You first draw a few lines on the foreground and background, and keep doing this until your foreground is sufficiently separated from the background. It is covered here: https://docs.opencv.org/trunk/d8/d83/tutorial_py_grabcut.htmlas well as in this video: https://www.youtube.com/watch?v=kAwxLTDDAwU

我推荐使用 OpenCV 的抓取算法。您首先在前景和背景上画几条线，然后继续这样做，直到前景与背景充分分离。此处涵盖：https: //docs.opencv.org/trunk/d8/d83/tutorial_py_grabcut.html以及此视频：https: //www.youtube.com/watch?v=kAwxLTDDAwU