C++ OpenCV 透视图

OpenCV warpperspective

提问by Hien

For some reason whenever I use OpenCV's warpPerspective() function, the final warped image does not contain everything in the original image. The left part of the image seems to get cut off. I think the reason why this is happening is because the warped image is created at the leftmost position of the canvas for the warpPerspective(). Is there some way to fix this? Thanks

出于某种原因，每当我使用 OpenCV 的 warpPerspective() 函数时，最终的变形图像都不包含原始图像中的所有内容。图像的左侧部分似乎被切断了。我认为发生这种情况的原因是扭曲的图像是在画布的最左侧位置为 warpPerspective() 创建的。有没有办法解决这个问题？谢谢

回答by Matt Freeman

The problem occurs because the homography maps part of the image to negative x,y values which are outside the image area so cannot be plotted. what we wish to do is to offset the warped output by some number of pixels to 'shunt' the entire warped image into positive coordinates(and hence inside the image area).

出现问题是因为单应性将图像的一部分映射到图像区域之外的负 x,y 值，因此无法绘制。我们希望做的是将扭曲输出偏移一定数量的像素，以将整个扭曲图像“分流”到正坐标（因此在图像区域内）。

Homographies can be combined using matrix multiplication (which is why they are so powerful). If A and B are homographies, then AB represents the homography which applies B first, and then A.

可以使用矩阵乘法组合单应性（这就是它们如此强大的原因）。如果 A 和 B 是单应性，则 AB 表示先应用 B 再应用 A 的单应性。

Because of this all we need to do to offset the output is create the homography matrix for a translation by some offset, and then pre-multiply that by our original homography matrix

因此，我们需要做的就是通过一些偏移量为平移创建单应矩阵，然后将其与原始单应矩阵相乘

A 2D homography matrix looks like this :

二维单应矩阵如下所示：

[R11,R12,T1]
[R21,R22,T2]
[ P , P , 1]

where R represents a rotation matrix, T represents a translation, and P represents a perspective warp. And so a purely translational homography looks like this:

其中 R 表示旋转矩阵，T 表示平移，P 表示透视扭曲。所以一个纯粹的平移单应性看起来像这样：

[ 1 , 0 , x_offset]
[ 0 , 1 , y_offset]
[ 0 , 0 ,    1    ]

So just premultiply your homography by a matrix similar to the above, and your output image will be offset.

因此，只需将您的单应性乘以类似于上述的矩阵，您的输出图像就会被抵消。

(Make sure you use matrix multiplication, not element wise multiplication!)

（确保您使用矩阵乘法，而不是元素乘法！）

回答by Sam

The secret comes in two parts: the transform matrix (homography), and the resulting image size.

秘密来自两部分：变换矩阵（单应性）和生成的图像大小。

• calculate a correct transform by using the getPerspectiveTransform(). Take 4 points from the original image, calculate their correct position in the destination, put them in two vectors in the same order, and use them to compute the perspective transform matrix.

• Make sure the destination image size (third parameter for the warpPerspective()) is exactly what you want. Define it as Size(myWidth, myHeight).

• 使用 getPerspectiveTransform() 计算正确的变换。从原始图像中取4个点，计算它们在目的地的正确位置，将它们按相同的顺序放在两个向量中，并用它们来计算透视变换矩阵。

• 确保目标图像大小（warpPerspective() 的第三个参数）正是您想要的。将其定义为 Size(myWidth, myHeight)。

回答by Vinoj John Hosan

I have done one method... It is working.

我已经做了一种方法......它正在起作用。

  perspectiveTransform(obj_corners,scene_corners,H);
int maxCols(0),maxRows(0);

 for(int i=0;i<scene_corners.size();i++)
{
   if(maxRows < scene_corners.at(i).y)
        maxRows = scene_corners.at(i).y;
   if(maxCols < scene_corners.at(i).x)
        maxCols = scene_corners.at(i).x;
}

I just find the maximum of the x points and y points respectively and put it on

我只是分别找到 x 点和 y 点的最大值并将其放在

warpPerspective( tmp, transformedImage, homography, Size( maxCols, maxRows ) );

回答by Alessandro Jacopson

Try the below homography_warp.

试试下面的homography_warp。

void homography_warp(const cv::Mat& src, const cv::Mat& H, cv::Mat& dst);

srcis the source image.

src是源图像。

His your homography.

H是你的单应性。

dstis the warped image.

dst是扭曲的图像。

homography_warpadjust your homography as described by https://stackoverflow.com/users/1060066/matt-freemanin his answer https://stackoverflow.com/a/8229116/15485

homography_warp按照https://stackoverflow.com/users/1060066/matt-freeman在他的回答https://stackoverflow.com/a/8229116/15485 中的描述调整你的单应性

// Convert a vector of non-homogeneous 2D points to a vector of homogenehous 2D points.
void to_homogeneous(const std::vector< cv::Point2f >& non_homogeneous, std::vector< cv::Point3f >& homogeneous)
{
    homogeneous.resize(non_homogeneous.size());
    for (size_t i = 0; i < non_homogeneous.size(); i++) {
        homogeneous[i].x = non_homogeneous[i].x;
        homogeneous[i].y = non_homogeneous[i].y;
        homogeneous[i].z = 1.0;
    }
}

// Convert a vector of homogeneous 2D points to a vector of non-homogenehous 2D points.
void from_homogeneous(const std::vector< cv::Point3f >& homogeneous, std::vector< cv::Point2f >& non_homogeneous)
{
    non_homogeneous.resize(homogeneous.size());
    for (size_t i = 0; i < non_homogeneous.size(); i++) {
        non_homogeneous[i].x = homogeneous[i].x / homogeneous[i].z;
        non_homogeneous[i].y = homogeneous[i].y / homogeneous[i].z;
    }
}

// Transform a vector of 2D non-homogeneous points via an homography.
std::vector<cv::Point2f> transform_via_homography(const std::vector<cv::Point2f>& points, const cv::Matx33f& homography)
{
    std::vector<cv::Point3f> ph;
    to_homogeneous(points, ph);
    for (size_t i = 0; i < ph.size(); i++) {
        ph[i] = homography*ph[i];
    }
    std::vector<cv::Point2f> r;
    from_homogeneous(ph, r);
    return r;
}

// Find the bounding box of a vector of 2D non-homogeneous points.
cv::Rect_<float> bounding_box(const std::vector<cv::Point2f>& p)
{
    cv::Rect_<float> r;
    float x_min = std::min_element(p.begin(), p.end(), [](const cv::Point2f& lhs, const cv::Point2f& rhs) {return lhs.x < rhs.x; })->x;
    float x_max = std::max_element(p.begin(), p.end(), [](const cv::Point2f& lhs, const cv::Point2f& rhs) {return lhs.x < rhs.x; })->x;
    float y_min = std::min_element(p.begin(), p.end(), [](const cv::Point2f& lhs, const cv::Point2f& rhs) {return lhs.y < rhs.y; })->y;
    float y_max = std::max_element(p.begin(), p.end(), [](const cv::Point2f& lhs, const cv::Point2f& rhs) {return lhs.y < rhs.y; })->y;
    return cv::Rect_<float>(x_min, y_min, x_max - x_min, y_max - y_min);
}

// Warp the image src into the image dst through the homography H.
// The resulting dst image contains the entire warped image, this
// behaviour is the same of Octave's imperspectivewarp (in the 'image'
// package) behaviour when the argument bbox is equal to 'loose'.
// See http://octave.sourceforge.net/image/function/imperspectivewarp.html
void homography_warp(const cv::Mat& src, const cv::Mat& H, cv::Mat& dst)
{
    std::vector< cv::Point2f > corners;
    corners.push_back(cv::Point2f(0, 0));
    corners.push_back(cv::Point2f(src.cols, 0));
    corners.push_back(cv::Point2f(0, src.rows));
    corners.push_back(cv::Point2f(src.cols, src.rows));

    std::vector< cv::Point2f > projected = transform_via_homography(corners, H);
    cv::Rect_<float> bb = bounding_box(projected);

    cv::Mat_<double> translation = (cv::Mat_<double>(3, 3) << 1, 0, -bb.tl().x, 0, 1, -bb.tl().y, 0, 0, 1);

    cv::warpPerspective(src, dst, translation*H, bb.size());
}

回答by john ktejik

Matt's answer is a good start, and he is correct in saying you need to multiply your homography by

马特的回答是一个好的开始，他说你需要将单应性乘以是正确的

[ 1 , 0 , x_offset]
[ 0 , 1 , y_offset]
[ 0 , 0 ,    1    ]

But he does not specify what x_offset and y_offset are. Other answers have said just take the perspective transform, but that is not correct. You want to take the INVERSE perspective transform.

但他没有具体说明 x_offset 和 y_offset 是什么。其他答案说只是采取透视变换，但这是不正确的。您想采用 INVERSE 透视变换。

Just because a point 0,0 transforms into, say, -10,-10, does not mean that shifting the image by 10,10 will result in a non-cropped image. This is because point 10,10 does not necessarily map into 0,0.
What you want to do is find out what point wouldmap into 0,0, and shift the image by that much. To do that you take the inverse (cv2.invert) of the homography and apply perspectiveTransform.

仅仅因为点 0,0 转换为 -10,-10，并不意味着将图像移动 10,10 将导致未裁剪的图像。这是因为点 10,10 不一定映射到 0,0。
你想要做的是找出哪个点会映射到 0,0，并将图像移动那么多。为此，您需要取单应性的逆 ( cv2.invert) 并应用透视变换。

does not imply:

并不意味着：

You need to apply a reverse transform to find the correct points.

您需要应用反向变换来找到正确的点。

This will get the correct x_offset and y_offset to align your top left point. From there to find the correct bounding box and fit the entire image perfectly, you need to figure out the skew (how much the image slants left or up after your normal, non-inverse, transformation) and add that amount to your x_offset and y_offset as well.

这将获得正确的 x_offset 和 y_offset 以对齐您的左上角。从那里找到正确的边界框并完美地适合整个图像，您需要弄清楚倾斜（在正常的非逆变换后图像向左或向上倾斜多少）并将该量添加到您的 x_offset 和 y_offset以及。

EDIT: This is all theory. Images are a few pixels off in my tests, I'm not sure why.

编辑：这都是理论。在我的测试中图像有几个像素，我不知道为什么。

回答by DanielHsH

warpPerspective() works fine. No need to rewrite it. You probably use it incorrectly.

warpPerspective() 工作正常。没有必要重写它。你可能用错了。

Remember the following tips:

请记住以下提示：

1. (0,0) pixels is not in the center but rather left-upper corner. So if you magnify the image x2 you will lose the lower and right parts, not the border (like in matlab).
2. If you warp image twice it is better to multiply transformations and activate the function once.
3. I think it works only on char/int matrices and not on float/double.
4. When you have a transformation, first zoom/skew/rotation/perspective are applied and finally the translation. So if part of the image is missing just change the transation (two upper rows of last column) in the matrix.

1. (0,0) 像素不在中心而是在左上角。因此，如果您放大图像 x2，您将丢失下部和右侧部分，而不是边框​​（如在 matlab 中）。
2. 如果您将图像扭曲两次，最好乘以变换并激活一次该功能。
3. 我认为它只适用于 char/int 矩阵，而不适用于 float/double。
4. 当您进行转换时，首先应用缩放/倾斜/旋转/透视，最后应用平移。因此，如果缺少部分图像，只需更改矩阵中的转换（最后一列的上两行）。

回答by user3094631

this is my solution

这是我的解决方案

since third parameter in "warpPerspective()" is a transformation matrix,

由于“warpPerspective()”中的第三个参数是一个变换矩阵，

we can make a transformation matrix , which moves the image backward first ,then rotates the image,finally moves the image forward .

我们可以制作一个变换矩阵，先将图像向后移动，然后旋转图像，最后将图像向前移动。

In my case,I have a image with height of 160 px and width of 160 px. I want to rotate the image around [80,80] instead of around [0,0]

就我而言，我有一个高度为 160 像素，宽度为 160 像素的图像。我想围绕 [80,80] 而不是围绕 [0,0] 旋转图像

first,moves the image backward (that means T1)

首先，将图像向后移动（即 T1）

then rotates the image (that means R)

然后旋转图像（即 R）

finally moves the image forward (that means T2)

最后将图像向前移动（即 T2）

void rotateImage(Mat &src_img,int degree)
{
float radian=(degree/180.0)*M_PI;
Mat R(3,3,CV_32FC1,Scalar(0));
R.at<float>(0,0)=cos(radian);R.at<float>(0,1)=-sin(radian);
R.at<float>(1,0)=sin(radian);R.at<float>(1,1)=cos(radian);
R.at<float>(2,2)=1;
Mat T1(3,3,CV_32FC1,Scalar(0));
T1.at<float>(0,2)=-80;
T1.at<float>(1,2)=-80;
T1.at<float>(0,0)=1;
T1.at<float>(1,1)=1;
T1.at<float>(2,2)=1;
Mat T2(3,3,CV_32FC1,Scalar(0));
T2.at<float>(0,2)=80;
T2.at<float>(1,2)=80;
T2.at<float>(0,0)=1;
T2.at<float>(1,1)=1;
T2.at<float>(2,2)=1;

std::cerr<<T1<<std::endl;
std::cerr<<R<<std::endl;
std::cerr<<T2<<std::endl;
std::cerr<<T2*R*T1<<"\n"<<std::endl;

cv::warpPerspective(src_img, src_img, T2*R*T1, src_img.size(), cv::INTER_LINEAR);
}

回答by Sanster

Here is a opencv-python solution for your problem, I put it on github: https://github.com/Sanster/notes/blob/master/opencv/warpPerspective.md

这是针对您的问题的 opencv-python 解决方案，我将其放在 github 上：https: //github.com/Sanster/notes/blob/master/opencv/warpPerspective.md

The key point is as user3094631said, get two translation matrix(T1, T2) and apply to the Rotate matrix(M) T2*M*T1

关键是正如user3094631所说，得到两个平移矩阵（T1，T2）并应用于旋转矩阵（M）T2*M*T1

In the code I give, T1 is from the center point of origin image, and T2 is from the left-top point of the transformed boundingBox. The transformed boundingBox comes from origin corner points:

在我给出的代码中，T1 来自原始图像的中心点，T2 来自转换后的 boundingBox 的左上点。转换后的 boundingBox 来自原点角点：

height = img.shape[0]
width = img.shape[1]
#..get T1
#..get M
pnts = np.asarray([
    [0, 0],
    [width, 0],
    [width, height],
    [0, height]
    ], dtype=np.float32)
pnts = np.array([pnts])
dst_pnts = cv2.perspectiveTransform(pnts, M * T1)[0]
dst_pnts = np.asarray(dst_pnts, dtype=np.float32)
bbox = cv2.boundingRect(dst_pnts)
T2 = np.matrix([[1., 0., 0 - bbox[0]],
                [0., 1., 0 - bbox[1]],
                [0., 0., 1.]])