Java 计算机视觉 - 使用 OpenCV 过滤凸包和凸性缺陷
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/18143077/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Computer Vision - filtering convex hulls and convexity defects with OpenCV
提问by Marek
I have the problem with processing digital signals. I am trying to detect fingertips, similar to the solution that is presented here: Hand and finger detection using JavaCV.
我有处理数字信号的问题。我正在尝试检测指尖，类似于此处提供的解决方案：使用 JavaCV 检测手和手指。
However, I am not using JavaCV but OpenCV for android which is slightly different. I have managed to do all the steps presented in the tutorial, but filtering of convex hulls and convexity defects. This is how my image looks like:
但是，我没有使用 JavaCV，而是使用 OpenCV for android，这略有不同。我已经完成了教程中介绍的所有步骤，但过滤了凸包和凸性缺陷。这是我的图像的样子：
Here is an image in another resolution:
这是另一种分辨率的图像：
As you can clearly see, There is to many yellow points (convex hulls) and also to many red points (convexity deffects). Sometimes between 2 yellow points there is no red point, which is quite strange (how are convex hulls calculated?)
您可以清楚地看到，有很多黄色点（凸包）和很多红色点（凸面效果）。有时在2个黄点之间没有红点，这很奇怪（凸包是如何计算的？）
What I need is to create simillar filtering function like in the link provided before, but using data structures of OpenCV.
我需要的是像之前提供的链接一样创建类似的过滤功能，但使用 OpenCV 的数据结构。
Convex Hulls are type of MatOfInt ... Convexity defects are type of MatOfInt4 ...
凸包是 MatOfInt 类型...凸面缺陷是 MatOfInt4 类型...
I created also some additional data structures, because stupid OpenCV uses different types of data containing same data, in different methods...
我还创建了一些额外的数据结构，因为愚蠢的 OpenCV 使用不同类型的包含相同数据的数据，以不同的方法......
convexHullMatOfInt = new MatOfInt();
convexHullPointArrayList = new ArrayList<Point>();
convexHullMatOfPoint = new MatOfPoint();
convexHullMatOfPointArrayList = new ArrayList<MatOfPoint>();
Here is what I did so far but it is not working good. The problem is probably with converting data in a wrong way:
这是我到目前为止所做的，但效果不佳。问题可能在于以错误的方式转换数据：
Creating convex hulls and convexity defects:
创建凸包和凸性缺陷：
public void calculateConvexHulls()
{
convexHullMatOfInt = new MatOfInt();
convexHullPointArrayList = new ArrayList<Point>();
convexHullMatOfPoint = new MatOfPoint();
convexHullMatOfPointArrayList = new ArrayList<MatOfPoint>();
try {
//Calculate convex hulls
if(aproximatedContours.size() > 0)
{
Imgproc.convexHull( aproximatedContours.get(0), convexHullMatOfInt, false);
for(int j=0; j < convexHullMatOfInt.toList().size(); j++)
convexHullPointArrayList.add(aproximatedContours.get(0).toList().get(convexHullMatOfInt.toList().get(j)));
convexHullMatOfPoint.fromList(convexHullPointArrayList);
convexHullMatOfPointArrayList.add(convexHullMatOfPoint);
}
} catch (Exception e) {
// TODO Auto-generated catch block
Log.e("Calculate convex hulls failed.", "Details below");
e.printStackTrace();
}
}
public void calculateConvexityDefects()
{
mConvexityDefectsMatOfInt4 = new MatOfInt4();
try {
Imgproc.convexityDefects(aproximatedContours.get(0), convexHullMatOfInt, mConvexityDefectsMatOfInt4);
if(!mConvexityDefectsMatOfInt4.empty())
{
mConvexityDefectsIntArrayList = new int[mConvexityDefectsMatOfInt4.toArray().length];
mConvexityDefectsIntArrayList = mConvexityDefectsMatOfInt4.toArray();
}
} catch (Exception e) {
Log.e("Calculate convex hulls failed.", "Details below");
e.printStackTrace();
}
}
Filtering:
过滤：
public void filterCalculatedPoints()
{
ArrayList<Point> tipPts = new ArrayList<Point>();
ArrayList<Point> foldPts = new ArrayList<Point>();
ArrayList<Integer> depths = new ArrayList<Integer>();
fingerTips = new ArrayList<Point>();
for (int i = 0; i < mConvexityDefectsIntArrayList.length/4; i++)
{
tipPts.add(contours.get(0).toList().get(mConvexityDefectsIntArrayList[4*i]));
tipPts.add(contours.get(0).toList().get(mConvexityDefectsIntArrayList[4*i+1]));
foldPts.add(contours.get(0).toList().get(mConvexityDefectsIntArrayList[4*i+2]));
depths.add(mConvexityDefectsIntArrayList[4*i+3]);
}
int numPoints = foldPts.size();
for (int i=0; i < numPoints; i++) {
if ((depths.get(i).intValue()) < MIN_FINGER_DEPTH)
continue;
// look at fold points on either side of a tip
int pdx = (i == 0) ? (numPoints-1) : (i - 1);
int sdx = (i == numPoints-1) ? 0 : (i + 1);
int angle = angleBetween(tipPts.get(i), foldPts.get(pdx), foldPts.get(sdx));
if (angle >= MAX_FINGER_ANGLE) // angle between finger and folds too wide
continue;
// this point is probably a fingertip, so add to list
fingerTips.add(tipPts.get(i));
}
}
Results (white points - fingertips after filtering):
结果（白点 - 过滤后的指尖）：
Could you help me to write proper function for filtering?
你能帮我写出合适的过滤函数吗？
UPDATE 14.08.2013
2013 年 8 月 14 日更新
I use standard openCV function for contour approximation. I have to change approximation value with resolution change, and hand-to-camera distance, which is quite hard to do. If the resolution is smaller, then finger consist of less pixel, thus approximation value should be lover. Same with the distance. Keeping it high will result in completely losing the finger. So I think approximation is not good approach to resolving the problem, however small value could be useful to speed up calculations:
我使用标准的 openCV 函数进行轮廓近似。我必须随着分辨率的变化和手到相机的距离来改变近似值，这很难做到。如果分辨率较小，则手指由较少的像素组成，因此近似值应为lover。距离也一样。保持高位会导致完全失去手指。所以我认为近似不是解决问题的好方法，但是小值可能有助于加速计算：
Imgproc.approxPolyDP(frame, frame, 2 , true);
If I use high values, then the result is like on the image below, which would be good only if distance and resolution wouldn't change. Also, I am quite surprised that default methods for hulls points and defects points doesn't have useful arguments to pass (min angle, distance etc)...
如果我使用高值，则结果如下图所示，只有在距离和分辨率不变的情况下才会好。此外，我很惊讶船体点和缺陷点的默认方法没有有用的参数来传递（最小角度、距离等）......
Image below presents the effect that I would like to achieve always, independently from resolution or hand-to-camera distance. Also I don't want to see any yellow points when I close my palm...
下图展示了我希望始终实现的效果，与分辨率或手到相机的距离无关。我也不想在我合上手掌时看到任何黄点...
To sum everything up, I would like to know:
总而言之，我想知道：
- how to filter the points
- how can I make resolution and distance independent approximation which will always work
- if someone knows or have some materials (graphical representation, explanation) about those data structures used in OpenCV, I would be happy to read it. (Mat, MatOfInt, MatOfPoint, MatOfPoint2, MatOfPoint4 etc.)
- 如何过滤点
- 如何使分辨率和距离无关的近似值始终有效
- 如果有人知道或有关于 OpenCV 中使用的那些数据结构的一些材料（图形表示、解释），我很乐意阅读它。（Mat、MatOfInt、MatOfPoint、MatOfPoint2、MatOfPoint4 等）
回答by weeska
I think you missed that point:
我想你错过了这一点：
Hull creation and defect analysis are speeded up by utilizing a low-polygon approximation of the contour rather than the original.
船体创建和缺陷分析通过使用轮廓的低多边形近似而不是原始轮廓来加速。
回答by jayprich
The convex hull at low res can be used to identify the position of the hand as a whole, it is not useful for fingers but does provide a region of interest and appropriate scale.
低分辨率的凸包可用于识别整个手的位置，它对手指没有用，但确实提供了感兴趣的区域和适当的比例。
The higher resolution analysis should then be applied to your approximated contour, it is easy to skip any points that do not pass the "length and angle" criteria from the last two, though you may wish to "average in" instead of "skip entirely".
然后应将更高分辨率的分析应用于您的近似轮廓，很容易跳过不通过最后两个“长度和角度”标准的任何点，尽管您可能希望“平均”而不是“完全跳过” ”。
Your code example is a single pass of calculating convexity defects and then removing them .. that is a logic error .. you need to remove points as you go .. (a) it is faster and simpler to do everything in one-pass (b) it avoids removing points at a first pass and having to add them back later because any removal changes previous calcs.
您的代码示例是一次计算凸性缺陷然后删除它们的单程..这是一个逻辑错误..您需要边走边删除点..（a）在一次通过中完成所有事情更快，更简单（ b) 它避免了在第一次通过时删除点并且稍后必须将它们添加回来，因为任何删除都会更改以前的计算。
This basic technique is very simple and so works for a basic open palm. It doesn't intrinsically understand a hand or a gesture though, so tuning the scale, angle and length parameters is only ever going to get you "so far".
这种基本技术非常简单，因此适用于基本的张开手掌。不过，它本质上并不理解手或手势，因此调整比例、角度和长度参数只会让您“到目前为止”。
References to Techniques: filter length and angle "Convexity defect" Simen Andresen blog http://simena86.github.io/blog/2013/08/12/hand-tracking-and-recognition-with-opencv/
技术参考：滤波器长度和角度“凸度缺陷” Simen Andresen 博客http://simena86.github.io/blog/2013/08/12/hand-tracking-and-recognition-with-opencv/
Kinect SDK based C# Library with added finger direction detection http://candescentnui.codeplex.com/http://blog.candescent.ch/2011/11/improving-finger-detection.html
基于 Kinect SDK 的 C# 库添加了手指方向检测 http://candescentnui.codeplex.com/ http://blog.candescent.ch/2011/11/improving-finger-detection.html
"Self-growing and organized neural gas" (SGONG) Prof Nikos Papamarkos http://www.papamarkos.gr/uploaded-files/Hand%20gesture%20recognition%20using%20a%20neural%20network%20shape%20fitting%20technique.pdf
“自我成长和有组织的神经气体”（SGONG）Nikos Papamarkos 教授http://www.papamarkos.gr/uploaded-files/Hand%20gesture%20recognition%20using%20a%20neural%20network%20shape%20fit%20technique.pdf
Commercial product David Holz & Michael Buckwald founders of "Leap Motion" http://www.engadget.com/2013/03/11/leap-motion-michael-buckwald-interview/
商业产品 David Holz 和 Michael Buckwald “Leap Motion”的创始人http://www.engadget.com/2013/03/11/leap-motion-michael-buckwald-interview/
