Java中的图像处理边缘检测

Image Processing Edge Detection in Java

提问by ThePrince

This is my situation. It involves aligning a scanned image which will account for incorrect scanning. I must align the scanned image with my Java program.

这是我的情况。它涉及对齐扫描图像，这将解释不正确的扫描。我必须将扫描的图像与我的 Java 程序对齐。

These are more details:

这些是更多详细信息：

• There is a table-like form printed on a sheet of paper, which will be scanned into an image file.
• I will open the picture with Java, and I will have an OVERLAY of text boxes.
• The text boxes are supposed to align correctly with the scanned image.
• In order to align correctly, my Java program must analyze the scanned image and detect the coordinates of the edges of the table on the scanned image, and thus position the image and the textboxes so that the textboxes and the image both align properly (in case of incorrect scanning)

• 在一张纸上打印了一张类似表格的表格，该表格将被扫描成一个图像文件。
• 我会用Java打开图片，我会有一个文本框的OVERLAY。
• 文本框应该与扫描的图像正确对齐。
• 为了正确对齐，我的 Java 程序必须分析扫描图像并检测扫描图像上表格边缘的坐标，从而定位图像和文本框，使文本框和图像都正确对齐（以防万一不正确的扫描）

You see, the guy scanning the image might not necessarily place the image in a perfectly correct position, so I need my program to automatically align the scanned image as it loads it. This program will be reusable on many of such scanned images, so I need the program to be flexible in this way.

你看，扫描图像的人可能不一定把图像放在一个完全正确的位置，所以我需要我的程序在加载它时自动对齐扫描的图像。该程序可在许多此类扫描图像上重复使用，因此我需要该程序以这种方式灵活。

My question is one of the following:

我的问题是以下之一：

1. How can I use Java to detect the y coordinate of the upper edge of the table and the x-coordinate of the leftmost edge of the table. The table is a a regular table with many cells, with black thin border, printed on a white sheet of paper (horizontal printout)

2. If an easier method exists to automatically align the scanned image in such a way that all scanned images will have the graphical table align to the same x, y coordinates, then share this method :).

3. If you don't know the answer to the above to questions, do tell me where I should start. I don't know much about graphics java programming and I have about 1 month to finish this program. Just assume that I have a tight schedule and I have to make the graphics part as simple as possible for me.

1. 如何使用Java检测表格上边缘的y坐标和表格最左边缘的x坐标。表格是一张普通表格，有很多单元格，黑色细边框，打印在一张白纸上（水平打印）

2. 如果存在一种更简单的方法来自动对齐扫描图像，使所有扫描图像都将图形表对齐到相同的 x、y 坐标，那么共享此方法:)。

3. 如果您不知道上述问题的答案，请告诉我应该从哪里开始。我不太了解图形Java编程，我有大约1个月的时间来完成这个程序。假设我的日程安排很紧，我必须让图形部分对我来说尽可能简单。

Cheers and thank you.

干杯和谢谢你。

采纳答案by ThePrince

I researched the libraries but in the end I found it more convenient to code up my own edge detection methods.

我研究了这些库，但最后我发现编写自己的边缘检测方法更方便。

The class below will detect black/grayed out edges of a scanned sheet of paper that contains such edges, and will return the x and y coordinate of the edges of the sheet of paper, starting from the rightmost end (reverse = true) or from lower end (reverse = true) or from the top edge (reverse = false) or from left edge (reverse = false). Also...the program will take ranges along vertical edges (rangex) measured in pixels, and horizontal ranges (rangey) measured in pixels. The ranges determine outliers in the points received.

下面的类将检测包含此类边缘的扫描纸张的黑色/灰色边缘，并返回纸张边缘的 x 和 y 坐标，从最右端开始（反向 = true）或从下端 (reverse = true) 或从顶部边缘 (reverse = false) 或从左边缘 (reverse = false)。此外...该程序将沿以像素为单位测量的垂直边缘 (rangex) 和以像素为单位测量的水平范围 (rangey) 获取范围。范围确定接收点中的异常值。

The program does 4 vertical cuts using the specified arrays, and 4 horizontal cuts. It retrieves the values of the dark dots. It uses the ranges to eliminate outliers. Sometimes, a little spot on the paper may cause an outlier point. The smaller the range, the fewer the outliers. However, sometimes the edge is slightly tilted, so you don't want to make the range too small.

该程序使用指定的阵列进行 4 次垂直切割，以及 4 次水平切割。它检索暗点的值。它使用范围来消除异常值。有时，纸上的一个小点可能会导致异常点。范围越小，异常值越少。但是，有时边缘会稍微倾斜，因此您不想将范围设置得太小。

Have fun. It works perfectly for me.

玩得开心。它非常适合我。

import java.awt.image.BufferedImage;
import java.awt.Color;
import java.util.ArrayList;
import java.lang.Math;
import java.awt.Point;
public class EdgeDetection {

    public App ap;
        public int[] horizontalCuts = {120, 220, 320, 420};
        public int[] verticalCuts = {300, 350, 375, 400};



    public void printEdgesTest(BufferedImage image, boolean reversex, boolean reversey, int rangex, int rangey){
        int[] mx = horizontalCuts;
        int[] my = verticalCuts;

            //you are getting edge points here
            //the "true" parameter indicates that it performs a cut starting at 0. (left edge)
        int[] xEdges = getEdges(image, mx, reversex, true);
        int edgex = getEdge(xEdges, rangex);
        for(int x = 0; x < xEdges.length; x++){
            System.out.println("EDGE = " + xEdges[x]);
        }
        System.out.println("THE EDGE = " + edgex);
            //the "false" parameter indicates you are doing your cut starting at the end (image.getHeight)
            //and ending at 0
            //if the parameter was true, it would mean it would start the cuts at y = 0
        int[] yEdges = getEdges(image, my, reversey, false);
        int edgey = getEdge(yEdges, rangey);
        for(int y = 0; y < yEdges.length; y++){
            System.out.println("EDGE = " + yEdges[y]);
        }
        System.out.println("THE EDGE = " + edgey);
    }

    //This function takes an array of coordinates...detects outliers, 
    //and computes the average of non-outlier points.

    public int getEdge(int[] edges, int range){
        ArrayList<Integer> result = new ArrayList<Integer>();
        boolean[] passes = new boolean[edges.length];
        int[][] differences = new int[edges.length][edges.length-1];
        //THIS CODE SEGMENT SAVES THE DIFFERENCES BETWEEN THE POINTS INTO AN ARRAY
        for(int n = 0; n<edges.length; n++){
            for(int m = 0; m<edges.length; m++){
                if(m < n){
                    differences[n][m] = edges[n] - edges[m];
                }else if(m > n){
                    differences[n][m-1] = edges[n] - edges[m];
                }
            }
        }
         //This array determines which points are outliers or nots (fall within range of other points)
        for(int n = 0; n<edges.length; n++){
            passes[n] = false;
            for(int m = 0; m<edges.length-1; m++){
                if(Math.abs(differences[n][m]) < range){
                    passes[n] = true;
                    System.out.println("EDGECHECK = TRUE" + n);
                    break;
                }
            }
        }
         //Create a new array only using valid points
        for(int i = 0; i<edges.length; i++){
            if(passes[i]){
                result.add(edges[i]);
            }
        }

        //Calculate the rounded mean... This will be the x/y coordinate of the edge
        //Whether they are x or y values depends on the "reverse" variable used to calculate the edges array
        int divisor = result.size();
        int addend = 0;
        double mean = 0;
        for(Integer i : result){
            addend += i;
        }
        mean = (double)addend/(double)divisor;

        //returns the mean of the valid points: this is the x or y coordinate of your calculated edge.
        if(mean - (int)mean >= .5){
            System.out.println("MEAN " + mean);
            return (int)mean+1;
        }else{
            System.out.println("MEAN " + mean);
            return (int)mean;
        }       
    }


     //this function computes "dark" points, which include light gray, to detect edges.
     //reverse - when true, starts counting from x = 0 or y = 0, and ends at image.getWidth or image.getHeight()
     //verticalEdge - determines whether you want to detect a vertical edge, or a horizontal edge
     //arr[] - determines the coordinates of the vertical or horizontal cuts you will do
     //set the arr[] array according to the graphical layout of your scanned image
     //image - this is the image you want to detect black/white edges of
    public int[] getEdges(BufferedImage image, int[] arr, boolean reverse, boolean verticalEdge){
        int red = 255;
        int green = 255;
        int blue = 255;
        int[] result = new int[arr.length];
        for(int n = 0; n<arr.length; n++){
            for(int m = reverse ? (verticalEdge ? image.getWidth():image.getHeight())-1:0; reverse ? m>=0:m<(verticalEdge ? image.getWidth():image.getHeight());){
                Color c = new Color(image.getRGB(verticalEdge ? m:arr[n], verticalEdge ? arr[n]:m));
                red = c.getRed();
                green = c.getGreen();
                blue = c.getBlue();
                        //determine if the point is considered "dark" or not.
                        //modify the range if you want to only include really dark spots.
                        //occasionally, though, the edge might be blurred out, and light gray helps
                if(red<239 && green<239 && blue<239){
                    result[n] = m;
                    break;
                }
                        //count forwards or backwards depending on reverse variable
                if(reverse){
                    m--;
                }else{
                    m++;
                }
            }
        }
    return result;
    }

}

回答by Edwin Buck

Edge detection is something that is typically done by enhancing the contrast between neighboring pixels, such that you get a easily detectable line, which is suitable for further processing.

边缘检测通常是通过增强相邻像素之间的对比度来完成的，这样您就可以获得一条易于检测的线条，适合进一步处理。

To do this, a "kernel" transforms a pixel according it the pixel's inital value, and the value of that pixel's neighbors. A good edge detection kernel will enhance the differences between neighboring pixels, and reduce the strength of a pixel with similar neigbors.

为此，“内核”根据像素的初始值和该像素的邻居的值来转换像素。一个好的边缘检测内核会增强相邻像素之间的差异，并降低具有相似相邻像素的像素的强度。

I would start by looking at the Sobel operator. This might not return results that are immediately useful to you; however, it will get you far closer than you would be if you were to approach the problem with little knowledge of the field.

我将从查看 Sobel 算子开始。这可能不会返回对您立即有用的结果；但是，如果您在对该领域知之甚少的情况下解决问题，它将使您离得更近。

After you have some crisp clean edges, you can use larger kernels to detect points where it seems that a 90% bend in two lines occurs, that might give you the pixel coordinates of the outer rectangle, which might be enough for your purposes.

在获得一些清晰的边缘后，您可以使用更大的内核来检测似乎在两条线中发生 90% 弯曲的点，这可能会为您提供外部矩形的像素坐标，这可能足以满足您的目的。

With those outer coordinates, it still is a bit of math to make the new pixels be composted with the average values between the old pixels rotated and moved to "match". The results (especially if you do not know about anti-aliasing math) can be pretty bad, adding blur to the image.

使用这些外部坐标，将新像素与旋转并移动到“匹配”的旧像素之间的平均值进行堆肥仍然需要一些数学运算。结果（尤其是在您不了解抗锯齿数学的情况下）可能非常糟糕，会增加图像的模糊度。

Sharpening filters might be a solution, but they come with their own issues, mainly they make the picture sharper by adding graininess. Too much, and it is obvious that the original image is not a high-quality scan.

锐化滤镜可能是一种解决方案，但它们也有自己的问题，主要是通过增加颗粒感使图片更清晰。太多了，很明显原始图像不是高质量的扫描。

回答by Cloud

A similar such problem I've done in the past basically figured out the orientation of the form, re-aligned it, re-scaled it, and I was all set. You can use the Hough transform to to detect the angular offset of the image (ie: how much it is rotated), but you still need to detect the boundaries of the form. It also had to accommodate for the boundaries of the piece of paper itself.

我过去做过的一个类似这样的问题基本上弄清楚了表单的方向，重新对齐它，重新缩放它，我一切都准备好了。您可以使用霍夫变换来检测图像的角度偏移（即：旋转了多少），但您仍然需要检测表单的边界。它还必须适应纸张本身的边界。

This was a lucky break for me, because it basically showed a black and white image in the middle of a big black border.

这对我来说是一个幸运的突破，因为它基本上在一个大的黑色边框中间显示了一个黑白图像。

1. Apply an aggressive, 5x5 median filter to remove some noise.
2. Convert from grayscale to black and white (rescale intensity values from [0,255] to [0,1]).
3. Calculate the Principal Component Analysis (ie: calculate the Eigenvectors of the covariance matrix for your image from the calculated Eigenvalues) (http://en.wikipedia.org/wiki/Principal_component_analysis#Derivation_of_PCA_using_the_covariance_method) 4) This gives you a basis vector. You simply use that to re-orient your image to a standard basis matrix (ie: [1,0],[0,1]).

1. 应用积极的 5x5 中值滤波器来去除一些噪音。
2. 从灰度转换为黑白（将强度值从 [0,255] 重新调整为 [0,1]）。
3. 计算主成分分析（即：根据计算的特征值计算图像的协方差矩阵的特征向量）（http://en.wikipedia.org/wiki/Principal_component_analysis#Derivation_of_PCA_using_the_covariance_method）4）这为您提供了一个基向量。您只需使用它来将您的图像重新定向为标准基矩阵（即：[1,0],[0,1]）。

Your image is now aligned beautifully. I did this for normalizing the orientation of MRI scans of entire human brains.

您的图像现在已完美对齐。我这样做是为了标准化整个人类大脑的 MRI 扫描方向。

You also know that you have a massive black border around the actual image. You simply keep deleting rows from the top and bottom, and both sides of the image until they are all gone. You can temporarily apply a 7x7 median or mode filter to a copy of the image so far at this point. It helps rule out too much border remaining in the final image from thumbprints, dirt, etc.

您还知道实际图像周围有大量黑色边框。您只需不断地从图像的顶部和底部以及两侧删除行，直到它们全部消失。到目前为止，您可以暂时将 7x7 中值或模式过滤器应用于图像的副本。它有助于从指纹、污垢等中排除最终图像中剩余的过多边框。

回答by Gabriel Ambrósio Archanjo

Try to start from a simple scenario and then improve the approach.

尝试从一个简单的场景开始，然后改进方法。

1. Detect corners.
2. Find the corners in the boundaries of the form.
3. Using the form corners coordinates, calculate the rotation angle.
4. Rotate/scale the image.
5. Map the position of each field in the form relative to form origin coordinates.
6. Match the textboxes.

1. 检测角点。
2. 找到表单边界中的角。
3. 使用窗体角坐标，计算旋转角度。
4. 旋转/缩放图像。
5. 映射表单中每个字段相对于表单原点坐标的位置。
6. 匹配文本框。

The program presented at the end of this post does the steps 1 to 3. It was implemented using Marvin Framework. The image below shows the output image with the detected corners.

本文末尾提供的程序执行步骤 1 到 3。它是使用Marvin Framework实现的。下图显示了带有检测到的角点的输出图像。

The program also outputs: Rotation angle:1.6365770416167182

程序还输出：旋转角度：1.6365770416167182

Source code:

源代码：

import java.awt.Color;
import java.awt.Point;
import marvin.image.MarvinImage;
import marvin.io.MarvinImageIO;
import marvin.plugin.MarvinImagePlugin;
import marvin.util.MarvinAttributes;
import marvin.util.MarvinPluginLoader;

public class FormCorners {

public FormCorners(){
    // Load plug-in
    MarvinImagePlugin moravec = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.corner.moravec");
    MarvinAttributes attr = new MarvinAttributes();

    // Load image
    MarvinImage image = MarvinImageIO.loadImage("./res/printedForm.jpg");

    // Process and save output image
    moravec.setAttribute("threshold", 2000);
    moravec.process(image, null, attr);
    Point[] boundaries = boundaries(attr);
    image = showCorners(image, boundaries, 12);
    MarvinImageIO.saveImage(image, "./res/printedForm_output.jpg");

    // Print rotation angle
    double angle =  (Math.atan2((boundaries[1].y*-1)-(boundaries[0].y*-1),boundaries[1].x-boundaries[0].x) * 180 / Math.PI);
    angle =  angle >= 0 ? angle : angle + 360;
    System.out.println("Rotation angle:"+angle);
}

private Point[] boundaries(MarvinAttributes attr){
    Point upLeft = new Point(-1,-1);
    Point upRight = new Point(-1,-1);
    Point bottomLeft = new Point(-1,-1);
    Point bottomRight = new Point(-1,-1);
    double ulDistance=9999,blDistance=9999,urDistance=9999,brDistance=9999;
    double tempDistance=-1;
    int[][] cornernessMap = (int[][]) attr.get("cornernessMap");

    for(int x=0; x<cornernessMap.length; x++){
        for(int y=0; y<cornernessMap[0].length; y++){
            if(cornernessMap[x][y] > 0){
                if((tempDistance = Point.distance(x, y, 0, 0)) < ulDistance){
                    upLeft.x = x; upLeft.y = y;
                    ulDistance = tempDistance;
                } 
                if((tempDistance = Point.distance(x, y, cornernessMap.length, 0)) < urDistance){
                    upRight.x = x; upRight.y = y;
                    urDistance = tempDistance;
                }
                if((tempDistance = Point.distance(x, y, 0, cornernessMap[0].length)) < blDistance){
                    bottomLeft.x = x; bottomLeft.y = y;
                    blDistance = tempDistance;
                }
                if((tempDistance = Point.distance(x, y, cornernessMap.length, cornernessMap[0].length)) < brDistance){
                    bottomRight.x = x; bottomRight.y = y;
                    brDistance = tempDistance;
                }
            }
        }
    }
    return new Point[]{upLeft, upRight, bottomRight, bottomLeft};
}

private MarvinImage showCorners(MarvinImage image, Point[] points, int rectSize){
    MarvinImage ret = image.clone();
    for(Point p:points){
        ret.fillRect(p.x-(rectSize/2), p.y-(rectSize/2), rectSize, rectSize, Color.red);
    }
    return ret;
}

public static void main(String[] args) {
    new FormCorners();
}
}