Android SDK：获取原始预览相机图像而不显示它

Android SDK: Get raw preview camera image without displaying it

提问by Dawamaha

I want to do image processing with a raw image without showing it on the screen, which obviously reduces performance.

我想对原始图像进行图像处理而不在屏幕上显示，这显然会降低性能。

According to the answers to this thread Taking picture from camera without previewit was not possible in Android 1.5, but does anybody know if it is possible in Android 4 (API level 15)?

根据此线程的答案，在没有预览的情况下从相机拍摄照片在 Android 1.5 中是不可能的，但是有人知道在 Android 4（API 级别 15）中是否可能？

回答by Eddy Talvala

In Android 4, the simplest option to receiving raw image data without displaying it on screen is to use the Camera.setPreviewTexture()call to route the preview frames to the GPU.

在 Android 4 中，接收原始图像数据而不在屏幕上显示它的最简单选项是使用Camera.setPreviewTexture()调用将预览帧路由到 GPU。

You can use this in two ways:

您可以通过两种方式使用它：

1. Do your actual processing on the GPU: Set up an OpenGL context (OpenGL ES 2 tutorial), and create a SurfaceTextureobject in that context. Then pass that object to setPreviewTexture, and start preview. Then, in your OpenGL code, you can call SurfaceTexture.updateTexImage, and the texture ID associated with the SurfaceTexture will be updated to the latest preview frame from the camera. You can also read back the RGB texture data to the CPU for further processing using glReadPixels, if desired.
2. Do your processing on the CPU: You can simply create a dummy SurfaceTexture object without any OpenGL context set up. Pass any integer you want as the texture ID, and connect the SurfaceTexture to the camera using setPreviewTexture. As long as you don't call updateTexImage, the SurfaceTexture will simply discard all data passed into it by the camera. Then set up preview callbacks using setPreviewCallback, and use that data (typically in a YUV format) for CPU processing. This is probably less efficient than #1, but does not require knowing OpenGL.

1. 在 GPU 上进行实际处理：设置 OpenGL 上下文（OpenGL ES 2 教程），并在该上下文中创建SurfaceTexture对象。然后将该对象传递给 setPreviewTexture，并开始预览。然后，在您的 OpenGL 代码中，您可以调用 SurfaceTexture.updateTexImage，与 SurfaceTexture 关联的纹理 ID 将更新为来自相机的最新预览帧。如果需要，您还可以使用 glReadPixels 将 RGB 纹理数据读回 CPU 以进行进一步处理。
2. 在 CPU 上进行处理：您可以简单地创建一个虚拟 SurfaceTexture 对象，而无需设置任何 OpenGL 上下文。传递任何您想要的整数作为纹理 ID，并使用 setPreviewTexture 将 SurfaceTexture 连接到相机。只要您不调用 updateTexImage，SurfaceTexture 就会简单地丢弃由相机传入的所有数据。然后使用setPreviewCallback设置预览回调，并使用该数据（通常为 YUV 格式）进行 CPU 处理。这可能比 #1 效率低，但不需要了解 OpenGL。

回答by bond

Since I am not allowed to comment. Regarding Eddy's answer. You need to work with this in the NDK as using the Java interface will negate any performance benefit. Having to work with a PixelBuffer is absolutely insane from a performance standpoint. Your conversion from RGBA888 to YUV also needs to be done in C.

因为我不允许发表评论。关于艾迪的回答。您需要在 NDK 中使用它，因为使用 Java 接口将抵消任何性能优势。从性能的角度来看，不得不使用 PixelBuffer 是绝对疯狂的。您从 RGBA888 到 YUV 的转换也需要在 C 中完成。

Do not try using a TextureView as is will be even worse. You would have to copy the Pixels into a Bitmap then from a Bitmap into an Array all before the conversion to YUV. This, by itself, takes almost 30% of the cpu utilization on a brand spanking new Nexus 7 2013.

不要尝试使用 TextureView，因为它会更糟。在转换为 YUV 之前，您必须将像素复制到位图，然后从位图复制到数组。这本身就占据了全新 Nexus 7 2013 品牌近 30% 的 CPU 利用率。

The most efficient way is to talk to Camera.h directly and bypass all of the Android APIs. You can create your own buffer and intercept the YUV data before it goes anywhere else.

最有效的方法是直接与 Camera.h 对话并绕过所有 Android API。您可以创建自己的缓冲区并在 YUV 数据进入其他任何地方之前拦截它。

回答by Alex Cohn

Showing preview on the screen does not have performance consequences. On all devices I met, the camera output is "wired" to a surface or texture with no CPU involved, all color conversion and scaling taken care of by dedicated hardware.

在屏幕上显示预览不会对性能产生影响。在我遇到的所有设备上，相机输出都“连接”到表面或纹理，不涉及 CPU，所有颜色转换和缩放都由专用硬件处理。

There may be other reasons to "hide" the preview, but keep in mind that the purpose of the API initially was to make sure that the end user sees whatever arrives from the camera to the application, for privacy and security reasons.

“隐藏”预览可能还有其他原因，但请记住，出于隐私和安全原因，API 最初的目的是确保最终用户看到从相机到达应用程序的任何内容。