java 如何使用 JNI 在 C 中获取原始 Android 相机缓冲区？

How do I get the raw Android camera buffer in C using JNI?

提问by Nick

I've been searching Google and StackOverflow exhaustively and cannot find this. Maybe I'm missing something obvious. Thanks!

我一直在 Google 和 StackOverflow 上进行详尽的搜索，但找不到这个。也许我错过了一些明显的东西。谢谢！

(This is because the Java implementation of the preview callback [even with buffer] is too inefficient.)

（这是因为预览回调的 Java 实现 [即使有缓冲区] 效率太低。）

采纳答案by Andrey Ermakov

I made a little investigation on topic. This presentation(from p.277, Chinese) helped a lot.

我对这个话题做了一点调查。这个介绍（来自第 277 页，中文）帮助很大。

Camera preview call chain

相机预览调用链

As others mentioned, you can get a buffer using a Camera.setPreviewCallbackmethod.
Here's how it happens there (a verbose version):

正如其他人提到的，您可以使用一种Camera.setPreviewCallback方法获取缓冲区。
这是它在那里发生的方式（详细版本）：

1. User calls Camera.startPreview()which is a native function.
2. android_hardware_Camera_startPreviewcalls startPreviewmethod of C++ Cameraclass.
3. Cameracalls a startPreviewmethod of ICamerainterface
4. ICameramakes an IPCcall to remote client.
5. It calls a setCameraModemethod of CameraServiceclass.
6. CameraServicesets a window to display a preview and calls a startPreviewmethod of CameraHardwareInterfaceclass.
7. The latter tries to call a start_previewmethod on particular camera_device_tdevice.
   I didn't looked up further but it should perform a call to the driver.
8. When image arrives, dataCallbackof CameraServiceis invoked.
9. It passes data to handlePreviewDatamethod of client.
10. Client either copies the buffer or sends it directly to the ICameraClient.
11. ICameraClientsends it over IPCto the Camera.
12. Cameracalls a registered listener and passes buffer to JNI.
13. It invokes a callback in Java class. If user provided a buffer with Camera.addCallbackBufferthen it copies to the buffer first.
14. Finally Java class Camerahandles the message and invokes a onPreviewFramemethod of Camera.PreviewCallback.

1. 用户调用Camera.startPreview()这是一个本机函数。
2. android_hardware_Camera_startPreview调用startPreviewC++Camera类的方法。
3. Camera调用接口的startPreview方法ICamera
4. ICamera使一个IPC远程客户端调用。
5. 它调用类的setCameraMode方法CameraService。
6. CameraService设置一个窗口来显示预览并调用类的startPreview方法CameraHardwareInterface。
7. 后者尝试start_preview在特定camera_device_t设备上调用方法。
   我没有再往上看，但它应该会呼叫司机。
8. 当图像到达时，dataCallbackofCameraService被调用。
9. 它将数据传递给handlePreviewData客户端的方法。
10. 客户端要么复制缓冲区，要么直接将其发送到ICameraClient.
11. ICameraClient将其发送IPC到Camera.
12. Camera调用已注册的侦听器并将缓冲区传递给JNI.
13. 它调用 Java 类中的回调。如果用户提供了一个缓冲区，Camera.addCallbackBuffer那么它首先复制到缓冲区。
14. 最后 Java 类Camera处理消息并调用 的onPreviewFrame方法Camera.PreviewCallback。

As you can see 2 IPCcalls were invoked and buffer was copied at least twice on steps 10, 11. First instance of raw buffer which is returned by camera_device_tis hosted in another process and you cannot access it due to security checks in CameraService.

正如您所看到的IPC，在第 10 步和第 11 步中调用了2个调用并且缓冲区至少被复制了两次。 返回的原始缓冲区的第一个实例camera_device_t托管在另一个进程中，由于安全检查，您无法访问它CameraService。

Preview surface

预览表面

However, when you set a preview surface using either Camera.setPreviewTextureor Camera.setPreviewDisplayit is be passed directly to the camera device and refreshed in realtime without participation of all the chain above. As it's documentation says:

但是，当您使用Camera.setPreviewTexture或设置预览表面时，Camera.setPreviewDisplay它会直接传递到相机设备并实时刷新，而无需上述所有链的参与。正如它的文档所说：

Handle onto a raw buffer that is being managed by the screen compositor.

处理由屏幕合成器管理的原始缓冲区。

Java class Surfacehas a method to retrieve it's contents:

Java 类Surface有一个方法来检索它的内容：

public static native Bitmap screenshot(int width, int height, int minLayer, int maxLayer);

But this API is hidden. See i.e. this questionfor a way to use it.

但是这个API是隐藏的。请参阅 ie这个问题以了解使用它的方法。

回答by Eddy Talvala

There is no public API to do what you want; the only official (that is, guaranteed to work) method is the Java-level preview callbacks set up through calling Camera.setPreviewCallback(). In Android > 3.0, you can also use Camera.setPreviewTexture()to route preview data to the GPU, and process it there using GLES (or read it back to the CPU). The GPU path is what the ICS AOSP camera application uses for its video effects.

没有公共 API 可以做你想做的事；唯一的官方（即保证有效）方法是通过调用Camera.setPreviewCallback()设置的 Java 级预览回调。在 Android > 3.0 中，您还可以使用Camera.setPreviewTexture()将预览数据路由到 GPU，并在那里使用 GLES 对其进行处理（或将其读回 CPU）。GPU 路径是 ICS AOSP 相机应用程序用于其视频效果的路径。

Presumably, OpenCV and others have looked through the Android framework native code, and have bypassed the Java Camera API, talking to the services below directly.

想必是 OpenCV 等人翻遍了 Android 框架原生代码，绕过了 Java Camera API，直接与下面的服务对话。

This is fairly dangerous, because there is absolutely no guarantee that those interfaces won't change between Android versions, since they are not part of the public API. Using them may be fine now, and then when a user upgrades their device, your app will stop working.

这是相当危险的，因为绝对不能保证这些接口不会在 Android 版本之间改变，因为它们不是公共 API 的一部分。现在使用它们可能没问题，然后当用户升级他们的设备时，您的应用程序将停止工作。

回答by Jeff

Have you taken a look at OpenCV for Android. Their advanced tutorialsshow how to use JNI and there is a NativeProcessor object in their camera package.

你有没有看过OpenCV for Android。他们的高级教程展示了如何使用 JNI，并且他们的相机包中有一个 NativeProcessor 对象。