Java 在Android中从视频图像中获取帧
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/1893072/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Getting frames from Video Image in Android
提问by Alexander Stolz
I've implemented a simple application which shows the camera picture on the screen. What I like to do now is grab a single frame and process it as bitmap. From what I could find out to this point it is not an easy thing to do.
我已经实现了一个简单的应用程序，它在屏幕上显示相机图片。我现在喜欢做的是抓取单个帧并将其作为位图处理。从我所能发现的到这一点，这不是一件容易的事情。
I've tried using the onPreviewFrame method with which you get the current frame as a byte array and tried to decode it with the BitmapFactory class but it returns null. The format of the frame is a headerless YUV which could be translated to bitmap but it takes too long on a phone. Also I've read that the onPreviewFrame method has contraints on the runtime, if it takes too long the application could crash.
我尝试使用 onPreviewFrame 方法，通过该方法将当前帧作为字节数组获取，并尝试使用 BitmapFactory 类对其进行解码，但它返回 null。帧的格式是无头的 YUV，可以转换为位图，但在手机上需要很长时间。我还读到 onPreviewFrame 方法对运行时有限制，如果时间过长，应用程序可能会崩溃。
So what is the right way to do this?
那么这样做的正确方法是什么？
采纳答案by Tim
In API 17+, you can do conversion to RGBA888 from NV21 with the 'ScriptIntrinsicYuvToRGB' RenderScript. This allows you to easily process preview frames without manually encoding/decoding frames:
在 API 17+ 中，您可以使用 'ScriptIntrinsicYuvToRGB' RenderScript 从 NV21 转换为 RGBA888。这使您可以轻松处理预览帧，而无需手动编码/解码帧：
@Override
public void onPreviewFrame(byte[] data, Camera camera) {
Bitmap bitmap = Bitmap.createBitmap(r.width(), r.height(), Bitmap.Config.ARGB_8888);
Allocation bmData = renderScriptNV21ToRGBA888(
mContext,
r.width(),
r.height(),
data);
bmData.copyTo(bitmap);
}
public Allocation renderScriptNV21ToRGBA888(Context context, int width, int height, byte[] nv21) {
RenderScript rs = RenderScript.create(context);
ScriptIntrinsicYuvToRGB yuvToRgbIntrinsic = ScriptIntrinsicYuvToRGB.create(rs, Element.U8_4(rs));
Type.Builder yuvType = new Type.Builder(rs, Element.U8(rs)).setX(nv21.length);
Allocation in = Allocation.createTyped(rs, yuvType.create(), Allocation.USAGE_SCRIPT);
Type.Builder rgbaType = new Type.Builder(rs, Element.RGBA_8888(rs)).setX(width).setY(height);
Allocation out = Allocation.createTyped(rs, rgbaType.create(), Allocation.USAGE_SCRIPT);
in.copyFrom(nv21);
yuvToRgbIntrinsic.setInput(in);
yuvToRgbIntrinsic.forEach(out);
return out;
}
回答by Alexander Stolz
Ok what we ended up doing is using the onPreviewFrame method and decoding the data in a seperate Thread using a method which can be found in the android help group.
好的，我们最终做的是使用 onPreviewFrame 方法并使用可在 android 帮助组中找到的方法在单独的线程中解码数据。
decodeYUV(argb8888, data, camSize.width, camSize.height);
Bitmap bitmap = Bitmap.createBitmap(argb8888, camSize.width,
camSize.height, Config.ARGB_8888);
...
...
// decode Y, U, and V values on the YUV 420 buffer described as YCbCr_422_SP by Android
// David Manpearl 081201
public void decodeYUV(int[] out, byte[] fg, int width, int height)
throws NullPointerException, IllegalArgumentException {
int sz = width * height;
if (out == null)
throw new NullPointerException("buffer out is null");
if (out.length < sz)
throw new IllegalArgumentException("buffer out size " + out.length
+ " < minimum " + sz);
if (fg == null)
throw new NullPointerException("buffer 'fg' is null");
if (fg.length < sz)
throw new IllegalArgumentException("buffer fg size " + fg.length
+ " < minimum " + sz * 3 / 2);
int i, j;
int Y, Cr = 0, Cb = 0;
for (j = 0; j < height; j++) {
int pixPtr = j * width;
final int jDiv2 = j >> 1;
for (i = 0; i < width; i++) {
Y = fg[pixPtr];
if (Y < 0)
Y += 255;
if ((i & 0x1) != 1) {
final int cOff = sz + jDiv2 * width + (i >> 1) * 2;
Cb = fg[cOff];
if (Cb < 0)
Cb += 127;
else
Cb -= 128;
Cr = fg[cOff + 1];
if (Cr < 0)
Cr += 127;
else
Cr -= 128;
}
int R = Y + Cr + (Cr >> 2) + (Cr >> 3) + (Cr >> 5);
if (R < 0)
R = 0;
else if (R > 255)
R = 255;
int G = Y - (Cb >> 2) + (Cb >> 4) + (Cb >> 5) - (Cr >> 1)
+ (Cr >> 3) + (Cr >> 4) + (Cr >> 5);
if (G < 0)
G = 0;
else if (G > 255)
G = 255;
int B = Y + Cb + (Cb >> 1) + (Cb >> 2) + (Cb >> 6);
if (B < 0)
B = 0;
else if (B > 255)
B = 255;
out[pixPtr++] = 0xff000000 + (B << 16) + (G << 8) + R;
}
}
}
链接：http: //groups.google.com/group/android-developers/browse_thread/thread/c85e829ab209ceea/3f180a16a4872b58?lnk=gst&q=onpreviewframe#3f180a16a4872b58
回答by Codevalley
I actually tried the code given the previous answer found that the Colorvalues are not exact. I checked it by taking both the preview and the camera.takePicture which directly returns a JPEG array. And the colors were very different. After a little bit more searching I found another example to convert the PreviewImage from YCrCb to RGB:
我实际上尝试了给出先前答案的代码，发现 Colorvalues 不准确。我通过预览和直接返回 JPEG 数组的 camera.takePicture 来检查它。而且颜色非常不同。经过多一点搜索，我找到了另一个将 PreviewImage 从 YCrCb 转换为 RGB 的示例：
static public void decodeYUV420SP(int[] rgb, byte[] yuv420sp, int width, int height) {
final int frameSize = width * height;
for (int j = 0, yp = 0; j < height; j++) {
int uvp = frameSize + (j >> 1) * width, u = 0, v = 0;
for (int i = 0; i < width; i++, yp++) {
int y = (0xff & ((int) yuv420sp[yp])) - 16;
if (y < 0) y = 0;
if ((i & 1) == 0) {
v = (0xff & yuv420sp[uvp++]) - 128;
u = (0xff & yuv420sp[uvp++]) - 128;
}
int y1192 = 1192 * y;
int r = (y1192 + 1634 * v);
int g = (y1192 - 833 * v - 400 * u);
int b = (y1192 + 2066 * u);
if (r < 0) r = 0; else if (r > 262143) r = 262143;
if (g < 0) g = 0; else if (g > 262143) g = 262143;
if (b < 0) b = 0; else if (b > 262143) b = 262143;
rgb[yp] = 0xff000000 | ((r << 6) & 0xff0000) | ((g >> 2) & 0xff00) | ((b >> 10) & 0xff);
}
}
}
The color values given by this and the takePicture() exactly match. I thought I should post it here. This is where I got this code from.Hope this helps.
由 this 和 takePicture() 给出的颜色值完全匹配。我想我应该把它贴在这里。 这是我从那里得到这个代码的地方。希望这可以帮助。
回答by Miao Wang
Tim's RenderScript solution is great. Two comments here though:
Tim 的 RenderScript 解决方案很棒。不过这里有两条评论：
- Create and reuse
RenderScript rs, andAllocation in, out. Creating them every frame will hurt the performance. - RenderScript support librarycan help you back support to Android 2.3.
- 创建和重用
RenderScript rs，以及Allocation in, out。每一帧都创建它们会影响性能。 - RenderScript 支持库可以帮助您恢复对 Android 2.3 的支持。
