JavaScript：可靠地提取视频帧
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/32699721/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
JavaScript: Extract video frames reliably
提问by Ian Wizard
I'm working on a client-side project which lets a user supply a video file and apply basic manipulations to it. I'm trying to extract the frames from the video reliably. At the moment I have a <video>which I'm loading selected video into, and then pulling out each frame as follows:
我正在开发一个客户端项目，该项目允许用户提供视频文件并对其应用基本操作。我正在尝试可靠地从视频中提取帧。目前我有一个<video>我正在加载选定的视频，然后按如下方式拉出每一帧：
- Seek to the beginning
- Pause the video
- Draw
<video>to a<canvas> - Capture the frame from the canvas with
.toDataUrl() - Seek forward by 1 / 30 seconds (1 frame).
- Rinse and repeat
- 寻找起点
- 暂停视频
- 画
<video>到一个<canvas> - 从画布捕获帧
.toDataUrl() - 向前搜索 1 / 30 秒（1 帧）。
- 冲洗并重复
This is a rather inefficient process, and more specifically, is proving unreliable as I'm often getting stuck frames. This seems to be from it not updating the actual <video>element before it draws to the canvas.
这是一个相当低效的过程，更具体地说，事实证明这是不可靠的，因为我经常卡住帧。这似乎是因为它<video>在绘制到画布之前没有更新实际元素。
I'd rather not have to upload the original video to the server just to split the frames, and then download them back to the client.
我宁愿不必将原始视频上传到服务器只是为了分割帧，然后将它们下载回客户端。
Any suggestions for a better way to do this are greatly appreciated. The only caveat is that I need it to work with any format the browser supports (decoding in JS isn't a great option).
任何关于更好的方法的建议都非常感谢。唯一的警告是我需要它与浏览器支持的任何格式一起使用（在 JS 中解码不是一个很好的选择）。
采纳答案by Kaiido
Mostly taken from this great answerby GameAlchemist:
Since browsers doesn't respect videos' framerates, but instead "use of some tricks to make a match between the frame-rate of the movie and the refresh-rate of the screen", your assumption that every 30th of a second, a new frame will be painted is quite inaccurate.
However, the timeupdateeventshould fire when the currentTime has changed, and we can assume that a new frame was painted.
由于浏览器不尊重视频的帧率，而是“使用一些技巧来匹配电影的帧率和屏幕的刷新率”，您假设每 30 秒，一个新的画框会画得很不准确。
然而，当 currentTime 改变时timeupdate事件应该触发，我们可以假设一个新的帧被绘制。
So, I would do it like so :
所以，我会这样做：
document.querySelector('input').addEventListener('change', extractFrames, false);
function extractFrames() {
var video = document.createElement('video');
var array = [];
var canvas = document.createElement('canvas');
var ctx = canvas.getContext('2d');
var pro = document.querySelector('#progress');
function initCanvas(e) {
canvas.width = this.videoWidth;
canvas.height = this.videoHeight;
}
function drawFrame(e) {
this.pause();
ctx.drawImage(this, 0, 0);
/*
this will save as a Blob, less memory consumptive than toDataURL
a polyfill can be found at
https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement/toBlob#Polyfill
*/
canvas.toBlob(saveFrame, 'image/jpeg');
pro.innerHTML = ((this.currentTime / this.duration) * 100).toFixed(2) + ' %';
if (this.currentTime < this.duration) {
this.play();
}
}
function saveFrame(blob) {
array.push(blob);
}
function revokeURL(e) {
URL.revokeObjectURL(this.src);
}
function onend(e) {
var img;
// do whatever with the frames
for (var i = 0; i < array.length; i++) {
img = new Image();
img.onload = revokeURL;
img.src = URL.createObjectURL(array[i]);
document.body.appendChild(img);
}
// we don't need the video's objectURL anymore
URL.revokeObjectURL(this.src);
}
video.muted = true;
video.addEventListener('loadedmetadata', initCanvas, false);
video.addEventListener('timeupdate', drawFrame, false);
video.addEventListener('ended', onend, false);
video.src = URL.createObjectURL(this.files[0]);
video.play();
}<input type="file" accept="video/*" />
<p id="progress"></p>回答by Kaiido
Here's a working function that was tweaked from this question:
这是从这个问题调整的工作功能：
async function extractFramesFromVideo(videoUrl, fps=25) {
return new Promise(async (resolve) => {
// fully download it first (no buffering):
let videoBlob = await fetch(videoUrl).then(r => r.blob());
let videoObjectUrl = URL.createObjectURL(videoBlob);
let video = document.createElement("video");
let seekResolve;
video.addEventListener('seeked', async function() {
if(seekResolve) seekResolve();
});
video.src = videoObjectUrl;
// workaround chromium metadata bug (https://stackoverflow.com/q/38062864/993683)
while((video.duration === Infinity || isNaN(video.duration)) && video.readyState < 2) {
await new Promise(r => setTimeout(r, 1000));
video.currentTime = 10000000*Math.random();
}
let duration = video.duration;
let canvas = document.createElement('canvas');
let context = canvas.getContext('2d');
let [w, h] = [video.videoWidth, video.videoHeight]
canvas.width = w;
canvas.height = h;
let frames = [];
let interval = 1 / fps;
let currentTime = 0;
while(currentTime < duration) {
video.currentTime = currentTime;
await new Promise(r => seekResolve=r);
context.drawImage(video, 0, 0, w, h);
let base64ImageData = canvas.toDataURL();
frames.push(base64ImageData);
currentTime += interval;
}
resolve(frames);
});
});
}
}
Usage:
用法：
let frames = await extractFramesFromVideo("https://example.com/video.webm");
Note that there's currently no easy way to determine the actual/natural frame rate of a video unless perhaps you use ffmpeg.js, but that's a 10+ megabyte javascript file (since it's an emscripten port of the actual ffmpeg library, which is obviously huge).
请注意，目前没有简单的方法来确定视频的实际/自然帧速率，除非您可能使用ffmpeg.js，但这是一个 10+ 兆字节的 javascript 文件（因为它是实际 ffmpeg 库的 emscripten 端口，这显然是巨大的）。
