javascript 如何使用带有 XHR onProgress 函数的压缩/gzipped 内容？

How can I use deflated/gzipped content with an XHR onProgress function?

提问by Jimmy Sawczuk

I've seen a bunch of similar questions to this get asked before, but I haven't found one that describes my current problem exactly, so here goes:

我之前看过一堆类似的问题，但我还没有找到一个能准确描述我当前问题的问题，所以这里是：

I have a page which loads a large (between 0.5 and 10 MB) JSON document via AJAX so that the client-side code can process it. Once the file is loaded, I don't have any problems that I don't expect. However, it takes a long time to download, so I tried leveraging the XHR Progress APIto render a progress bar to indicate to the user that the document is loading. This worked well.

我有一个页面，它通过 AJAX 加载一个大的（0.5 到 10 MB 之间）JSON 文档，以便客户端代码可以处理它。一旦文件被加载，我就没有任何我不期望的问题。然而，下载需要很长时间，所以我尝试利用XHR Progress API来呈现一个进度条，以向用户表明文档正在加载。这工作得很好。

Then, in an effort to speed things up, I tried compressing the output on the server side via gzip and deflate. This worked too, with tremendous gains, however, my progress bar stopped working.

然后，为了加快速度，我尝试通过 gzip 和 deflate 压缩服务器端的输出。这也有效，获得了巨大的收益，但是，我的进度条停止工作。

I've looked into the issue for a while and found that if a proper Content-Lengthheader isn't sent with the requested AJAX resource, the onProgressevent handler cannot function as intended because it doesn't know how far along in the download it is. When this happens, a property called lengthComputableis set to falseon the event object.

我已经研究了一段时间，发现如果Content-Length没有与请求的 AJAX 资源一起发送正确的标头，则onProgress事件处理程序无法按预期运行，因为它不知道下载完成了多远。发生这种情况时，在事件对象上将调用的属性lengthComputable设置为false。

This made sense, so I tried setting the header explicitly with both the uncompressed and the compressed length of the output. I can verify that the header is being sent, and I can verify that my browser knows how to decompress the content. But the onProgresshandler still reports lengthComputable = false.

这是有道理的，所以我尝试使用输出的未压缩和压缩长度显式设置标头。我可以验证头是否正在发送，并且我可以验证我的浏览器是否知道如何解压缩内容。但是onProgress处理程序仍然报告lengthComputable = false.

So my question is: is there a way to gzipped/deflated content with the AJAX Progress API?And if so, what am I doing wrong right now?

所以我的问题是：有没有办法使用 AJAX Progress API 压缩/压缩内容？如果是这样，我现在做错了什么？

---

---

This is how the resource appears in the Chrome Network panel, showing that compression is working:

这是资源在 Chrome 网络面板中的显示方式，表明压缩正在起作用：

These are the relevant requestheaders, showing that the request is AJAX and that Accept-Encodingis set properly:

这些是相关的请求标头，显示请求是 AJAX 并且Accept-Encoding设置正确：

GET /dashboard/reports/ajax/load HTTP/1.1
Connection: keep-alive
Cache-Control: no-cache
Pragma: no-cache
Accept: application/json, text/javascript, */*; q=0.01
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.99 Safari/537.22
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

These are the relevant responseheaders, showing that the Content-Lengthand Content-Typeare being set correctly:

这些是相关的响应头，表明Content-Length和Content-Type设置正确：

HTTP/1.1 200 OK
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Content-Encoding: deflate
Content-Type: application/json
Date: Tue, 26 Feb 2013 18:59:07 GMT
Expires: Thu, 19 Nov 1981 08:52:00 GMT
P3P: CP="CAO PSA OUR"
Pragma: no-cache
Server: Apache/2.2.8 (Unix) mod_ssl/2.2.8 OpenSSL/0.9.8g PHP/5.4.7
X-Powered-By: PHP/5.4.7
Content-Length: 223879
Connection: keep-alive

For what it's worth, I've tried this on both a standard (http) and secure (https) connection, with no differences: the content loads fine in the browser, but isn't processed by the Progress API.

对于它的价值，我已经在标准 (http) 和安全 (https) 连接上尝试过，没有任何区别：内容在浏览器中加载良好，但未被 Progress API 处理。

---

---

Per Adam's suggestion, I tried switching the server side to gzip encoding with no success or change. Here are the relevant response headers:

根据Adam 的建议，我尝试将服务器端切换为 gzip 编码，但没有成功或更改。以下是相关的响应头：

HTTP/1.1 200 OK
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Content-Encoding: gzip
Content-Type: application/json
Date: Mon, 04 Mar 2013 22:33:19 GMT
Expires: Thu, 19 Nov 1981 08:52:00 GMT
P3P: CP="CAO PSA OUR"
Pragma: no-cache
Server: Apache/2.2.8 (Unix) mod_ssl/2.2.8 OpenSSL/0.9.8g PHP/5.4.7
X-Powered-By: PHP/5.4.7
Content-Length: 28250
Connection: keep-alive

Just to repeat: the content is being downloaded and decoded properly, it's just the progress API that I'm having trouble with.

重复一遍：正在正确下载和解码内容，这只是我遇到问题的进度 API。

---

---

Per Bertrand's request, here's the request:

根据Bertrand 的请求，请求如下：

$.ajax({
    url: '<url snipped>',
    data: {},
    success: onDone,
    dataType: 'json',
    cache: true,
    progress: onProgress || function(){}
});

And here's the onProgressevent handler I'm using (it's not too crazy):

这是onProgress我正在使用的事件处理程序（并不太疯狂）：

function(jqXHR, evt)
{
    // yes, I know this generates Infinity sometimes
    var pct = 100 * evt.position / evt.total;

    // just a method that updates some styles and javascript
    updateProgress(pct);
});

采纳答案by Jimmy Sawczuk

I wasn't able to solve the issue of using onProgresson the compressed content itself, but I came up with this semi-simple workaround. In a nutshell: send a HEADrequest to the server at the same time as a GETrequest, and render the progress bar once there's enough information to do so.

我无法解决在onProgress压缩内容本身上使用的问题，但我想出了这个半简单的解决方法。简而言之：在HEAD请求的同时向服务器发送GET请求，并在有足够的信息时呈现进度条。

---

---

function loader(onDone, onProgress, url, data)
{
    // onDone = event handler to run on successful download
    // onProgress = event handler to run during a download
    // url = url to load
    // data = extra parameters to be sent with the AJAX request
    var content_length = null;

    self.meta_xhr = $.ajax({
        url: url,
        data: data,
        dataType: 'json',
        type: 'HEAD',
        success: function(data, status, jqXHR)
        {
            content_length = jqXHR.getResponseHeader("X-Content-Length");
        }
    });

    self.xhr = $.ajax({
        url: url,
        data: data,
        success: onDone,
        dataType: 'json',
        progress: function(jqXHR, evt)
        {
            var pct = 0;
            if (evt.lengthComputable)
            {
                pct = 100 * evt.position / evt.total;
            }
            else if (self.content_length != null)
            {
                pct = 100 * evt.position / self.content_length;
            }

            onProgress(pct);
        }
    });
}

And then to use it:

然后使用它：

loader(function(response)
{
    console.log("Content loaded! do stuff now.");
},
function(pct)
{
    console.log("The content is " + pct + "% loaded.");
},
'<url here>', {});

---

---

On the server side, set the X-Content-Lengthheader on both the GETand the HEADrequests (which should represent the uncompressedcontent length), and abort sending the content on the HEADrequest.

在服务器端，设置请求X-Content-Length头GET和HEAD请求头（它应该代表未压缩的内容长度），并中止发送HEAD请求的内容。

In PHP, setting the header looks like:

在 PHP 中，设置标题如下所示：

header("X-Content-Length: ".strlen($payload));

And then abort sending the content if it's a HEADrequest:

如果是HEAD请求，则中止发送内容：

if ($_SERVER['REQUEST_METHOD'] == "HEAD")
{
    exit;
}

---

---

Here's what it looks like in action:

这是它的实际操作：

The reason the HEADtakes so long in the below screenshot is because the server still has to parse the file to know how long it is, but that's something I can definitely improve on, and it's definitely an improvement from where it was.

HEAD在下面的屏幕截图中花费这么长时间的原因是因为服务器仍然需要解析文件以知道它有多长，但这是我绝对可以改进的地方，而且它绝对是从原来的位置进行的改进。

回答by Nat

A slightly more elegant variation on your solution would be to set a header like 'x-decompressed-content-length' or whatever in your HTTP response with the full decompressed value of the content in bytes and read it off the xhr object in your onProgress handler.

您的解决方案的一个稍微优雅的变体是在您的 HTTP 响应中设置一个像“x-decompressed-content-length”这样的标头，并使用内容的完整解压缩值（以字节为单位），然后从 onProgress 中的 xhr 对象中读取它处理程序。

Your code might look something like:

您的代码可能类似于：

request.onProgress = function (e) {
  var contentLength;
  if (e.lengthComputable) {
    contentLength = e.total;
  } else {
    contentLength = parseInt(e.target.getResponseHeader('x-decompressed-content-length'), 10);
  }
  progressIndicator.update(e.loaded / contentLength);
};

回答by Ivan Castellanos

Don't get stuck just because there isn't a native solution; a hack of one line can solve your problem without messing with Apache configuration (that in some hostings is prohibited or very restricted):

不要因为没有本地解决方案而陷入困境；一行 hack 可以解决您的问题，而不会弄乱 Apache 配置（在某些主机中被禁止或非常受限）：

PHP to the rescue:

PHP 来拯救：

var size = <?php echo filesize('file.json') ?>;

That's it, you probably already know the rest, but just as a reference here it is:

就是这样，您可能已经知道其余部分，但作为参考，它是：

<script>
var progressBar = document.getElementById("p"),
    client = new XMLHttpRequest(),
    size = <?php echo filesize('file.json') ?>;

progressBar.max = size;

client.open("GET", "file.json")

function loadHandler () {
  var loaded = client.responseText.length;
  progressBar.value = loaded;
}

client.onprogress = loadHandler;

client.onloadend = function(pe) {
  loadHandler();
  console.log("Success, loaded: " + client.responseText.length + " of " + size)
}
client.send()
</script>

Live example:

现场示例：

Another SO user thinks I am lying about the validity of this solution so here it is live: http://nyudvik.com/zip/, it is gzip-ed and the real file weights 8 MB

另一个 SO 用户认为我在这个解决方案的有效性上撒谎，所以它是实时的：http: //nyudvik.com/zip/，它是 gzip 格式的，实际文件重 8 MB

---

---

Related links:

相关链接：

• SO: Content-Length not sent when gzip compression enabled in Apache?
• Apache Module mod_deflate doc
• PHP filsize function doc

• SO：在 Apache 中启用 gzip 压缩时未发送内容长度？
• Apache 模块 mod_deflate 文档
• PHP filsize 函数文档

回答by AdamJonR

Try changing your server encoding to gzip.

尝试将您的服务器编码更改为 gzip。

Your request header shows three potential encodings (gzip,deflate,sdch), so the server can pick any one of those three. By the response header, we can see that your server is choosing to respond with deflate.

您的请求标头显示了三种潜在的编码（gzip、deflate、sdch），因此服务器可以选择这三种中的任何一种。通过响应头，我们可以看到您的服务器选择使用 deflate 进行响应。

Gzip is an encoding format that includes a deflate payload in addition to additional headers and footer (which includes the original uncompressed length) and a different checksum algorithm:

Gzip 是一种编码格式，除了额外的页眉和页脚（包括原始未压缩长度）和不同的校验和算法之外，还包括 deflate 有效负载：

Gzip at Wikipedia

维基百科上的 Gzip

Deflate has some issues. Due to legacy issues dealing with improper decoding algorithms, client implementations of deflate have to run through silly checks just to figure out which implementation they're dealing with, and unfortunately, they often still get it wrong:

Deflate 有一些问题。由于处理不正确解码算法的遗留问题，deflate 的客户端实现必须通过愚蠢的检查才能找出他们正在处理的实现，不幸的是，他们仍然经常出错：

Why use deflate instead of gzip for text files served by Apache?

为什么对 Apache 提供的文本文件使用 deflate 而不是 gzip？

In the case of your question, the browser probably sees a deflate file coming down the pipe and just throws up its arms and says, "When I don't even know exactly how I'll end up decoding this thing, how can you expect me to worry about getting the progress right, human?"

就你的问题而言，浏览器可能会看到一个 deflate 文件从管道中传来，然后就举起手臂说：“当我什至不知道我最终将如何解码这个东西时，你怎么能指望我要担心进展是否正确，人类？”

If you switch your server configuration so the response is gzipped (i.e., gzip shows up as the content-encoding), I'm hopeful your script works as you'd hoped/expected it would.

如果您切换服务器配置以便响应被 gzip 压缩（即 gzip 显示为内容编码），我希望您的脚本能够像您希望/预期的那样工作。

回答by Andrin von Rechenberg

We have created a library that estimates the progress and always sets lengthComputableto true.

我们创建了一个估计进度并始终设置lengthComputable为 true 的库。

Chrome 64 still has this issue (see Bug)

Chrome 64 仍然有这个问题（见Bug）

It is a javascript shim that you can include in your page which fixes this issue and you can use the standard new XMLHTTPRequest()normally.

它是一个 javascript shim，您可以将其包含在解决此问题的页面中，并且您可以new XMLHTTPRequest()正常使用标准。

The javascript library can be found here:

可以在此处找到 javascript 库：

https://github.com/AirConsole/xmlhttprequest-length-computable

https://github.com/AirConsole/xmlhttprequest-length-computable

回答by David Mulder

The only solution I can think of is manually compressing the data (rather than leaving it to the server and browser), as that allows you to use the normal progress bar and should still give you considerable gains over the uncompressed version. If for example the system only is required to work in latest generation web browsers you can for example zip it on the server side (whatever language you use, I am sure there is a zip function or library) and on the client side you can use zip.js. If more browser support is required you can check this SO answerfor a number of compression and decompression functions (just choose one which is supported in the server side language you're using). Overall this should be reasonably simple to implement, although it will perform worse (though still good probably) than native compression/decompression. (Btw, after giving it a bit more thought it could in theory perform even better than the native version in case you would choose a compression algorithm which fits the type of data you're using and the data is sufficiently big)

我能想到的唯一解决方案是手动压缩数据（而不是将其留给服务器和浏览器），因为这样可以让您使用正常的进度条，并且与未压缩的版本相比，它仍然会给您带来可观的收益。例如，如果系统只需要在最新一代的 Web 浏览器中工作，您可以在服务器端（无论您使用什么语言，我确定有一个 zip 函数或库）并在客户端将其压缩，您可以使用zip.js。如果需要更多浏览器支持，您可以查看此 SO 答案对于许多压缩和解压缩功能（只需选择您正在使用的服务器端语言支持的一种）。总的来说，这应该相当容易实现，尽管它的性能会比原生压缩/解压差（尽管可能仍然很好）。（顺便说一句，如果您选择适合您正在使用的数据类型并且数据足够大的压缩算法，那么在对其进行更多思考之后，理论上它可以比本地版本表现得更好）

Another option would be to use a websocket and load the data in parts where you parse/handle every part at the same time it's loaded (you don't need websockets for that, but doing 10's of http requests after eachother can be quite a hassle). Whether this is possible depends on the specific scenario, but to me it sounds like report data is the kind of data that can be loaded in parts and isn't required to be first fully downloaded.

另一种选择是使用 websocket 并在加载数据的同时解析/处理每个部分的部分加载数据（您不需要 websockets，但是在彼此之后执行 10 个 http 请求可能会很麻烦）。这是否可能取决于具体情况，但对我来说，报告数据是一种可以分批加载的数据，不需要首先完全下载。

回答by Bertrand

I do not clearly understand the issue, it should not happen since the decompression should done by the browser.

我不清楚这个问题，它不应该发生，因为解压应该由浏览器完成。

You may try to move away from jQuery or hack jQuery because the $.ajax does not seems to work well with binary data:

您可能会尝试远离 jQuery 或 hack jQuery，因为 $.ajax 似乎不适用于二进制数据：

Ref: http://blog.vjeux.com/2011/javascript/jquery-binary-ajax.html

参考：http: //blog.vjeux.com/2011/javascript/jquery-binary-ajax.html

You could try to do your own implementation of the ajax request See: https://developer.mozilla.org/en-US/docs/DOM/XMLHttpRequest/Using_XMLHttpRequest#Handling_binary_data

您可以尝试自己实现 ajax 请求请参阅：https: //developer.mozilla.org/en-US/docs/DOM/XMLHttpRequest/Using_XMLHttpRequest#Handling_binary_data

You could try to uncompress the json the content by javascript (see resources in comments).

您可以尝试通过 javascript 解压缩 json 内容（请参阅评论中的资源）。

* UPDATE 2 *

* 更新 2 *

the $.ajax function does not support the progress event handler or it is not part of the jQuery documentation (see comment below).

$.ajax 函数不支持进度事件处理程序，或者它不是 jQuery 文档的一部分（请参阅下面的评论）。

here is a way to get this handler work but I never tried it myself: http://www.dave-bond.com/blog/2010/01/JQuery-ajax-progress-HMTL5/

这是让这个处理程序工作的一种方法，但我自己从未尝试过：http: //www.dave-bond.com/blog/2010/01/JQuery-ajax-progress-HMTL5/

* UPDATE 3 *

* 更新 3 *

The solution use tierce third party library to extend (?) jQuery ajax functionnality, so my suggestion do not apply

该解决方案使用 tierce 第三方库来扩展 (?) jQuery ajax 功能，所以我的建议不适用