Python IOError：请求数据读取错误

Question

提问by Sidmitra

I seem to be getting an IOError: request data read error quite a lot when i'm doing an Ajax upload. For example out of every 5 file uploads it errors out on atleast 3.

我在执行 Ajax 上传时似乎经常遇到 IOError: request data read 错误。例如，每 5 个文件上传至少 3 个。

Other people seem to have had the same issue. Eg.

其他人似乎也有同样的问题。例如。

Some other observations:

其他一些观察：

It's definitely not my internet connection or a browser issue. Seems to be happening on all browsers chrome/FF/opera.
I'm running django 1.1.1 Apache/2.2.14 (Ubuntu) mod_ssl/2.2.14 OpenSSL/0.9.8k mod_wsgi/2.8 Python/2.6.5on Lucid.
It is also not the file size. I can sometimes upload 1+ MB files but fail on 180 Kb files.

这绝对不是我的互联网连接或浏览器问题。似乎发生在所有浏览器 chrome/FF/opera 上。
我在 Lucid 上运行django 1.1.1 Apache/2.2.14 (Ubuntu) mod_ssl/2.2.14 OpenSSL/0.9.8k mod_wsgi/2.8 Python/2.6.5。
它也不是文件大小。我有时可以上传 1+ MB 的文件，但无法上传 180 Kb 的文件。

Traceback

追溯

Traceback (most recent call last):

  File "/home/ubuntu/.virtualenvs/anonymous_app/lib/python2.6/site-packages/django/core/handlers/base.py", line 98, in get_response
    response = middleware_method(request, e)

  File "/home/ubuntu/.virtualenvs/anonymous_app/lib/python2.6/site-packages/django/core/handlers/base.py", line 92, in get_response
    response = callback(request, *callback_args, **callback_kwargs)

  File "/home/ubuntu/.virtualenvs/anonymous_app/lib/python2.6/site-packages/django/contrib/auth/decorators.py", line 78, in __call__
    return self.view_func(request, *args, **kwargs)

  File "/home/ubuntu/webapps/anonymous_app/app/do_work/views/__init__.py", line 391, in some_form_ajax_upload
    f = request.FILES.get('file_upload')

  File "/home/ubuntu/.virtualenvs/anonymous_app/lib/python2.6/site-packages/django/core/handlers/wsgi.py", line 187, in _get_files
    self._load_post_and_files()

  File "/home/ubuntu/.virtualenvs/anonymous_app/lib/python2.6/site-packages/django/core/handlers/wsgi.py", line 137, in _load_post_and_files
    self._post, self._files = self.parse_file_upload(self.META, self.environ['wsgi.input'])

  File "/home/ubuntu/.virtualenvs/anonymous_app/lib/python2.6/site-packages/django/http/__init__.py", line 124, in parse_file_upload
    return parser.parse()

  File "/home/ubuntu/.virtualenvs/anonymous_app/lib/python2.6/site-packages/django/http/multipartparser.py", line 133, in parse
    for item_type, meta_data, field_stream in Parser(stream, self._boundary):

  File "/home/ubuntu/.virtualenvs/anonymous_app/lib/python2.6/site-packages/django/http/multipartparser.py", line 606, in __iter__
    for sub_stream in boundarystream:

  File "/home/ubuntu/.virtualenvs/anonymous_app/lib/python2.6/site-packages/django/http/multipartparser.py", line 420, in next
    return LazyStream(BoundaryIter(self._stream, self._boundary))

  File "/home/ubuntu/.virtualenvs/anonymous_app/lib/python2.6/site-packages/django/http/multipartparser.py", line 446, in __init__
    unused_char = self._stream.read(1)

  File "/home/ubuntu/.virtualenvs/anonymous_app/lib/python2.6/site-packages/django/http/multipartparser.py", line 299, in read
    out = ''.join(parts())

  File "/home/ubuntu/.virtualenvs/anonymous_app/lib/python2.6/site-packages/django/http/multipartparser.py", line 292, in parts
    chunk = self.next()

  File "/home/ubuntu/.virtualenvs/anonymous_app/lib/python2.6/site-packages/django/http/multipartparser.py", line 314, in next
    output = self._producer.next()

  File "/home/ubuntu/.virtualenvs/anonymous_app/lib/python2.6/site-packages/django/http/multipartparser.py", line 375, in next
    data = self.flo.read(self.chunk_size)

  File "/home/ubuntu/.virtualenvs/anonymous_app/lib/python2.6/site-packages/django/http/multipartparser.py", line 405, in read
    return self._file.read(num_bytes)

IOError: request data read error


<WSGIRequest
GET:<QueryDict: {}>,
POST:<could not parse>,
COOKIES:{'__utma': '168279989.1688771210.1285773436.1285773436.1285773436.1',
 '__utmb': '168279989.20.10.1285773436',
 '__utmc': '168279989',
 '__utmz': '168279989.1285773436.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)',
 'beta': 'True',
 'sessionid': 'b1ecf92f2bba13e1885d07803e10aa03',
 'timezone_offset': '-330'},
META:{'CONTENT_LENGTH': '188575',
 'CONTENT_TYPE': 'multipart/form-data; boundary=---------------------------57602381214905740261171925981',
 'DOCUMENT_ROOT': '/htdocs',
 'GATEWAY_INTERFACE': 'CGI/1.1',
 'HTTPS': '1',
 'HTTP_ACCEPT': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
 'HTTP_ACCEPT_CHARSET': 'ISO-8859-1,utf-8;q=0.7,*;q=0.7',
 'HTTP_ACCEPT_ENCODING': 'gzip,deflate',
 'HTTP_ACCEPT_LANGUAGE': 'en-us,en;q=0.5',
 'HTTP_CONNECTION': 'keep-alive',
 'HTTP_COOKIE': 'beta=True; __utma=168279989.1688771210.1285773436.1285773436.1285773436.1; __utmb=168279989.20.10.1285773436; __utmc=168279989; __utmz=168279989.1285773436.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); sessionid=b1ecf92f2bba13e1885d07803e10aa03; timezone_offset=-330',
 'HTTP_HOST': 'xxxxxx.compute-1.amazonaws.com',
 'HTTP_KEEP_ALIVE': '115',
 'HTTP_REFERER': 'https://ec2-184-72-79-96.compute-1.amazonaws.com/do-my-somees/enter/some-documents/',
 'HTTP_USER_AGENT': 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.10) Gecko/20100915 Ubuntu/10.04 (lucid) Firefox/3.6.10',
 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin',
 'PATH_INFO': u'/do-my-somees/enter/some-documents/ajax-upload/Other-some-Document/',
 'PATH_TRANSLATED': '/home/ubuntu/webapps/anonymous_app/settings/apache/qa.wsgi.py/do-my-somees/enter/some-documents/ajax-upload/Other-some-Document/',
 'QUERY_STRING': '',
 'REMOTE_ADDR': '',
 'REMOTE_PORT': '15561',
 'REQUEST_METHOD': 'POST',
 'REQUEST_URI': '/do-my-somees/enter/some-documents/ajax-upload/Other-some-Document/',
 'SCRIPT_FILENAME': '/home/ubuntu/webapps/anonymous_app/settings/apache/qa.wsgi.py',
 'SCRIPT_NAME': u'',
 'SERVER_ADDR': '10.196.142.182',
 'SERVER_ADMIN': 'dev@anonymous_app.com',
 'SERVER_NAME': 'ec2-184-72-79-96.compute-1.amazonaws.com',
 'SERVER_PORT': '443',
 'SERVER_PROTOCOL': 'HTTP/1.1',
 'SERVER_SIGNATURE': '<address>Apache/2.2.14 (Ubuntu) Server at ec2-184-72-79-96.compute-1.amazonaws.com Port 443</address>\n',
 'SERVER_SOFTWARE': 'Apache/2.2.14 (Ubuntu)',
 'SSL_TLS_SNI': 'ec2-184-72-79-96.compute-1.amazonaws.com',
 'mod_wsgi.application_group': 'qa.anonymous_app.com|',
 'mod_wsgi.callable_object': 'application',
 'mod_wsgi.listener_host': '',
 'mod_wsgi.listener_port': '443',
 'mod_wsgi.process_group': '',
 'mod_wsgi.reload_mechanism': '0',
 'mod_wsgi.script_reloading': '1',
 'mod_wsgi.version': (2, 8),
 'wsgi.errors': <mod_wsgi.Log object at 0xb9456860>,
 'wsgi.file_wrapper': <built-in method file_wrapper of mod_wsgi.Adapter object at 0xb936a968>,
 'wsgi.input': <mod_wsgi.Input object at 0xb9720e30>,
 'wsgi.multiprocess': True,
 'wsgi.multithread': False,
 'wsgi.run_once': False,
 'wsgi.url_scheme': 'https',
 'wsgi.version': (1, 0)}>

Answer 1

回答by fetzig

as you might think, this is no django error.

正如您所想，这不是 Django 错误。

see https://groups.google.com/group/django-users/browse_thread/thread/946936f69c012d96

见https://groups.google.com/group/django-users/browse_thread/thread/946936f69c012d96

have the error myself (but IE ajax requests only, no file upload, just post data).

我自己有错误（但只有 IE ajax 请求，没有文件上传，只是发布数据）。

will add an complete answer if i ever find out how to fix this.

如果我知道如何解决这个问题，我会添加一个完整的答案。

Answer 2

回答by Sidmitra

This issue has been open for a long time and has something to do with lower level libraries. I was using boto to upload files to S3. A temporary stopgap I found was to add an explicit HTTP socket timeout of 10 seconds. I haven't seen the error after that. You can do that by creating a boto config on the server:

这个问题已经开放了很长时间，与较低级别的库有关。我正在使用 boto 将文件上传到 S3。我发现的一个临时权宜之计是添加一个 10 秒的显式 HTTP 套接字超时。在那之后我没有看到错误。您可以通过在服务器上创建 boto 配置来做到这一点：

#/etc/boto.cfg 
[Boto]
http_socket_timeout=10

Also make sure the file is readable by the app. See my original post on google group: https://groups.google.com/forum/#!topic/boto-users/iWtvuECAcn4

还要确保应用程序可以读取该文件。请参阅我在 google group 上的原始帖子：https: //groups.google.com/forum/#!topic/boto-users/ iWtvuECAcn4

Answer 3

回答by guettli

I get this exception, too. In the Apache error logfile I see this:

我也有这个例外。在 Apache 错误日志文件中，我看到：

[Wed Aug 17 08:30:45 2011] [error] [client 10.114.48.206] (70014)End of file found: mod_wsgi (pid=9722): Unable to get bucket brigade for request., referer: https://egs-work/modwork/beleg/188074/edit/
[Wed Aug 17 08:30:45 2011] [error] [client 10.114.48.206] mod_wsgi (pid=3572): Exception occurred processing WSGI script '/home/modwork_egs_p/modwork_egs/apache/django_wsgi.py'.
[Wed Aug 17 08:30:45 2011] [error] [client 10.114.48.206] IOError: failed to write data

Versions:

版本：

apache2-prefork-2.2.15-3.7.x86_64
apache2-mod_wsgi-3.3-1.8.x86_64 WSGIDaemonProcess with threads=1
mod_ssl/2.2.15
Linux egs-work 2.6.34.8-0.2-default #1 SMP 2011-04-06 18:11:26 +0200 x86_64 x86_64 x86_64 GNU/Linux
openSUSE 11.3 (x86_64)

First I was confused, because the last line "failed to writedata" does not fit to the django code "load post data". But I guess that django wants to write an error page to the client. But the client has canceled the tcp connection. And now http 500 page can't be written to the client.

首先我很困惑，因为最后一行“写入数据失败”不适合 Django 代码“加载发布数据”。但是我猜django想给客户端写一个错误页面。但是客户端已经取消了tcp连接。现在http 500页面无法写入客户端。

The client disconnected after sending the request, and before getting the response:

客户端在发送请求后和得到响应之前断开连接：

The user closed the browser or navigated to an other page.
The user pressed the reload button.

用户关闭浏览器或导航到其他页面。
用户按下了重新加载按钮。

I have seen this only with POST-Requests (not GET). If POST is used, the webserver does read at least twice: First to get the headers, the second to get the data. The second read fails.

我只在 POST-Requests（不是 GET）中看到过这个。如果使用 POST，网络服务器至少读取两次：第一次获取标题，第二次获取数据。第二次读取失败。

It is easy to reproduce:

很容易重现：

Insert some code which waits before the first access to request.POST happens (be sure, that no middleware accesses request.POST before time.sleep()):

插入一些在第一次访问 request.POST 之前等待的代码（确保没有中间件在 time.sleep() 之前访问 request.POST）：

def edit(request):
    import time
    time.sleep(3)
    #.....

Now do a big POST (e.g. file upload). I don't know the apache buffer size. But 5 MB should be enough. When the browser shows the hourglass, browse to an other page. The browser will cancel the request and the exception should be in the logfile.

现在做一个大的POST（例如文件上传）。我不知道 apache 缓冲区大小。但是 5 MB 应该足够了。当浏览器显示沙漏时，浏览到其他页面。浏览器将取消请求，异常应该在日志文件中。

This is my Middleware, since I don't want to get the above traceback in our logfiles:

这是我的中间件，因为我不想在我们的日志文件中获得上述回溯：

class HandleExceptionMiddleware:

    def process_exception(self, request, exception):
        if isinstance(exception, IOError) and 'request data read error' in unicode(exception):
            logging.info('%s %s: %s: Request was canceled by the client.' % (
                    request.build_absolute_uri(), request.user, exception))
            return HttpResponseServerError()

Answer 4

回答by Subhranath Chunder

Taking this from the thread: Getting rid of Django IOErrors

从线程中获取这个：摆脱 Django IOErrors

Extending the possible solution by @dlowe for Django 1.3, to suppress the IOError in concern, we can write the full working example as:

通过@dlowe 为 Django 1.3 扩展可能的解决方案，为了抑制相关的 IOError，我们可以将完整的工作示例编写为：

settings.py

设置.py

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'filters': {
        'supress_unreadable_post': {
            '()': 'common.logging.SuppressUnreadablePost',
        }
    },
    'handlers': {
        'mail_admins': {
            'level': 'ERROR',
            'class': 'django.utils.log.AdminEmailHandler',
            'filters': ['supress_unreadable_post'],
        }
    },
    'loggers': {
        'django.request': {
            'handlers': ['mail_admins'],
            'level': 'ERROR',
            'propagate': True,
        },
    }
}

common/logging.py

常见的/logging.py

import sys, traceback

class SuppressUnreadablePost(object):
    def filter(self, record):
        _, exception, tb = sys.exc_info()
        if isinstance(exception, IOError):
            for _, _, function, _ in traceback.extract_tb(tb):
                if function == '_get_raw_post_data':
                    return False
        return True

Answer 5

回答by user2275915

We were seeing this error on uploads to Django Rest Framework when the content-type header was incorrectly set to application/json. The post was actually multipart form data. The errors stopped when we removed the incorrect content-type header.

当内容类型标头错误设置为 application/json 时，我们在上传到 Django Rest Framework 时看到此错误。该帖子实际上是多部分表单数据。当我们删除不正确的内容类型标头时，错误停止了。

Answer 6

回答by Ricardo Silva

I got this error when validating my site on a Win8 machine with IE 10. When I tested file upload from IE the upload was stucked on 1% and after +/- 1 minute I got the error on server logs. I just discovered that it was caused by the TrendMicro complement. Once I disabled the complement the upload occurred without any problem.

我在使用 IE 10 的 Win8 机器上验证我的网站时遇到此错误。当我测试从 IE 上传文件时，上传停留在 1% 上，在 +/- 1 分钟后，我在服务器日志中收到错误消息。我才发现是TrendMicro的补全引起的。一旦我禁用了补充，上传就没有任何问题。

Answer 7

回答by Aamir Adnan

This happened to me recently. I was using django-ajax-uploaderand small files were uploading successfully but large files e.g. 100MB were breaking in between with IOError: request data read error.

这最近发生在我身上。我正在使用django-ajax-uploader并且小文件成功上传，但大文件（例如 100MB）在IOError: request data read error.

I checked my Apache configuration and found these setting RequestReadTimeout header=90 body=90which means Allow 90 seconds to receive the request including the headers and 90 seconds for receiving the request body.

我检查了我的 Apache 配置，发现这些设置RequestReadTimeout header=90 body=90意味着Allow 90 seconds to receive the request including the headers and 90 seconds for receiving the request body.

File is received in chunks at backend, which means if file size is big 90seconds are not enough for some uploads. So how to determine the best value (seconds) for the requests?

文件在后端以块的形式接收，这意味着如果文件大小很大90，对于某些上传来说，秒是不够的。那么如何确定请求的最佳值（秒）？

So I have used this setting:

所以我使用了这个设置：

RequestReadTimeout header=90,MinRate=500 body=90,MinRate=500

Defining the MinRatesolves the issue for me. The above setting states that:

定义MinRate解决了我的问题。上述设置指出：

Allow at least 90 seconds to receive the request body. If the client sends data, increase the timeout by 1 second for every 500 bytes received

等待至少 90 秒来接收请求正文。如果客户端发送数据，每收到 500 个字节，超时时间增加 1 秒

As the client is sending data continuously (ajax upload) it makes sense to automatically increase the timeout if data is received. More information/variations about the RequestReadTimeoutcan be found here.

由于客户端连续发送数据（ajax 上传），因此在收到数据时自动增加超时时间是有意义的。RequestReadTimeout可以在此处找到有关的更多信息/变体。

Python IOError：请求数据读取错误

提问by Sidmitra

回答by fetzig

回答by Sidmitra

回答by guettli

回答by Subhranath Chunder

settings.py

设置.py

common/logging.py

常见的/logging.py

回答by user2275915

回答by Ricardo Silva

回答by Aamir Adnan

相关推荐

最近更新

标签

Python IOError：请求数据读取错误

提问by Sidmitra

回答by fetzig

回答by Sidmitra

回答by guettli

回答by Subhranath Chunder

settings.py

设置.py

common/logging.py

常见的/logging.py

回答by user2275915

回答by Ricardo Silva

回答by Aamir Adnan

相关推荐

Python 时间段后停止代码

Python 将 URL 转换为屏幕截图（脚本）

Python 与 numpy.timedelta64 的时差（以秒为单位）

Python 如何检查一个目录是否是另一个目录的子目录

相关推荐

最近更新

标签