{"action":"created","issue":{"url":"https://api.github.com/repos/mopemope/meinheld/issues/62","labels_url":"https://api.github.com/repos/mopemope/meinheld/issues/62/labels{/name}","comments_url":"https://api.github.com/repos/mopemope/meinheld/issues/62/comments","events_url":"https://api.github.com/repos/mopemope/meinheld/issues/62/events","html_url":"https://github.com/mopemope/meinheld/issues/62","id":106517514,"number":62,"title":"Encoding issue while uploading file in django application","user":{"login":"HiddenFox","id":2194158,"avatar_url":"https://avatars.githubusercontent.com/u/2194158?v=3","gravatar_id":"","url":"https://api.github.com/users/HiddenFox","html_url":"https://github.com/HiddenFox","followers_url":"https://api.github.com/users/HiddenFox/followers","following_url":"https://api.github.com/users/HiddenFox/following{/other_user}","gists_url":"https://api.github.com/users/HiddenFox/gists{/gist_id}","starred_url":"https://api.github.com/users/HiddenFox/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HiddenFox/subscriptions","organizations_url":"https://api.github.com/users/HiddenFox/orgs","repos_url":"https://api.github.com/users/HiddenFox/repos","events_url":"https://api.github.com/users/HiddenFox/events{/privacy}","received_events_url":"https://api.github.com/users/HiddenFox/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-09-15T09:23:14Z","updated_at":"2015-09-16T02:07:44Z","closed_at":null,"body":"I have tested all the scenarios when my django application run without meinheld, and there is no exception.\r\nBut exceptions occurs if application is run by meinheld.\r\n\r\n## Senario 1\r\nThe uploaded file is just a text file. It is wired that I uploaded a file of size 200kb successfully. Then I copy the content of this file and paste to double its size. The exception occurred when the second file is uploaded. \r\n\r\nI tried to debug and found that function 'meinheld.input.read' returns a str which cannot concat to bytes.\r\n\r\nThe call stack is:\r\n```\r\nTraceback (most recent call last):\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/core/handlers/base.py\", line 132, in get_response\r\n    response = wrapped_callback(request, *callback_args, **callback_kwargs)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/contrib/admin/options.py\", line 616, in wrapper\r\n    return self.admin_site.admin_view(view)(*args, **kwargs)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/views/decorators/cache.py\", line 57, in _wrapped_view_func\r\n    response = view_func(request, *args, **kwargs)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/contrib/admin/sites.py\", line 233, in inner\r\n    return view(request, *args, **kwargs)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/views/decorators/csrf.py\", line 58, in wrapped_view\r\n    return view_func(*args, **kwargs)\r\n  File \"/home/dev/d5_django_server/apps/product/admin.py\", line 129, in change_view\r\n    return self.changeform_view(request, object_id, form_url, extra_context)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/utils/decorators.py\", line 34, in _wrapper\r\n    return bound_func(*args, **kwargs)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/utils/decorators.py\", line 106, in _wrapped_view\r\n    result = middleware.process_view(request, view_func, args, kwargs)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/middleware/csrf.py\", line 174, in process_view\r\n    request_csrf_token = request.POST.get('csrfmiddlewaretoken', '')\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/core/handlers/wsgi.py\", line 137, in _get_post\r\n    self._load_post_and_files()\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/http/request.py\", line 260, in _load_post_and_files\r\n    self._post, self._files = self.parse_file_upload(self.META, data)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/http/request.py\", line 225, in parse_file_upload\r\n    return parser.parse()\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/http/multipartparser.py\", line 149, in parse\r\n    for item_type, meta_data, field_stream in Parser(stream, self._boundary):\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/http/multipartparser.py\", line 626, in __iter__\r\n    for sub_stream in boundarystream:\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/http/multipartparser.py\", line 439, in __next__\r\n    return LazyStream(BoundaryIter(self._stream, self._boundary))\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/http/multipartparser.py\", line 466, in __init__\r\n    unused_char = self._stream.read(1)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/http/multipartparser.py\", line 337, in read\r\n    out = b''.join(parts())\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/http/multipartparser.py\", line 330, in parts\r\n    chunk = next(self)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/http/multipartparser.py\", line 352, in __next__\r\n    output = next(self._producer)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/http/multipartparser.py\", line 414, in __next__\r\n    data = self.flo.read(self.chunk_size)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/http/request.py\", line 292, in read\r\n    return self._stream.read(*args, **kwargs)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/core/handlers/wsgi.py\", line 57, in read\r\n    result = self.buffer + self._read_limited(size - len(self.buffer))\r\nTypeError: can't concat bytes to str\r\n```\r\n\r\n## Senario 2\r\nWhen a jpg is uploaded, the exception is different. \r\n\r\nThe call stack is:\r\n```\r\nTraceback (most recent call last):\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/core/handlers/base.py\", line 132, in get_response\r\n    response = wrapped_callback(request, *callback_args, **callback_kwargs)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/contrib/admin/options.py\", line 616, in wrapper\r\n    return self.admin_site.admin_view(view)(*args, **kwargs)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/views/decorators/cache.py\", line 57, in _wrapped_view_func\r\n    response = view_func(request, *args, **kwargs)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/contrib/admin/sites.py\", line 233, in inner\r\n    return view(request, *args, **kwargs)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/views/decorators/csrf.py\", line 58, in wrapped_view\r\n    return view_func(*args, **kwargs)\r\n  File \"/home/dev/d5_django_server/apps/product/admin.py\", line 129, in change_view\r\n    return self.changeform_view(request, object_id, form_url, extra_context)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/utils/decorators.py\", line 34, in _wrapper\r\n    return bound_func(*args, **kwargs)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/utils/decorators.py\", line 106, in _wrapped_view\r\n    result = middleware.process_view(request, view_func, args, kwargs)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/middleware/csrf.py\", line 174, in process_view\r\n    request_csrf_token = request.POST.get('csrfmiddlewaretoken', '')\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/core/handlers/wsgi.py\", line 137, in _get_post\r\n    self._load_post_and_files()\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/http/request.py\", line 260, in _load_post_and_files\r\n    self._post, self._files = self.parse_file_upload(self.META, data)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/http/request.py\", line 225, in parse_file_upload\r\n    return parser.parse()\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/http/multipartparser.py\", line 149, in parse\r\n    for item_type, meta_data, field_stream in Parser(stream, self._boundary):\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/http/multipartparser.py\", line 626, in __iter__\r\n    for sub_stream in boundarystream:\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/http/multipartparser.py\", line 439, in __next__\r\n    return LazyStream(BoundaryIter(self._stream, self._boundary))\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/http/multipartparser.py\", line 466, in __init__\r\n    unused_char = self._stream.read(1)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/http/multipartparser.py\", line 337, in read\r\n    out = b''.join(parts())\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/http/multipartparser.py\", line 330, in parts\r\n    chunk = next(self)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/http/multipartparser.py\", line 352, in __next__\r\n    output = next(self._producer)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/http/multipartparser.py\", line 414, in __next__\r\n    data = self.flo.read(self.chunk_size)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/http/request.py\", line 292, in read\r\n    return self._stream.read(*args, **kwargs)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/core/handlers/wsgi.py\", line 57, in read\r\n    result = self.buffer + self._read_limited(size - len(self.buffer))\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/site-packages/django/core/handlers/wsgi.py\", line 45, in _read_limited\r\n    result = self.stream.read(size)\r\n  File \"/home/dev/virtualenv/python3/lib/python3.4/codecs.py\", line 313, in decode\r\n    (result, consumed) = self._buffer_decode(data, self.errors, final)\r\nUnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 1412: invalid start byte\r\n```\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/mopemope/meinheld/issues/comments/140601668","html_url":"https://github.com/mopemope/meinheld/issues/62#issuecomment-140601668","issue_url":"https://api.github.com/repos/mopemope/meinheld/issues/62","id":140601668,"user":{"login":"HiddenFox","id":2194158,"avatar_url":"https://avatars.githubusercontent.com/u/2194158?v=3","gravatar_id":"","url":"https://api.github.com/users/HiddenFox","html_url":"https://github.com/HiddenFox","followers_url":"https://api.github.com/users/HiddenFox/followers","following_url":"https://api.github.com/users/HiddenFox/following{/other_user}","gists_url":"https://api.github.com/users/HiddenFox/gists{/gist_id}","starred_url":"https://api.github.com/users/HiddenFox/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HiddenFox/subscriptions","organizations_url":"https://api.github.com/users/HiddenFox/orgs","repos_url":"https://api.github.com/users/HiddenFox/repos","events_url":"https://api.github.com/users/HiddenFox/events{/privacy}","received_events_url":"https://api.github.com/users/HiddenFox/received_events","type":"User","site_admin":false},"created_at":"2015-09-16T02:07:44Z","updated_at":"2015-09-16T02:07:44Z","body":"@methane Thanks for your response. \r\nI tried this patch by: \r\n1. Clone the latest code from github. \r\n2. Modify server.c according to the commit. \r\n3. python setup.py install\r\n\r\nAnd I got another exception every time a form is committed. Even if the form has no file to be posted.\r\nI am not sure if it is caused by the change or my incorrect installation.\r\n\r\nHere is the exception:\r\n```\r\nTraceback (most recent call last):\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/django/core/handlers/base.py\", line 132, in get_response\r\n    response = wrapped_callback(request, *callback_args, **callback_kwargs)\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/django/contrib/admin/options.py\", line 616, in wrapper\r\n    return self.admin_site.admin_view(view)(*args, **kwargs)\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/django/views/decorators/cache.py\", line 57, in _wrapped_view_func\r\n    response = view_func(request, *args, **kwargs)\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/django/contrib/admin/sites.py\", line 233, in inner\r\n    return view(request, *args, **kwargs)\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/django/views/decorators/csrf.py\", line 58, in wrapped_view\r\n    return view_func(*args, **kwargs)\r\n  File \"/home/han/workspace/d5/d5_django_server/apps/product/admin.py\", line 129, in change_view\r\n    return self.changeform_view(request, object_id, form_url, extra_context)\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/django/utils/decorators.py\", line 34, in _wrapper\r\n    return bound_func(*args, **kwargs)\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/django/utils/decorators.py\", line 110, in _wrapped_view\r\n    response = view_func(request, *args, **kwargs)\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/django/utils/decorators.py\", line 30, in bound_func\r\n    return func.__get__(self, type(self))(*args2, **kwargs2)\r\n  File \"/usr/lib/python3.4/contextlib.py\", line 30, in inner\r\n    return func(*args, **kwds)\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/django/contrib/admin/options.py\", line 1466, in changeform_view\r\n    if all_valid(formsets) and form_validated:\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/django/forms/formsets.py\", line 439, in all_valid\r\n    if not formset.is_valid():\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/django/forms/formsets.py\", line 304, in is_valid\r\n    self.errors\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/django/forms/formsets.py\", line 278, in errors\r\n    self.full_clean()\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/django/forms/formsets.py\", line 327, in full_clean\r\n    self._errors.append(form.errors)\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/django/forms/forms.py\", line 176, in errors\r\n    self.full_clean()\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/django/forms/forms.py\", line 394, in full_clean\r\n    self._post_clean()\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/django/forms/models.py\", line 430, in _post_clean\r\n    self.instance.full_clean(exclude=exclude, validate_unique=False)\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/django/db/models/base.py\", line 1125, in full_clean\r\n    self.clean_fields(exclude=exclude)\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/django/db/models/base.py\", line 1167, in clean_fields\r\n    setattr(self, f.attname, f.clean(raw_value, self))\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/django/db/models/fields/__init__.py\", line 589, in clean\r\n    self.run_validators(value)\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/django/db/models/fields/__init__.py\", line 541, in run_validators\r\n    v(value)\r\n  File \"/home/han/workspace/d5/d5_django_server/d5/files/validator.py\", line 53, in __call__\r\n    actual_size = file_obj.file.size\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/django/db/models/fields/files.py\", line 51, in _get_file\r\n    self._file = self.storage.open(self.name, 'rb')\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/django/core/files/storage.py\", line 37, in open\r\n    return self._open(name, mode)\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/storages/backends/s3boto.py\", line 388, in _open\r\n    f = self.file_class(name, mode, self)\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/storages/backends/s3boto.py\", line 103, in __init__\r\n    self.key = storage.bucket.get_key(self._storage._encode_name(name))\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/boto/s3/bucket.py\", line 192, in get_key\r\n    key, resp = self._get_key_internal(key_name, headers, query_args_l)\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/boto/s3/bucket.py\", line 199, in _get_key_internal\r\n    query_args=query_args)\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/boto/s3/connection.py\", line 664, in make_request\r\n    retry_handler=retry_handler\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/boto/connection.py\", line 1071, in make_request\r\n    retry_handler=retry_handler)\r\n  File \"/home/han/workspace/python/env/python3/lib/python3.4/site-packages/boto/connection.py\", line 943, in _mexe\r\n    request.body, request.headers)\r\n  File \"/usr/lib/python3.4/http/client.py\", line 1088, in request\r\n    self._send_request(method, url, body, headers)\r\n  File \"/usr/lib/python3.4/http/client.py\", line 1126, in _send_request\r\n    self.endheaders(body)\r\n  File \"/usr/lib/python3.4/http/client.py\", line 1084, in endheaders\r\n    self._send_output(message_body)\r\n  File \"/usr/lib/python3.4/http/client.py\", line 922, in _send_output\r\n    self.send(msg)\r\n  File \"/usr/lib/python3.4/http/client.py\", line 857, in send\r\n    self.connect()\r\n  File \"/usr/lib/python3.4/http/client.py\", line 1231, in connect\r\n    server_hostname=server_hostname)\r\n  File \"/usr/lib/python3.4/ssl.py\", line 365, in wrap_socket\r\n    _context=self)\r\n  File \"/usr/lib/python3.4/ssl.py\", line 577, in __init__\r\n    server_hostname)\r\nTypeError: _wrap_socket() argument 1 must be _socket.socket, not SSLSocket\r\n```"}}