{"action":"created","issue":{"url":"https://api.github.com/repos/Valloric/ycmd/issues/53","labels_url":"https://api.github.com/repos/Valloric/ycmd/issues/53/labels{/name}","comments_url":"https://api.github.com/repos/Valloric/ycmd/issues/53/comments","events_url":"https://api.github.com/repos/Valloric/ycmd/issues/53/events","html_url":"https://github.com/Valloric/ycmd/issues/53","id":48964338,"number":53,"title":"UnicodeEncodeError in filename completion","user":{"login":"blueyed","id":9766,"avatar_url":"https://avatars.githubusercontent.com/u/9766?v=3","gravatar_id":"","url":"https://api.github.com/users/blueyed","html_url":"https://github.com/blueyed","followers_url":"https://api.github.com/users/blueyed/followers","following_url":"https://api.github.com/users/blueyed/following{/other_user}","gists_url":"https://api.github.com/users/blueyed/gists{/gist_id}","starred_url":"https://api.github.com/users/blueyed/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/blueyed/subscriptions","organizations_url":"https://api.github.com/users/blueyed/orgs","repos_url":"https://api.github.com/users/blueyed/repos","events_url":"https://api.github.com/users/blueyed/events{/privacy}","received_events_url":"https://api.github.com/users/blueyed/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":13,"created_at":"2014-11-16T09:28:00Z","updated_at":"2015-07-03T15:10:53Z","closed_at":null,"body":"When using filename completion for a directory including utf8 filename, like `bürokratie`, the following error is shown:\r\n\r\n    UnicodeEncodeError: 'ascii' codec can't encode character u'\\xfc' in position 0: ordinal not in range(128)\r\n\r\nThe completion however pops up and contains the entries, but when typing the first char of some entry, it gets closed.\r\n\r\nThe error does not seem to happen with a single-char entry like `ü`, so it's probably related to some code that handles matching entries.\r\n\r\n    Traceback (most recent call last):\r\n      File \"…/vim/neobundles/YouCompleteMe/third_party/ycmd/third_party/bottle/bottle.py\", line 861, in _handle\r\n        return route.call(**args)\r\n      File \"…/vim/neobundles/YouCompleteMe/third_party/ycmd/third_party/bottle/bottle.py\", line 1734, in wrapper\r\n        rv = callback(*a, **ka)\r\n      File \"…/vim/neobundles/YouCompleteMe/third_party/ycmd/ycmd/../ycmd/watchdog_plugin.py\", line 100, in wrapper\r\n        return callback( *args, **kwargs )\r\n      File \"…/vim/neobundles/YouCompleteMe/third_party/ycmd/ycmd/../ycmd/hmac_plugin.py\", line 54, in wrapper\r\n        body = callback( *args, **kwargs )\r\n      File \"…/vim/neobundles/YouCompleteMe/third_party/ycmd/ycmd/../ycmd/handlers.py\", line 100, in GetCompletions\r\n        completer.ComputeCandidates( request_data ),\r\n      File \"…/vim/neobundles/YouCompleteMe/third_party/ycmd/ycmd/../ycmd/completers/general/general_completer_store.py\", line 83, in ComputeCandidates\r\n        candidates += completer.ComputeCandidates( request_data )\r\n      File \"…/vim/neobundles/YouCompleteMe/third_party/ycmd/ycmd/../ycmd/completers/completer.py\", line 161, in ComputeCandidates\r\n        request_data[ 'query' ] )\r\n      File \"…/vim/neobundles/YouCompleteMe/third_party/ycmd/ycmd/../ycmd/completers/completer.py\", line 216, in FilterAndSortCandidates\r\n        ToUtf8IfNeeded( query ) )\r\n    UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-2: ordinal not in range(128)\r\n\r\nCode reference: https://github.com/blueyed/ycmd/blob/master/ycmd/completers/completer.py#L214-216\r\n\r\nThe error seems a bit weird, because it appears to happen with the statement itself, and not e.g. inside the ToUtf8IfNeeded function?!\r\n"},"comment":{"url":"https://api.github.com/repos/Valloric/ycmd/issues/comments/118371113","html_url":"https://github.com/Valloric/ycmd/issues/53#issuecomment-118371113","issue_url":"https://api.github.com/repos/Valloric/ycmd/issues/53","id":118371113,"user":{"login":"rexdf","id":4038241,"avatar_url":"https://avatars.githubusercontent.com/u/4038241?v=3","gravatar_id":"","url":"https://api.github.com/users/rexdf","html_url":"https://github.com/rexdf","followers_url":"https://api.github.com/users/rexdf/followers","following_url":"https://api.github.com/users/rexdf/following{/other_user}","gists_url":"https://api.github.com/users/rexdf/gists{/gist_id}","starred_url":"https://api.github.com/users/rexdf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rexdf/subscriptions","organizations_url":"https://api.github.com/users/rexdf/orgs","repos_url":"https://api.github.com/users/rexdf/repos","events_url":"https://api.github.com/users/rexdf/events{/privacy}","received_events_url":"https://api.github.com/users/rexdf/received_events","type":"User","site_admin":false},"created_at":"2015-07-03T15:10:53Z","updated_at":"2015-07-03T15:10:53Z","body":"error happens when `  return extract< std::string >( str( string_or_unicode ).encode( \"utf8\" ) );` receive a non-ascii string_or_unicode. \r\n\r\nErrors thrown by `youcompleteme\\third_party\\ycmd\\cpp\\boostparts\\libs\\python\\src\\errors.cpp @ 62`\r\n\r\nThere is a file `ycmd测试.txt` in same directory with `.ycm_extra_conf.py`. It just finds `ycmd` but not `ycmd测试.txt`.\r\n\r\nthen throw error next loop in `CandidatesFromObjectList` / `for ( int i = 0; i < num_candidates; ++i ) {`\r\nand this error break the for loop, it never return back. so we cannot get file list for python.\r\n\r\nthe loop after get half str \"ycmd\", the following error is thrown by \r\n```\r\n  File \"D:\\vim-home\\.vim\\bundle\\YouCompleteMe\\third_party\\ycmd\\ycmd\\..\\ycmd\\completers\\cpp\\clang_completer.py\", line 260, in OnFileReadyToParse\r\n    with self._files_being_compiled.GetExclusive( filename ):\r\n  File \"D:\\vim-home\\.vim\\bundle\\YouCompleteMe\\third_party\\ycmd\\ycmd\\..\\ycmd\\completers\\cpp\\ephemeral_values_set.py\", line 56, in __enter_\r\nraise RuntimeError( ALREADY_PARSING_MESSAGE )\r\nRuntimeError: File already being parsed.\r\n```\r\n\r\nThe following is about ycmd_core (error thrown by boost-python but not python, and it breaks c++ loop):\r\n\r\n```log\r\n0:018> p\r\nint 0n2782\r\nycm_core!YouCompleteMe::`anonymous namespace'::CandidatesFromObjectList+0x29b:\r\n00007ffa`5d03272b 488d9424f8000000 lea     rdx,[rsp+0F8h]\r\n0:018> da 0x562cbe0\r\n00000000`0562cbe0  \"ycmd\"\r\n0:018> p\r\n(224c.1010): C++ EH exception - code e06d7363 (first chance)\r\nFirst chance exceptions are reported before any exception handling.\r\nThis exception may be expected and handled.\r\nKERNELBASE!RaiseException+0x6c:\r\n00007ffa`7e228b9c 488b8c24c0000000 mov     rcx,qword ptr [rsp+0C0h] ss:00000000`0562c7d0=0000e28ca51d3767\r\n0:018> kn\r\n # Child-SP          RetAddr           Call Site\r\n00 00000000`0562c710 00007ffa`5d51c366 KERNELBASE!RaiseException+0x6c\r\n01 00000000`0562c7f0 00007ffa`5d135070 MSVCR120D!_CxxThrowException+0x116 [f:\\dd\\vctools\\crt\\crtw32\\eh\\throw.cpp @ 154]\r\n02 00000000`0562c890 00007ffa`5cfbd98b ycm_core!boost::python::throw_error_already_set+0x30 [d:\\vim-home\\.vim\\bundle\\youcompleteme\\third_party\\ycmd\\cpp\\boostparts\\libs\\python\\src\\errors.cpp @ 62]\r\n03 00000000`0562c8d0 00007ffa`5cfc0ddd ycm_core!boost::python::expect_non_null<_object>+0x2b [d:\\vim-home\\.vim\\bundle\\youcompleteme\\third_party\\ycmd\\cpp\\boostparts\\boost\\python\\errors.hpp @ 46]\r\n04 00000000`0562c900 00007ffa`5d1c74ca ycm_core!boost::python::api::object::object+0x2d [d:\\vim-home\\.vim\\bundle\\youcompleteme\\third_party\\ycmd\\cpp\\boostparts\\boost\\python\\object_core.hpp @ 535]\r\n05 00000000`0562c930 00007ffa`5d032df9 ycm_core!boost::python::detail::str_base::str_base+0x3a [d:\\vim-home\\.vim\\bundle\\youcompleteme\\third_party\\ycmd\\cpp\\boostparts\\libs\\python\\src\\str.cpp @ 83]\r\n06 00000000`0562c960 00007ffa`5d03236d ycm_core!boost::python::str::str<boost::python::api::object>+0x59 [d:\\vim-home\\.vim\\bundle\\youcompleteme\\third_party\\ycmd\\cpp\\boostparts\\boost\\python\\str.hpp @ 163]\r\n07 00000000`0562c9b0 00007ffa`5d03270f ycm_core!YouCompleteMe::`anonymous namespace'::GetUtf8String+0xcd [d:\\vim-home\\.vim\\bundle\\youcompleteme\\third_party\\ycmd\\cpp\\ycm\\pythonsupport.cpp @ 47]\r\n08 00000000`0562cae0 00007ffa`5d031b03 ycm_core!YouCompleteMe::`anonymous namespace'::CandidatesFromObjectList+0x27f [d:\\vim-home\\.vim\\bundle\\youcompleteme\\third_party\\ycmd\\cpp\\ycm\\pythonsupport.cpp @ 64]\r\n09 00000000`0562ccd0 00007ffa`5d092a03 ycm_core!YouCompleteMe::FilterAndSortCandidates+0xe3 [d:\\vim-home\\.vim\\bundle\\youcompleteme\\third_party\\ycmd\\cpp\\ycm\\pythonsupport.cpp @ 87]\r\n```"}}