{"action":"created","issue":{"url":"https://api.github.com/repos/PyTables/PyTables/issues/467","labels_url":"https://api.github.com/repos/PyTables/PyTables/issues/467/labels{/name}","comments_url":"https://api.github.com/repos/PyTables/PyTables/issues/467/comments","events_url":"https://api.github.com/repos/PyTables/PyTables/issues/467/events","html_url":"https://github.com/PyTables/PyTables/pull/467","id":87663336,"number":467,"title":"Python3 Unicode string support in Atom.from_dtype","user":{"login":"tp199911","id":1941694,"avatar_url":"https://avatars.githubusercontent.com/u/1941694?v=3","gravatar_id":"","url":"https://api.github.com/users/tp199911","html_url":"https://github.com/tp199911","followers_url":"https://api.github.com/users/tp199911/followers","following_url":"https://api.github.com/users/tp199911/following{/other_user}","gists_url":"https://api.github.com/users/tp199911/gists{/gist_id}","starred_url":"https://api.github.com/users/tp199911/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tp199911/subscriptions","organizations_url":"https://api.github.com/users/tp199911/orgs","repos_url":"https://api.github.com/users/tp199911/repos","events_url":"https://api.github.com/users/tp199911/events{/privacy}","received_events_url":"https://api.github.com/users/tp199911/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/PyTables/PyTables/labels/strings","name":"strings","color":"5319e7"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2015-06-12T08:58:33Z","updated_at":"2015-06-22T10:55:29Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/PyTables/PyTables/pulls/467","html_url":"https://github.com/PyTables/PyTables/pull/467","diff_url":"https://github.com/PyTables/PyTables/pull/467.diff","patch_url":"https://github.com/PyTables/PyTables/pull/467.patch"},"body":"In Python3 strings show up as dtype e.g. `'<U50'`. The current code only looks for a Python2-style `'S'`-kind, resulting in\r\n```python\r\n    ValueError: unknown type: 'str1600'\r\n```\r\nupon calling e.g. `create_array` with a string array as `obj=` argument, since this case currently gets handed over to `from_type`."},"comment":{"url":"https://api.github.com/repos/PyTables/PyTables/issues/comments/114065027","html_url":"https://github.com/PyTables/PyTables/pull/467#issuecomment-114065027","issue_url":"https://api.github.com/repos/PyTables/PyTables/issues/467","id":114065027,"user":{"login":"tp199911","id":1941694,"avatar_url":"https://avatars.githubusercontent.com/u/1941694?v=3","gravatar_id":"","url":"https://api.github.com/users/tp199911","html_url":"https://github.com/tp199911","followers_url":"https://api.github.com/users/tp199911/followers","following_url":"https://api.github.com/users/tp199911/following{/other_user}","gists_url":"https://api.github.com/users/tp199911/gists{/gist_id}","starred_url":"https://api.github.com/users/tp199911/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tp199911/subscriptions","organizations_url":"https://api.github.com/users/tp199911/orgs","repos_url":"https://api.github.com/users/tp199911/repos","events_url":"https://api.github.com/users/tp199911/events{/privacy}","received_events_url":"https://api.github.com/users/tp199911/received_events","type":"User","site_admin":false},"created_at":"2015-06-22T10:55:29Z","updated_at":"2015-06-22T10:55:29Z","body":"#### I should note that this fix has still a few issues though:\r\n* it **does not** implement support for Unicode, it only allows use of Python3 strings, conditional on the fact that they can be casted as old-school (read: ascii) strings. If not this will generate the error\r\n`UnicodeEncodeError: 'ascii' codec can't encode characters in position xxx: ordinal not in range(128)`\r\n* array creation via create_array will still fail, due to the check in line 204 of `array.py`\r\n* this might create abundant (disk) memory use, as a `U` dtype has an itemsize 4x that of the equivalent `S`. Due to the first point this extra memory usage is useless.\r\n\r\n#### I propose the following:\r\nA structured approach when trying to create unicode atoms or arrays:\r\n* In the case of atoms, allow it as in the current patch, but issue a warning, that non-ascii strings will fail to save. Also, correct the itemsize (*does anyone know where this magic multiplier of 4 comes from, and if I can find it in the numpy code?* I'd rather not just put `4` in the code..)\r\n* In the case of a passed `obj` argument in `create_?array`, catch the `UnicodeEncodeError` and throw a custom exception?\r\n\r\nPlease give your input on this."}}