{"action":"created","issue":{"url":"https://api.github.com/repos/pyldap/pyldap/issues/1","labels_url":"https://api.github.com/repos/pyldap/pyldap/issues/1/labels{/name}","comments_url":"https://api.github.com/repos/pyldap/pyldap/issues/1/comments","events_url":"https://api.github.com/repos/pyldap/pyldap/issues/1/events","html_url":"https://github.com/pyldap/pyldap/issues/1","id":95501612,"number":1,"title":"Define backwards compatibility rules regarding encoding","user":{"login":"rbarrois","id":694541,"avatar_url":"https://avatars.githubusercontent.com/u/694541?v=3","gravatar_id":"","url":"https://api.github.com/users/rbarrois","html_url":"https://github.com/rbarrois","followers_url":"https://api.github.com/users/rbarrois/followers","following_url":"https://api.github.com/users/rbarrois/following{/other_user}","gists_url":"https://api.github.com/users/rbarrois/gists{/gist_id}","starred_url":"https://api.github.com/users/rbarrois/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rbarrois/subscriptions","organizations_url":"https://api.github.com/users/rbarrois/orgs","repos_url":"https://api.github.com/users/rbarrois/repos","events_url":"https://api.github.com/users/rbarrois/events{/privacy}","received_events_url":"https://api.github.com/users/rbarrois/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/pyldap/pyldap/labels/design","name":"design","color":"fef2c0"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":14,"created_at":"2015-07-16T18:54:43Z","updated_at":"2015-07-17T15:25:11Z","closed_at":null,"body":"The long-term goal of this repository is to be merged back into the upstream python-ldap package.\r\n\r\nHowever, the python-ldap API doesn't feel natural under Python3, mostly with its encoding handling (it uses bytes everywhere, even when unicode strings could be used).\r\n\r\nFrom the [RFCs](https://tools.ietf.org/html/rfc4511#section-4.1.2), some fields *MUST* be encoded as UTF-8, thus represented as unicode under Py2 / str under Py3.\r\n\r\nThis issue appears mostly on base DNs in filters and in return distinguished names:\r\n\r\n```python\r\n# In Py2:\r\n>>> base = u\"ou=people,dc=example,dc=org\"\r\n>>> results = connection.search_ext(base.encode('utf-8'), scope=ldap.SCOPE_SUBTREE)\r\n>>> results[0][0]  # results is a list of (dn, data)\r\nb'uid=rbarrois,ou=people,dc=example,dc=org'\r\n>>> results[0][1].keys()[0]\r\nb'distinguisedName'\r\n```\r\n\r\n* For filters, since data is fed from the calling program to python-ldap, it is simple to support both unicode and bytes and convert as needed\r\n* For output, we must decide on a type - either bytes or unicode.\r\n\r\nIf we want third party libraries integrating python-ldap to use the same code on both Py2 and Py3, we have only a few options:\r\n* Keep the Py2, bytes-only API for Py3\r\n* Change the Py2 API from python-ldap's version\r\n* Use a new object to represent \"modern\" connections\r\n\r\nThe goal is to be able to write the following code:\r\n\r\n```python\r\n# In Py3 (\"\" for str, b'' for bytes)\r\n>>> base = \"ou=people,dc=example,dc=org\"\r\n>>> results = connection.search_ext(base, scope=ldap.SCOPE_SUBTREE)\r\n>>> results[0]\r\n(\"uid=rbarrois,ou=people,dc=example,dc=org\", {\r\n    \"uid\": [b'rbarrois'],\r\n    \"picture\": [b'XXX.UZYGFUYGFUEY...'],\r\n})\r\n\r\n# In Py2 (u\"\" for unicode, '' for bytes)\r\n>>> base = u\"ou=people,dc=example,dc=org\"\r\n>>> results = connection.search_ext(base, scope=ldap.SCOPE_SUBTREE)\r\n>>> results[0]\r\n(u\"uid=rbarrois,ou=people,dc=example,dc=org\", {\r\n    u\"uid\": ['rbarrois'],\r\n    u\"picture\": ['XXX.UZYGFUYGFUEY...'],\r\n})\r\n```"},"comment":{"url":"https://api.github.com/repos/pyldap/pyldap/issues/comments/122311382","html_url":"https://github.com/pyldap/pyldap/issues/1#issuecomment-122311382","issue_url":"https://api.github.com/repos/pyldap/pyldap/issues/1","id":122311382,"user":{"login":"rbarrois","id":694541,"avatar_url":"https://avatars.githubusercontent.com/u/694541?v=3","gravatar_id":"","url":"https://api.github.com/users/rbarrois","html_url":"https://github.com/rbarrois","followers_url":"https://api.github.com/users/rbarrois/followers","following_url":"https://api.github.com/users/rbarrois/following{/other_user}","gists_url":"https://api.github.com/users/rbarrois/gists{/gist_id}","starred_url":"https://api.github.com/users/rbarrois/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rbarrois/subscriptions","organizations_url":"https://api.github.com/users/rbarrois/orgs","repos_url":"https://api.github.com/users/rbarrois/repos","events_url":"https://api.github.com/users/rbarrois/events{/privacy}","received_events_url":"https://api.github.com/users/rbarrois/received_events","type":"User","site_admin":false},"created_at":"2015-07-17T15:25:11Z","updated_at":"2015-07-17T15:25:11Z","body":"Not sure about **what** we agree on :)\r\n\r\nThe current implementation (the one I wrote some time ago) *already* does the following:\r\n- On Py2, keep the old, clumsy API\r\n- On Py3, use ``PyStr`` instead of ``PyBytes`` for DN / filter / attribute names\r\n\r\nThe problem with this approach is that libraries relying on pyldap must have different code paths depending on whether they run under Py2 or under Py3:\r\n- If they wish to provide a \"unicode/str\" interface, they need to play the ``.encode('utf-8')`` / ``.decode('utf-8')`` dans on Py2\r\n- If they prefer the \"bytes only\" version, they need to play that dance on Py3.\r\n\r\nSince having inconsistent APIs between Py2 and Py3 seems weird, I see 3 options:\r\n\r\n1. Break backwards compatibility and use unicode on Py2\r\n2. Add an optional flag on methods to opt-in to unicode mode on Py2, on by default on Py3\r\n3. Go back to the bytes-only API and provide a nicer wrapper for both Py2 and Py3\r\n\r\nIn my opinion, we should provide a simple way for library users to \"upgrade\" to the unicode behaviour when they enable Py3 support.\r\nThis could be easy if, under Py3, functions receiving bytes where they expect str fail with a message akin to ``Under Py3, this method takes bytes as argument. If you're adding Py3 support to your library, see pyldap.readthedocs.org/#upgrading-from-py2 for help``.\r\n\r\nI'll try to implement option 2 during the week-end."}}