{"action":"created","issue":{"url":"https://api.github.com/repos/4teamwork/ftw.zipexport/issues/33","labels_url":"https://api.github.com/repos/4teamwork/ftw.zipexport/issues/33/labels{/name}","comments_url":"https://api.github.com/repos/4teamwork/ftw.zipexport/issues/33/comments","events_url":"https://api.github.com/repos/4teamwork/ftw.zipexport/issues/33/events","html_url":"https://github.com/4teamwork/ftw.zipexport/pull/33","id":72389646,"number":33,"title":"Normalize paths in the ZIP file","user":{"login":"jone","id":7469,"avatar_url":"https://avatars.githubusercontent.com/u/7469?v=3","gravatar_id":"","url":"https://api.github.com/users/jone","html_url":"https://github.com/jone","followers_url":"https://api.github.com/users/jone/followers","following_url":"https://api.github.com/users/jone/following{/other_user}","gists_url":"https://api.github.com/users/jone/gists{/gist_id}","starred_url":"https://api.github.com/users/jone/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jone/subscriptions","organizations_url":"https://api.github.com/users/jone/orgs","repos_url":"https://api.github.com/users/jone/repos","events_url":"https://api.github.com/users/jone/events{/privacy}","received_events_url":"https://api.github.com/users/jone/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-05-01T09:09:54Z","updated_at":"2015-05-04T12:52:57Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/4teamwork/ftw.zipexport/pulls/33","html_url":"https://github.com/4teamwork/ftw.zipexport/pull/33","diff_url":"https://github.com/4teamwork/ftw.zipexport/pull/33.diff","patch_url":"https://github.com/4teamwork/ftw.zipexport/pull/33.patch"},"body":"The representations normally do not normalize the filenames and paths added to the ZIP.\r\nThe source of the names by a title field, for example.\r\nThis can lead to problems when characters are used which are invalid on the file system.\r\n\r\nThis pull-request adds path normalization for all added files directly in the generator.\r\n\r\n**Allowed characters:**\r\n- Alphanumeric characters (``A-Z``, ``a-z``, ``0-9``)\r\n- Normal brackets (`(` and `)`)\r\n- Dots, dashes, underscores (`.`, `_`, `-`)\r\n- OS path seperator (usually `/`, on windows `\\`; separates directories / files)\r\n- ~~Umlauts (generated with unicode data tools so that we do not miss exotic umlauts)~~\r\n\r\n**Invalid characters:**\r\nInvalid characters are replaced with a dash `-`.\r\n\r\n**Slashes:**\r\nSlashes (`/`) and backslashes (`\\`) are always converted to forward slashes (`/`),\r\nsince the zipfile standard module always uses forward slashes.\r\nhttp://stackoverflow.com/a/5040351/880628\r\n\r\n**Umlauts:**\r\n~~The list of umlauts is generated once and cached.\r\nIt is generated by combining all combination characters (diacritics) with all characters in the alphabet and try to combine them.~~\r\n\r\n**External usage:**\r\nRepresentations may rely on knowing the actually used path.\r\nIf this is the case, they must use the `normalize_path` function before adding the path, so that they know the normalize path:\r\n```python\r\nfrom ftw.zipexport.utils import normalize_path\r\n\r\n    def get_files(self, path_prefix=u\"\", recursive=True, toplevel=True):\r\n        ....\r\n        path = normalize_path(path)\r\n        yield (path, data)\r\n```\r\n\r\n**Customization (copied from readme)**:\r\n\r\nPaths are automatically using Plone's ``IFileNameNormalizer``.\r\nIf you want to use a custom normalization, you  can pass it to the ``ZipGenerator``:\r\n\r\n```python\r\n    from ftw.zipexport.generation import ZipGenerator\r\n    from Products.CMFPlone.utils import safe_unicode\r\n    import re\r\n\r\n    def normalize_path(path):\r\n        path = safe_unicode(path).replace(u'\\\\', u'/')\r\n        return re.sub(ur'[^\\w\\-.\\/]', u'', path)\r\n\r\n    with ZipGenerator(path_normalizer=normalize_path) as zipgenerator:\r\n        ...\r\n```\r\n\r\nPass ``None`` to disable normalization completely:\r\n\r\n```python\r\n    from ftw.zipexport.generation import ZipGenerator\r\n\r\n    with ZipGenerator(path_normalizer=None) as zipgenerator:\r\n        ...\r\n```\r\n\r\nCloses #28 \r\n@shylux \r\n/cc @maethu @lukasgraf "},"comment":{"url":"https://api.github.com/repos/4teamwork/ftw.zipexport/issues/comments/98698554","html_url":"https://github.com/4teamwork/ftw.zipexport/pull/33#issuecomment-98698554","issue_url":"https://api.github.com/repos/4teamwork/ftw.zipexport/issues/33","id":98698554,"user":{"login":"jone","id":7469,"avatar_url":"https://avatars.githubusercontent.com/u/7469?v=3","gravatar_id":"","url":"https://api.github.com/users/jone","html_url":"https://github.com/jone","followers_url":"https://api.github.com/users/jone/followers","following_url":"https://api.github.com/users/jone/following{/other_user}","gists_url":"https://api.github.com/users/jone/gists{/gist_id}","starred_url":"https://api.github.com/users/jone/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jone/subscriptions","organizations_url":"https://api.github.com/users/jone/orgs","repos_url":"https://api.github.com/users/jone/repos","events_url":"https://api.github.com/users/jone/events{/privacy}","received_events_url":"https://api.github.com/users/jone/received_events","type":"User","site_admin":false},"created_at":"2015-05-04T12:52:57Z","updated_at":"2015-05-04T12:52:57Z","body":"@lukasgraf I've refactored to use Plone's ``IFileNameNormalizer`` instead.\r\nIt also possible to use a custom normalization function by passing it to the generator on initialization."}}