{"action":"created","issue":{"url":"https://api.github.com/repos/clab/wikipedia-parallel-titles/issues/1","labels_url":"https://api.github.com/repos/clab/wikipedia-parallel-titles/issues/1/labels{/name}","comments_url":"https://api.github.com/repos/clab/wikipedia-parallel-titles/issues/1/comments","events_url":"https://api.github.com/repos/clab/wikipedia-parallel-titles/issues/1/events","html_url":"https://github.com/clab/wikipedia-parallel-titles/issues/1","id":49788670,"number":1,"title":"scripts not working with unicode input on mac","user":{"login":"roeeaharoni","id":6319492,"avatar_url":"https://avatars.githubusercontent.com/u/6319492?v=3","gravatar_id":"","url":"https://api.github.com/users/roeeaharoni","html_url":"https://github.com/roeeaharoni","followers_url":"https://api.github.com/users/roeeaharoni/followers","following_url":"https://api.github.com/users/roeeaharoni/following{/other_user}","gists_url":"https://api.github.com/users/roeeaharoni/gists{/gist_id}","starred_url":"https://api.github.com/users/roeeaharoni/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/roeeaharoni/subscriptions","organizations_url":"https://api.github.com/users/roeeaharoni/orgs","repos_url":"https://api.github.com/users/roeeaharoni/repos","events_url":"https://api.github.com/users/roeeaharoni/events{/privacy}","received_events_url":"https://api.github.com/users/roeeaharoni/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2014-11-22T12:55:55Z","updated_at":"2015-12-01T17:57:45Z","closed_at":null,"body":"Hi!\r\n\r\nI Tried to use the tool according to the readme file, on macosx, with hebrew as the source language and arabic as the target. When I executed the following command (after installing the dependencies):\r\n\r\n./build-corpus.sh ar hewiki-20141102 > titles_he_ar.txt\r\n\r\nI got the following output:\r\n\r\nTarget language code: ar\r\nUsing hewiki-20141102-langlinks.sql.gz\r\nUsing hewiki-20141102-page.sql.gz\r\nReading page data from hewiki-20141102-page.sql.gz...\r\niconv: conversion from utf8 unsupported\r\niconv: try 'iconv -l' to get the list of supported encodings\r\n  read 0 documents\r\nReading langlinks data from hewiki-20141102-langlinks.sql.gz...\r\niconv: conversion from utf8 unsupported\r\niconv: try 'iconv -l' to get the list of supported encodings\r\n  read 0 documents\r\n\r\nI tried to fix this by changing the perl scripts that called iconv with parameter 'utf8' to call it with 'utf-8', and it seems to work fine now. \r\n\r\nBest regards,\r\nRoee\r\n"},"comment":{"url":"https://api.github.com/repos/clab/wikipedia-parallel-titles/issues/comments/161047786","html_url":"https://github.com/clab/wikipedia-parallel-titles/issues/1#issuecomment-161047786","issue_url":"https://api.github.com/repos/clab/wikipedia-parallel-titles/issues/1","id":161047786,"user":{"login":"wammar","id":1858803,"avatar_url":"https://avatars.githubusercontent.com/u/1858803?v=3","gravatar_id":"","url":"https://api.github.com/users/wammar","html_url":"https://github.com/wammar","followers_url":"https://api.github.com/users/wammar/followers","following_url":"https://api.github.com/users/wammar/following{/other_user}","gists_url":"https://api.github.com/users/wammar/gists{/gist_id}","starred_url":"https://api.github.com/users/wammar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wammar/subscriptions","organizations_url":"https://api.github.com/users/wammar/orgs","repos_url":"https://api.github.com/users/wammar/repos","events_url":"https://api.github.com/users/wammar/events{/privacy}","received_events_url":"https://api.github.com/users/wammar/received_events","type":"User","site_admin":false},"created_at":"2015-12-01T17:57:44Z","updated_at":"2015-12-01T17:57:44Z","body":"Hi Saad,\n\nTry the following command:\n ./build-corpus.sh en urwiki-20151123\n\nWaleed\nOn Dec 1, 2015 9:46 AM, \"Saad Ahmed\" <notifications@github.com> wrote:\n\n> @roeeaharoni <https://github.com/roeeaharoni> @redpony\n> <https://github.com/redpony> @wammar <https://github.com/wammar>\n>\n> —\n> Reply to this email directly or view it on GitHub\n> <https://github.com/clab/wikipedia-parallel-titles/issues/1#issuecomment-161044919>\n> .\n>\n"}}