{"action":"created","issue":{"url":"https://api.github.com/repos/certtools/intelmq/issues/290","labels_url":"https://api.github.com/repos/certtools/intelmq/issues/290/labels{/name}","comments_url":"https://api.github.com/repos/certtools/intelmq/issues/290/comments","events_url":"https://api.github.com/repos/certtools/intelmq/issues/290/events","html_url":"https://github.com/certtools/intelmq/issues/290","id":101596715,"number":290,"title":"DateTime timezone handling in harmonization","user":{"login":"sebix","id":199050,"avatar_url":"https://avatars.githubusercontent.com/u/199050?v=3","gravatar_id":"","url":"https://api.github.com/users/sebix","html_url":"https://github.com/sebix","followers_url":"https://api.github.com/users/sebix/followers","following_url":"https://api.github.com/users/sebix/following{/other_user}","gists_url":"https://api.github.com/users/sebix/gists{/gist_id}","starred_url":"https://api.github.com/users/sebix/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sebix/subscriptions","organizations_url":"https://api.github.com/users/sebix/orgs","repos_url":"https://api.github.com/users/sebix/repos","events_url":"https://api.github.com/users/sebix/events{/privacy}","received_events_url":"https://api.github.com/users/sebix/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/certtools/intelmq/labels/enhancement","name":"enhancement","color":"84b6eb"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/certtools/intelmq/milestones/2","html_url":"https://github.com/certtools/intelmq/milestones/Release%201","labels_url":"https://api.github.com/repos/certtools/intelmq/milestones/2/labels","id":719078,"number":2,"title":"Release 1","description":"","creator":{"login":"SYNchroACK","id":4184292,"avatar_url":"https://avatars.githubusercontent.com/u/4184292?v=3","gravatar_id":"","url":"https://api.github.com/users/SYNchroACK","html_url":"https://github.com/SYNchroACK","followers_url":"https://api.github.com/users/SYNchroACK/followers","following_url":"https://api.github.com/users/SYNchroACK/following{/other_user}","gists_url":"https://api.github.com/users/SYNchroACK/gists{/gist_id}","starred_url":"https://api.github.com/users/SYNchroACK/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SYNchroACK/subscriptions","organizations_url":"https://api.github.com/users/SYNchroACK/orgs","repos_url":"https://api.github.com/users/SYNchroACK/repos","events_url":"https://api.github.com/users/SYNchroACK/events{/privacy}","received_events_url":"https://api.github.com/users/SYNchroACK/received_events","type":"User","site_admin":false},"open_issues":31,"closed_issues":141,"state":"open","created_at":"2014-07-12T18:19:52Z","updated_at":"2015-09-02T10:40:18Z","due_on":"2014-08-31T07:00:00Z","closed_at":null},"comments":10,"created_at":"2015-08-18T08:04:33Z","updated_at":"2015-09-02T15:11:31Z","closed_at":null,"body":"The current implementation fails if the given time string has no timezone information. The code:\r\n\r\n    value = dateutil.parser.parse(value)\r\n    value = value.astimezone(pytz.utc)\r\n    value = value.isoformat()\r\n\r\nThe second line will fail if value is e.g. of format `20150101` with:\r\n\r\n    ValueError: astimezone() cannot be applied to a naive datetime\r\n\r\nRecommended way to convert timezones is:\r\n\r\n    value = pytz.utc.localize(value)\r\n\r\nBut this again assumes, that UTC is used. The question arises if we can always assume UTC for dates in collected data?\r\n\r\nThe Malware Domains parser will fail because of this bug."},"comment":{"url":"https://api.github.com/repos/certtools/intelmq/issues/comments/137119855","html_url":"https://github.com/certtools/intelmq/issues/290#issuecomment-137119855","issue_url":"https://api.github.com/repos/certtools/intelmq/issues/290","id":137119855,"user":{"login":"aaronkaplan","id":750019,"avatar_url":"https://avatars.githubusercontent.com/u/750019?v=3","gravatar_id":"","url":"https://api.github.com/users/aaronkaplan","html_url":"https://github.com/aaronkaplan","followers_url":"https://api.github.com/users/aaronkaplan/followers","following_url":"https://api.github.com/users/aaronkaplan/following{/other_user}","gists_url":"https://api.github.com/users/aaronkaplan/gists{/gist_id}","starred_url":"https://api.github.com/users/aaronkaplan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aaronkaplan/subscriptions","organizations_url":"https://api.github.com/users/aaronkaplan/orgs","repos_url":"https://api.github.com/users/aaronkaplan/repos","events_url":"https://api.github.com/users/aaronkaplan/events{/privacy}","received_events_url":"https://api.github.com/users/aaronkaplan/received_events","type":"User","site_admin":false},"created_at":"2015-09-02T15:11:31Z","updated_at":"2015-09-02T15:11:31Z","body":"\nOn Aug 24, 2015, at 12:03 AM, Tomás Lima <notifications@github.com> wrote:\n\n> @sebix , in the future the time.observation should be added by collectors.\nBut a collector did not split up the data yet and it's not the internal (JSON) format yet at that point.\nOnly after the parse. Therefore, only the parser can add time.observation IMHO.\n\nIt would be more accurate to have it at the collector, but he can't add that info (yet) at this stage IMHO.\n\nDid I miss anything?\n\na.\n\n"}}