{"action":"created","issue":{"url":"https://api.github.com/repos/hire-vladimir/SA-quandl/issues/3","labels_url":"https://api.github.com/repos/hire-vladimir/SA-quandl/issues/3/labels{/name}","comments_url":"https://api.github.com/repos/hire-vladimir/SA-quandl/issues/3/comments","events_url":"https://api.github.com/repos/hire-vladimir/SA-quandl/issues/3/events","html_url":"https://github.com/hire-vladimir/SA-quandl/issues/3","id":105924116,"number":3,"title":"convert quandl \"time\" field to _time","user":{"login":"hire-vladimir","id":8716177,"avatar_url":"https://avatars.githubusercontent.com/u/8716177?v=3","gravatar_id":"","url":"https://api.github.com/users/hire-vladimir","html_url":"https://github.com/hire-vladimir","followers_url":"https://api.github.com/users/hire-vladimir/followers","following_url":"https://api.github.com/users/hire-vladimir/following{/other_user}","gists_url":"https://api.github.com/users/hire-vladimir/gists{/gist_id}","starred_url":"https://api.github.com/users/hire-vladimir/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hire-vladimir/subscriptions","organizations_url":"https://api.github.com/users/hire-vladimir/orgs","repos_url":"https://api.github.com/users/hire-vladimir/repos","events_url":"https://api.github.com/users/hire-vladimir/events{/privacy}","received_events_url":"https://api.github.com/users/hire-vladimir/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/hire-vladimir/SA-quandl/labels/enhancement","name":"enhancement","color":"84b6eb"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-09-11T00:32:31Z","updated_at":"2015-09-11T11:48:58Z","closed_at":null,"body":"make this process easier vs using a eval after"},"comment":{"url":"https://api.github.com/repos/hire-vladimir/SA-quandl/issues/comments/139525613","html_url":"https://github.com/hire-vladimir/SA-quandl/issues/3#issuecomment-139525613","issue_url":"https://api.github.com/repos/hire-vladimir/SA-quandl/issues/3","id":139525613,"user":{"login":"hire-vladimir","id":8716177,"avatar_url":"https://avatars.githubusercontent.com/u/8716177?v=3","gravatar_id":"","url":"https://api.github.com/users/hire-vladimir","html_url":"https://github.com/hire-vladimir","followers_url":"https://api.github.com/users/hire-vladimir/followers","following_url":"https://api.github.com/users/hire-vladimir/following{/other_user}","gists_url":"https://api.github.com/users/hire-vladimir/gists{/gist_id}","starred_url":"https://api.github.com/users/hire-vladimir/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hire-vladimir/subscriptions","organizations_url":"https://api.github.com/users/hire-vladimir/orgs","repos_url":"https://api.github.com/users/hire-vladimir/repos","events_url":"https://api.github.com/users/hire-vladimir/events{/privacy}","received_events_url":"https://api.github.com/users/hire-vladimir/received_events","type":"User","site_admin":false},"created_at":"2015-09-11T11:48:58Z","updated_at":"2015-09-11T11:48:58Z","body":"done.\r\n\r\n```\r\ndiff --git a/bin/quandl.py b/bin/quandl.py\r\nindex af051c4..4ab2eab 100755\r\n--- a/bin/quandl.py\r\n+++ b/bin/quandl.py\r\n@@ -67,7 +67,7 @@ def die(msg):\r\n     exit(msg)\r\n\r\n\r\n-def quandl2splunk(quandl_data, show_info=False):\r\n+def quandl2splunk(quandl_data, show_info=False, convert_time=True):\r\n     payload = []\r\n     quandl_code = \"%s/%s\" % (quandl_data['dataset']['database_code'], quandl_data['dataset']['dataset_code'])\r\n     headers = quandl_data['dataset']['column_names']\r\n@@ -83,7 +83,11 @@ def quandl2splunk(quandl_data, show_info=False):\r\n             d = quandl_data['dataset']['data'][r]\r\n\r\n             for x in range(0, len(headers)):\r\n-                row[headers[x].replace(\" \", \"_\")] = d[x]\r\n+                h = headers[x].replace(\" \", \"_\")\r\n+                row[h] = d[x]\r\n+                if convert_time and (h in ['Date', 'Year', 'Month'] and re.match(\"^\\d{4}-\\d{2}-\\d{2}$\", d[x])):\r\n+                    row['_time'] = time.mktime(time.strptime(d[x], \"%Y-%m-%d\"))  # derive epoc from date\r\n+\r\n             payload.append(row)\r\n     logger.debug('quandl2splunk data=\"%s\"' % payload)\r\n\r\n@@ -94,7 +98,7 @@ def validate_args(keywords, argvals):\r\n     logger.info('function=\"validate_args\" calling getKeywordsAndOptions keywords=\"%s\" args=\"%s\"' % (str(keywords), str(argvals)\r\n\r\n     # validate args\r\n-    ALLOWED_OPTIONS = ['debug', 'metadata', 'auth_token', 'limit', 'rows', 'column_index', 'start_date', 'end_date', 'order', '\r\n+    ALLOWED_OPTIONS = ['debug', 'metadata', 'convert_time', 'auth_token', 'limit', 'rows', 'column_index', 'start_date', 'end_d\r\n     # MANDATORY_OPTIONS = ['dest']\r\n\r\n     for opt in argvals:\r\n@@ -182,8 +186,12 @@ if __name__ == '__main__':\r\n                 quandl_uri = \"https://www.quandl.com/api/v3/datasets/%s/%s/metadata.json\" % (quandl_database, set)\r\n             logger.debug('effective uri=\"%s\"' % quandl_uri)\r\n\r\n+            create_time = True\r\n+            if arg_on_and_enabled(argvals, \"convert_time\", rex=\"^(?:f|false|0|no)$\"):\r\n+                create_time = False\r\n+\r\n             quandl_data = json.loads(getDataPayload(quandl_uri))\r\n-            uber += quandl2splunk(quandl_data, quandl_show_info)\r\n+            uber += quandl2splunk(quandl_data, quandl_show_info, create_time)\r\n         # keeping all data into single array is waste of memory; need to figure out how to call outputResults multiple times, w\r\n         logger.info('sending events to splunk count=\"%s\"' % len(uber))\r\n         si.outputResults(uber)\r\n\r\n```"}}