{"action":"opened","issue":{"url":"https://api.github.com/repos/thunder-project/thunder/issues/125","labels_url":"https://api.github.com/repos/thunder-project/thunder/issues/125/labels{/name}","comments_url":"https://api.github.com/repos/thunder-project/thunder/issues/125/comments","events_url":"https://api.github.com/repos/thunder-project/thunder/issues/125/events","html_url":"https://github.com/thunder-project/thunder/issues/125","id":59856180,"number":125,"title":"SSL times out pulling from S3","user":{"login":"npyoung","id":1348767,"avatar_url":"https://avatars.githubusercontent.com/u/1348767?v=3","gravatar_id":"","url":"https://api.github.com/users/npyoung","html_url":"https://github.com/npyoung","followers_url":"https://api.github.com/users/npyoung/followers","following_url":"https://api.github.com/users/npyoung/following{/other_user}","gists_url":"https://api.github.com/users/npyoung/gists{/gist_id}","starred_url":"https://api.github.com/users/npyoung/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/npyoung/subscriptions","organizations_url":"https://api.github.com/users/npyoung/orgs","repos_url":"https://api.github.com/users/npyoung/repos","events_url":"https://api.github.com/users/npyoung/events{/privacy}","received_events_url":"https://api.github.com/users/npyoung/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-03-04T20:23:24Z","updated_at":"2015-03-04T20:23:24Z","closed_at":null,"body":"I'm getting these errors when running\r\n\r\n```python\r\ntsc.convertImagesToSeries(vol_path, vol_path + '/series', dims=(48,2048,1184), inputFormat='stack', dtype='uint16', blockSize=\"150M\", shuffle=True, overwrite=False)\r\n```\r\n\r\nIt looks like it's timing out when reading from S3. I increased boto's http timeout, but it didn't help. Can something be done at the Thunder level to set the SSL timeout, load things in chunks, or perhaps try-catch-retry?\r\n\r\n```\r\norg.apache.spark.api.python.PythonException: Traceback (most recent call last):\r\n  File \"/root/spark-1.2.1-bin-hadoop2.4/python/pyspark/worker.py\", line 107, in main\r\n    process()\r\n  File \"/root/spark-1.2.1-bin-hadoop2.4/python/pyspark/worker.py\", line 98, in process\r\n    serializer.dump_stream(func(split_index, iterator), outfile)\r\n  File \"/root/spark-1.2.1-bin-hadoop2.4/python/pyspark/rdd.py\", line 2081, in pipeline_func\r\n    return func(split, prev_func(split, iterator))\r\n  File \"/root/spark-1.2.1-bin-hadoop2.4/python/pyspark/rdd.py\", line 2081, in pipeline_func\r\n    return func(split, prev_func(split, iterator))\r\n  File \"/root/spark-1.2.1-bin-hadoop2.4/python/pyspark/rdd.py\", line 258, in func\r\n    return f(iterator)\r\n  File \"/root/spark-1.2.1-bin-hadoop2.4/python/pyspark/rdd.py\", line 1571, in combineLocally\r\n    merger.mergeValues(iterator)\r\n  File \"/root/spark-1.2.1-bin-hadoop2.4/python/pyspark/shuffle.py\", line 252, in mergeValues\r\n    for k, v in iterator:\r\n  File \"/usr/local/lib/python2.7/dist-packages/thunder_python-0.4.1-py2.7.egg/thunder/rdds/fileio/readers.py\", line 306, in readSplitFromS3\r\n    buf = key.get_contents_as_string()\r\n  File \"/usr/local/lib/python2.7/dist-packages/boto/s3/key.py\", line 1780, in get_contents_as_string\r\n    response_headers=response_headers)\r\n  File \"/usr/local/lib/python2.7/dist-packages/boto/s3/key.py\", line 1648, in get_contents_to_file\r\n    response_headers=response_headers)\r\n  File \"/usr/local/lib/python2.7/dist-packages/boto/s3/key.py\", line 1480, in get_file\r\n    query_args=None)\r\n  File \"/usr/local/lib/python2.7/dist-packages/boto/s3/key.py\", line 1533, in _get_file_internal\r\n    for bytes in self:\r\n  File \"/usr/local/lib/python2.7/dist-packages/boto/s3/key.py\", line 386, in next\r\n    data = self.resp.read(self.BufferSize)\r\n  File \"/usr/local/lib/python2.7/dist-packages/boto/connection.py\", line 413, in read\r\n    return http_client.HTTPResponse.read(self, amt)\r\n  File \"/usr/lib/python2.7/httplib.py\", line 567, in read\r\n    s = self.fp.read(amt)\r\n  File \"/usr/lib/python2.7/socket.py\", line 380, in read\r\n    data = self._sock.recv(left)\r\n  File \"/usr/lib/python2.7/ssl.py\", line 341, in recv\r\n    return self.read(buflen)\r\n  File \"/usr/lib/python2.7/ssl.py\", line 260, in read\r\n    return self._sslobj.read(len)\r\nSSLError: The read operation timed out\r\n```"}}