{"action":"created","issue":{"url":"https://api.github.com/repos/blaze/blaze/issues/1341","labels_url":"https://api.github.com/repos/blaze/blaze/issues/1341/labels{/name}","comments_url":"https://api.github.com/repos/blaze/blaze/issues/1341/comments","events_url":"https://api.github.com/repos/blaze/blaze/issues/1341/events","html_url":"https://github.com/blaze/blaze/issues/1341","id":120454691,"number":1341,"title":"Nondeterministic bug with specific datashape on load","user":{"login":"kwmsmith","id":288771,"avatar_url":"https://avatars.githubusercontent.com/u/288771?v=3","gravatar_id":"","url":"https://api.github.com/users/kwmsmith","html_url":"https://github.com/kwmsmith","followers_url":"https://api.github.com/users/kwmsmith/followers","following_url":"https://api.github.com/users/kwmsmith/following{/other_user}","gists_url":"https://api.github.com/users/kwmsmith/gists{/gist_id}","starred_url":"https://api.github.com/users/kwmsmith/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kwmsmith/subscriptions","organizations_url":"https://api.github.com/users/kwmsmith/orgs","repos_url":"https://api.github.com/users/kwmsmith/repos","events_url":"https://api.github.com/users/kwmsmith/events{/privacy}","received_events_url":"https://api.github.com/users/kwmsmith/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/blaze/blaze/labels/bug","name":"bug","color":"fc2929"},{"url":"https://api.github.com/repos/blaze/blaze/labels/odo","name":"odo","color":"fad8c7"}],"state":"open","locked":false,"assignee":{"login":"cowlicks","id":598099,"avatar_url":"https://avatars.githubusercontent.com/u/598099?v=3","gravatar_id":"","url":"https://api.github.com/users/cowlicks","html_url":"https://github.com/cowlicks","followers_url":"https://api.github.com/users/cowlicks/followers","following_url":"https://api.github.com/users/cowlicks/following{/other_user}","gists_url":"https://api.github.com/users/cowlicks/gists{/gist_id}","starred_url":"https://api.github.com/users/cowlicks/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cowlicks/subscriptions","organizations_url":"https://api.github.com/users/cowlicks/orgs","repos_url":"https://api.github.com/users/cowlicks/repos","events_url":"https://api.github.com/users/cowlicks/events{/privacy}","received_events_url":"https://api.github.com/users/cowlicks/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/blaze/blaze/milestones/8","html_url":"https://github.com/blaze/blaze/milestones/0.9.0","labels_url":"https://api.github.com/repos/blaze/blaze/milestones/8/labels","id":940625,"number":8,"title":"0.9.0","description":"","creator":{"login":"cpcloud","id":417981,"avatar_url":"https://avatars.githubusercontent.com/u/417981?v=3","gravatar_id":"","url":"https://api.github.com/users/cpcloud","html_url":"https://github.com/cpcloud","followers_url":"https://api.github.com/users/cpcloud/followers","following_url":"https://api.github.com/users/cpcloud/following{/other_user}","gists_url":"https://api.github.com/users/cpcloud/gists{/gist_id}","starred_url":"https://api.github.com/users/cpcloud/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cpcloud/subscriptions","organizations_url":"https://api.github.com/users/cpcloud/orgs","repos_url":"https://api.github.com/users/cpcloud/repos","events_url":"https://api.github.com/users/cpcloud/events{/privacy}","received_events_url":"https://api.github.com/users/cpcloud/received_events","type":"User","site_admin":false},"open_issues":11,"closed_issues":86,"state":"open","created_at":"2015-01-18T16:00:30Z","updated_at":"2015-12-04T21:49:20Z","due_on":"2016-02-01T05:00:00Z","closed_at":null},"comments":2,"created_at":"2015-12-04T18:05:16Z","updated_at":"2015-12-04T22:17:03Z","closed_at":null,"body":"The `string[25]` seems to be the culprit in the dshape below.\r\n\r\nWhen I make it a `string` type, things work fine.\r\n\r\nSo something's going on with the `string[N]` dshape type.\r\n\r\n```python\r\nfrom __future__ import print_function\r\nimport blaze as bz\r\n\r\nds = bz.dshape(\"\"\"var * {\r\n                  k: date,\r\n                  n: string[25],\r\n                  }\"\"\")\r\n\r\nd = bz.Data('./test.csv', dshape=ds)\r\nprint(bz.compute(d.nrows))\r\n```\r\n\r\ntest.csv:\r\n\r\n```\r\n2015-03-13,FOO THE BAR\r\n2014-01-29,BAZ THE QUUX\r\n```\r\n\r\nSometimes, I get an `OSError` with a weird `Temp(URL(CSV))` object:\r\n\r\n```python\r\n$ python breakage.py\r\n/Users/ksmith/work/odo/odo/core.py:65: FailedConversionWarning: Failed on CSV -> chunks(pandas.DataFrame). Working around\r\nError message:\r\nthe dtype <U25 is not supported for parsing\r\n  warn(FailedConversionWarning(A, B, e))\r\nTraceback (most recent call last):\r\n  File \"breakage.py\", line 31, in <module>\r\n    print(bz.compute(d.nrows))\r\n  File \"/Users/ksmith/anaconda/lib/python2.7/site-packages/multipledispatch/dispatcher.py\", line 164, in __call__\r\n    return func(*args, **kwargs)\r\n  File \"/Users/ksmith/work/blaze/blaze-repo/blaze/interactive.py\", line 171, in compute\r\n    return compute(expr, resources, **kwargs)\r\n[...]\r\nRuntimeError: maximum recursion depth exceeded in cmp\r\n```\r\n\r\nOther times (nondeterministically), I get a different exception:\r\n\r\n```python\r\n❯❯❯ python breakage.py\r\n/Users/ksmith/work/odo/odo/core.py:65: FailedConversionWarning: Failed on CSV -> chunks(pandas.DataFrame). Working around\r\nError message:\r\nthe dtype <U25 is not supported for parsing\r\n  warn(FailedConversionWarning(A, B, e))\r\nTraceback (most recent call last):\r\n  File \"breakage.py\", line 11, in <module>\r\n    print(bz.compute(d.nrows))\r\n  File \"/Users/ksmith/anaconda/lib/python2.7/site-packages/multipledispatch/dispatcher.py\", line 164, in __call__\r\n    return func(*args, **kwargs)\r\n  File \"/Users/ksmith/work/blaze/blaze-repo/blaze/interactive.py\", line 171, in compute\r\n    return compute(expr, resources, **kwargs)\r\n  File \"/Users/ksmith/anaconda/lib/python2.7/site-packages/multipledispatch/dispatcher.py\", line 164, in __call__\r\n    return func(*args, **kwargs)\r\n  File \"/Users/ksmith/work/blaze/blaze-repo/blaze/compute/core.py\", line 454, in compute\r\n    for e, dat in d2.items()\r\n  File \"/Users/ksmith/work/blaze/blaze-repo/blaze/compute/core.py\", line 455, in <genexpr>\r\n    if e in expr2\r\n  File \"/Users/ksmith/anaconda/lib/python2.7/site-packages/multipledispatch/dispatcher.py\", line 164, in __call__\r\n    return func(*args, **kwargs)\r\n  File \"/Users/ksmith/work/blaze/blaze-repo/blaze/compute/csv.py\", line 54, in pre_compute\r\n    return into(pd.DataFrame, data, dshape=leaf.dshape, **kwargs)\r\n  File \"/Users/ksmith/anaconda/lib/python2.7/site-packages/multipledispatch/dispatcher.py\", line 164, in __call__\r\n    return func(*args, **kwargs)\r\n  File \"/Users/ksmith/work/odo/odo/into.py\", line 40, in wrapped\r\n    return f(*args, **kwargs)\r\n  File \"/Users/ksmith/work/odo/odo/into.py\", line 50, in into_type\r\n    return convert(a, b, dshape=dshape, **kwargs)\r\n  File \"/Users/ksmith/work/odo/odo/core.py\", line 44, in __call__\r\n    return _transform(self.graph, *args, **kwargs)\r\n  File \"/Users/ksmith/work/odo/odo/core.py\", line 68, in _transform\r\n    **kwargs)\r\n  File \"/Users/ksmith/work/odo/odo/core.py\", line 60, in _transform\r\n    x = f(x, excluded_edges=excluded_edges, **kwargs)\r\n  File \"/Users/ksmith/work/odo/odo/backends/aws.py\", line 213, in text_to_temp_s3_text\r\n    return append(Temp(S3(subtype))(uri, **kwargs), data)\r\n  File \"/Users/ksmith/work/odo/odo/backends/aws.py\", line 80, in __init__\r\n    kwargs))\r\n  File \"/Users/ksmith/anaconda/lib/python2.7/site-packages/boto/s3/connection.py\", line 621, in create_bucket\r\n    response.status, response.reason, body)\r\nboto.exception.S3ResponseError: S3ResponseError: 403 Forbidden\r\n<?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\n<Error><Code>AccessDenied</Code><Message>Anonymous access is forbidden for this operation</Message><RequestId>63D846B536C3E749</RequestId><HostId>/9jvIf2d9y8tixIHSVE9LkNDiqFFO9GCj8LFFeqeARo3t/0/hn/FlC/68mgoHdfJoT8DYdh7s4E=</HostId></Error>\r\nException AttributeError: AttributeError(\"'Temp(S3(CSV))' object has no attribute 'object'\",) in <bound method Temp(S3(CSV)).__del__ of <odo.temp.Temp(S3(CSV)) object at 0x11365ac10>> ignored\r\n```\r\n\r\nThis one involves `Temp(S3(CSV))` for some strange reason.\r\n\r\nThe non-repeatability of this error is disturbing."},"comment":{"url":"https://api.github.com/repos/blaze/blaze/issues/comments/162097629","html_url":"https://github.com/blaze/blaze/issues/1341#issuecomment-162097629","issue_url":"https://api.github.com/repos/blaze/blaze/issues/1341","id":162097629,"user":{"login":"cpcloud","id":417981,"avatar_url":"https://avatars.githubusercontent.com/u/417981?v=3","gravatar_id":"","url":"https://api.github.com/users/cpcloud","html_url":"https://github.com/cpcloud","followers_url":"https://api.github.com/users/cpcloud/followers","following_url":"https://api.github.com/users/cpcloud/following{/other_user}","gists_url":"https://api.github.com/users/cpcloud/gists{/gist_id}","starred_url":"https://api.github.com/users/cpcloud/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cpcloud/subscriptions","organizations_url":"https://api.github.com/users/cpcloud/orgs","repos_url":"https://api.github.com/users/cpcloud/repos","events_url":"https://api.github.com/users/cpcloud/events{/privacy}","received_events_url":"https://api.github.com/users/cpcloud/received_events","type":"User","site_admin":false},"created_at":"2015-12-04T22:17:03Z","updated_at":"2015-12-04T22:17:03Z","body":"ok i got to the bottom of this \r\n\r\nthe most important thing to notice is this:\r\n\r\n```\r\nthe dtype <U25 is not supported for parsing\r\n  warn(FailedConversionWarning(A, B, e))\r\n```"}}