{"action":"created","issue":{"url":"https://api.github.com/repos/mapnik/mapnik-support/issues/31","labels_url":"https://api.github.com/repos/mapnik/mapnik-support/issues/31/labels{/name}","comments_url":"https://api.github.com/repos/mapnik/mapnik-support/issues/31/comments","events_url":"https://api.github.com/repos/mapnik/mapnik-support/issues/31/events","html_url":"https://github.com/mapnik/mapnik-support/issues/31","id":54912872,"number":31,"title":"Segmentation fault on WKB + unicode properties (Python)","user":{"login":"richterb","id":1935200,"avatar_url":"https://avatars.githubusercontent.com/u/1935200?v=3","gravatar_id":"","url":"https://api.github.com/users/richterb","html_url":"https://github.com/richterb","followers_url":"https://api.github.com/users/richterb/followers","following_url":"https://api.github.com/users/richterb/following{/other_user}","gists_url":"https://api.github.com/users/richterb/gists{/gist_id}","starred_url":"https://api.github.com/users/richterb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/richterb/subscriptions","organizations_url":"https://api.github.com/users/richterb/orgs","repos_url":"https://api.github.com/users/richterb/repos","events_url":"https://api.github.com/users/richterb/events{/privacy}","received_events_url":"https://api.github.com/users/richterb/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2015-01-20T17:27:29Z","updated_at":"2015-01-21T18:09:30Z","closed_at":null,"body":"We are having an issue with our Mapnik rendering code which I'll try to describe here. We are unable to reproduce it outside of our own environment, and we have a not-too-horrible workaround, so at this point we are rather looking for more ideas to pinpoint the problem, as I think our scope is still a bit too broad.\r\n\r\nUsing Mapnik's Python bindings, we are creating a MemoryDatasource in which we are adding Features one by one. Each Feature is built from a geometry object (WKT or WKB), and we add properties to it.\r\n\r\nWe are having a violent python crash (**segmentation fault**) in the following conditions:\r\n- **we are adding geometry into features using WKB** (`add_geometries_from_wkb`); the problem never occurs if we use WKT (`add_geometries_from_wkt`) instead\r\n- **we are running Ubuntu 14.04**; we don't get this crash under OS X\r\n- **at least one of the values we are setting as properties in the features is an unicode python string**. For example `f['prop'] = u'aaa'` may trigger the crash, while `f['prop'] = 'aaa'` will not.\r\n\r\nThe crash always occurs during one of the properties assignment (e.g. `f['prop'] = u'...'`). It never reaches the rendering step.\r\n\r\nThe crashing code is running inside an UWSGI daemon. Following our tests, it appears that in a mono-threaded single UWSGI process, the crash appears 100% of the time on a constant test dataset, while on a multithreaded setup (10 threads) it appears to crash 50% of the time. This occurs on a Mapnik built from master in the last few days.\r\n\r\nUnfortunately we are unable to reproduce it in a stand-alone script, but I'd be happy to hear some ideas of where to look at; since the conditions of the crash are pretty specific, it may ring a bell ;)\r\n\r\nOur code look roughly like this:\r\n\r\n```python\r\n    m = mapnik.Map(256, 256)\r\n    mapnik.load_map(m, 'ods.xml')\r\n\r\n    feature_gen = get_feature_generator() # A generator that yields geometry+properties objects\r\n\r\n    layer = m.layers[0]\r\n    ds = mapnik.MemoryDatasource()\r\n\r\n    ids = itertools.count(0)\r\n\r\n    fields = [\"ods:field1\", \"ods:field2\", \"ods:field3\", \"ods:field4\", \"ods:field5\", \"ods:field6\"]\r\n\r\n    for feature in feature_gen:\r\n        mapnik_context = mapnik.Context()\r\n        for field in fields:\r\n            mapnik_context.push(field)\r\n\r\n        f = mapnik.Feature(mapnik_context, ids.next())\r\n\r\n\t\tf.add_geometries_from_wkb(base64.decodestring(feature['geometry'])) # Using WKB \r\n\r\n\t\tfeature_properties = feature.get('properties', {})\r\n\r\n\t\tf['shape'] = feature_properties.get('shape')\r\n\t\tf['marker'] = feature_properties.get('marker')\r\n\t\tf['marker-icon'] = feature_properties.get('icon')\r\n\r\n\t\tds.add_feature(f)\r\n\r\n\t# The rest of the rendering usual stuff\r\n```"},"comment":{"url":"https://api.github.com/repos/mapnik/mapnik-support/issues/comments/70889460","html_url":"https://github.com/mapnik/mapnik-support/issues/31#issuecomment-70889460","issue_url":"https://api.github.com/repos/mapnik/mapnik-support/issues/31","id":70889460,"user":{"login":"springmeyer","id":20300,"avatar_url":"https://avatars.githubusercontent.com/u/20300?v=3","gravatar_id":"","url":"https://api.github.com/users/springmeyer","html_url":"https://github.com/springmeyer","followers_url":"https://api.github.com/users/springmeyer/followers","following_url":"https://api.github.com/users/springmeyer/following{/other_user}","gists_url":"https://api.github.com/users/springmeyer/gists{/gist_id}","starred_url":"https://api.github.com/users/springmeyer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/springmeyer/subscriptions","organizations_url":"https://api.github.com/users/springmeyer/orgs","repos_url":"https://api.github.com/users/springmeyer/repos","events_url":"https://api.github.com/users/springmeyer/events{/privacy}","received_events_url":"https://api.github.com/users/springmeyer/received_events","type":"User","site_admin":false},"created_at":"2015-01-21T18:09:30Z","updated_at":"2015-01-21T18:09:30Z","body":"Okay, thanks. BTW, you only need to create the `mapnik_context` once and you can reuse it for each feature."}}