{"action":"created","issue":{"url":"https://api.github.com/repos/ImageEngine/gaffer/issues/1517","labels_url":"https://api.github.com/repos/ImageEngine/gaffer/issues/1517/labels{/name}","comments_url":"https://api.github.com/repos/ImageEngine/gaffer/issues/1517/comments","events_url":"https://api.github.com/repos/ImageEngine/gaffer/issues/1517/events","html_url":"https://github.com/ImageEngine/gaffer/pull/1517","id":112821746,"number":1517,"title":"Parallel process image data","user":{"login":"HughMacdonald","id":625512,"avatar_url":"https://avatars.githubusercontent.com/u/625512?v=3","gravatar_id":"","url":"https://api.github.com/users/HughMacdonald","html_url":"https://github.com/HughMacdonald","followers_url":"https://api.github.com/users/HughMacdonald/followers","following_url":"https://api.github.com/users/HughMacdonald/following{/other_user}","gists_url":"https://api.github.com/users/HughMacdonald/gists{/gist_id}","starred_url":"https://api.github.com/users/HughMacdonald/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HughMacdonald/subscriptions","organizations_url":"https://api.github.com/users/HughMacdonald/orgs","repos_url":"https://api.github.com/users/HughMacdonald/repos","events_url":"https://api.github.com/users/HughMacdonald/events{/privacy}","received_events_url":"https://api.github.com/users/HughMacdonald/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/ImageEngine/gaffer/labels/pr-revisionRequired","name":"pr-revisionRequired","color":"FFAAAA"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":28,"created_at":"2015-10-22T14:56:47Z","updated_at":"2015-11-13T17:08:18Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/ImageEngine/gaffer/pulls/1517","html_url":"https://github.com/ImageEngine/gaffer/pull/1517","diff_url":"https://github.com/ImageEngine/gaffer/pull/1517.diff","patch_url":"https://github.com/ImageEngine/gaffer/pull/1517.patch"},"body":"This PR adds ImageAlgo methods for parallel processing of tiles/channels, and re-implements both ImagePlug::image() and ImageWriter to use this.\r\n\r\nImageAlgo has 4 new methods:\r\n\r\n* parallelProcessTiles\r\n  * Two versions\r\n    * One takes an imagePlug, a functor and an optional window to process (default will be imagePlug->dataWindowPlug()\r\n    * One takes the same parameters as above, plus a vector<string> of channel names, and will call the functor once per tile per channel\r\n  * This will call the defined functor once per tile (per channel)\r\n* parallelGatherTiles\r\n  * Same two versions as above\r\n  * Also takes:\r\n    * Gather functor, which will be called for the same tiles/channels, but in series.\r\n    * tileOrder, which defines the order that the gather functor is called in. One of:\r\n      * Unordered : No guarantee of ordering. Will tend towards bottom to top.\r\n      * BottomToTop : Guaranteed iteration through channels, tiles in X, tiles in Y, from the bottom\r\n      * TopToBottom : Same as BottomToTop, but from the top in Y\r\n\r\nImagePlug::image(), which previously dealt with its own parallelising, now uses parallelProcessTiles() to set off parallel processing of channel data to create an ImagePrimitive.\r\nThe parallelising that it used to do had a bug where it would often end up processing the same tiles multiple times, due to a misunderstanding of how tbb's blocked_range dealt with splitting up ranges. It would sometimes pass through ranges that were smaller than `grainsize`. As such, it now does all of the splitting using IDs, and then these are turned back into coordinates internally.\r\n\r\nImageWriter used to call ImagePlug::image() to get a full image before writing it. It now has its own classes for storing the data and writing it to an OIIO::ImageOutput object. \r\n\r\n\r\nThere is also a new test - ImageNodeTest.testCacheThreadSafety(), which uses GafferImageTest.traverseTiles() to call imagePlug->channelData() on each tile and channel."},"comment":{"url":"https://api.github.com/repos/ImageEngine/gaffer/issues/comments/156490202","html_url":"https://github.com/ImageEngine/gaffer/pull/1517#issuecomment-156490202","issue_url":"https://api.github.com/repos/ImageEngine/gaffer/issues/1517","id":156490202,"user":{"login":"HughMacdonald","id":625512,"avatar_url":"https://avatars.githubusercontent.com/u/625512?v=3","gravatar_id":"","url":"https://api.github.com/users/HughMacdonald","html_url":"https://github.com/HughMacdonald","followers_url":"https://api.github.com/users/HughMacdonald/followers","following_url":"https://api.github.com/users/HughMacdonald/following{/other_user}","gists_url":"https://api.github.com/users/HughMacdonald/gists{/gist_id}","starred_url":"https://api.github.com/users/HughMacdonald/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HughMacdonald/subscriptions","organizations_url":"https://api.github.com/users/HughMacdonald/orgs","repos_url":"https://api.github.com/users/HughMacdonald/repos","events_url":"https://api.github.com/users/HughMacdonald/events{/privacy}","received_events_url":"https://api.github.com/users/HughMacdonald/received_events","type":"User","site_admin":false},"created_at":"2015-11-13T17:08:18Z","updated_at":"2015-11-13T17:08:18Z","body":"It's not crashing on OSX or Linux for me...\r\n\r\nJohn significantly simplified the parallel imageHash, so please do try your end based on the latest commit in my parallelProcessImageData and see if it still happens..."}}