{"action":"created","issue":{"url":"https://api.github.com/repos/ImageEngine/gaffer/issues/1517","labels_url":"https://api.github.com/repos/ImageEngine/gaffer/issues/1517/labels{/name}","comments_url":"https://api.github.com/repos/ImageEngine/gaffer/issues/1517/comments","events_url":"https://api.github.com/repos/ImageEngine/gaffer/issues/1517/events","html_url":"https://github.com/ImageEngine/gaffer/pull/1517","id":112821746,"number":1517,"title":"Parallel process image data","user":{"login":"HughMacdonald","id":625512,"avatar_url":"https://avatars.githubusercontent.com/u/625512?v=3","gravatar_id":"","url":"https://api.github.com/users/HughMacdonald","html_url":"https://github.com/HughMacdonald","followers_url":"https://api.github.com/users/HughMacdonald/followers","following_url":"https://api.github.com/users/HughMacdonald/following{/other_user}","gists_url":"https://api.github.com/users/HughMacdonald/gists{/gist_id}","starred_url":"https://api.github.com/users/HughMacdonald/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HughMacdonald/subscriptions","organizations_url":"https://api.github.com/users/HughMacdonald/orgs","repos_url":"https://api.github.com/users/HughMacdonald/repos","events_url":"https://api.github.com/users/HughMacdonald/events{/privacy}","received_events_url":"https://api.github.com/users/HughMacdonald/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-10-22T14:56:47Z","updated_at":"2015-10-23T14:28:06Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/ImageEngine/gaffer/pulls/1517","html_url":"https://github.com/ImageEngine/gaffer/pull/1517","diff_url":"https://github.com/ImageEngine/gaffer/pull/1517.diff","patch_url":"https://github.com/ImageEngine/gaffer/pull/1517.patch"},"body":"This PR adds ImageAlgo methods for parallel processing of tiles/channels, and re-implements both ImagePlug::image() and ImageWriter to use this.\r\n\r\nImageAlgo has 4 new methods:\r\n\r\n* parallelProcessTiles\r\n  * Two versions\r\n    * One takes an imagePlug, a functor and an optional window to process (default will be imagePlug->dataWindowPlug()\r\n    * One takes the same parameters as above, plus a vector<string> of channel names, and will call the functor once per tile per channel\r\n  * This will call the defined functor once per tile (per channel)\r\n* parallelGatherTiles\r\n  * Same two versions as above\r\n  * Also takes:\r\n    * Gather functor, which will be called for the same tiles/channels, but in series.\r\n    * tileOrder, which defines the order that the gather functor is called in. One of:\r\n      * Unordered : No guarantee of ordering. Will tend towards bottom to top.\r\n      * BottomToTop : Guaranteed iteration through channels, tiles in X, tiles in Y, from the bottom\r\n      * TopToBottom : Same as BottomToTop, but from the top in Y\r\n\r\nImagePlug::image(), which previously dealt with its own parallelising, now uses parallelProcessTiles() to set off parallel processing of channel data to create an ImagePrimitive.\r\nThe parallelising that it used to do had a bug where it would often end up processing the same tiles multiple times, due to a misunderstanding of how tbb's blocked_range dealt with splitting up ranges. It would sometimes pass through ranges that were smaller than `grainsize`. As such, it now does all of the splitting using IDs, and then these are turned back into coordinates internally.\r\n\r\nImageWriter used to call ImagePlug::image() to get a full image before writing it. It now has its own classes for storing the data and writing it to an OIIO::ImageOutput object. \r\n\r\n\r\nThere is also a new test - ImageNodeTest.testCacheThreadSafety(), which uses GafferImageTest.traverseTiles() to call imagePlug->channelData() on each tile and channel."},"comment":{"url":"https://api.github.com/repos/ImageEngine/gaffer/issues/comments/150589863","html_url":"https://github.com/ImageEngine/gaffer/pull/1517#issuecomment-150589863","issue_url":"https://api.github.com/repos/ImageEngine/gaffer/issues/1517","id":150589863,"user":{"login":"HughMacdonald","id":625512,"avatar_url":"https://avatars.githubusercontent.com/u/625512?v=3","gravatar_id":"","url":"https://api.github.com/users/HughMacdonald","html_url":"https://github.com/HughMacdonald","followers_url":"https://api.github.com/users/HughMacdonald/followers","following_url":"https://api.github.com/users/HughMacdonald/following{/other_user}","gists_url":"https://api.github.com/users/HughMacdonald/gists{/gist_id}","starred_url":"https://api.github.com/users/HughMacdonald/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HughMacdonald/subscriptions","organizations_url":"https://api.github.com/users/HughMacdonald/orgs","repos_url":"https://api.github.com/users/HughMacdonald/repos","events_url":"https://api.github.com/users/HughMacdonald/events{/privacy}","received_events_url":"https://api.github.com/users/HughMacdonald/received_events","type":"User","site_admin":false},"created_at":"2015-10-23T14:28:06Z","updated_at":"2015-10-23T14:28:06Z","body":"I fixed the crash. After finding out what it was, I'm somewhat shocked that it wasn't crashing on OSX!\r\n\r\nI'm just working on ImagePlug::imageHash() at the moment. Would I be right in saying that it does make a difference in which order hashes are appended?"}}