{"action":"created","issue":{"url":"https://api.github.com/repos/openmicroscopy/bioformats/issues/1991","labels_url":"https://api.github.com/repos/openmicroscopy/bioformats/issues/1991/labels{/name}","comments_url":"https://api.github.com/repos/openmicroscopy/bioformats/issues/1991/comments","events_url":"https://api.github.com/repos/openmicroscopy/bioformats/issues/1991/events","html_url":"https://github.com/openmicroscopy/bioformats/pull/1991","id":108343293,"number":1991,"title":"cpp: bioformats: Add n-dimensional array lookup calculations","user":{"login":"rleigh-dundee","id":1528953,"avatar_url":"https://avatars.githubusercontent.com/u/1528953?v=3","gravatar_id":"","url":"https://api.github.com/users/rleigh-dundee","html_url":"https://github.com/rleigh-dundee","followers_url":"https://api.github.com/users/rleigh-dundee/followers","following_url":"https://api.github.com/users/rleigh-dundee/following{/other_user}","gists_url":"https://api.github.com/users/rleigh-dundee/gists{/gist_id}","starred_url":"https://api.github.com/users/rleigh-dundee/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rleigh-dundee/subscriptions","organizations_url":"https://api.github.com/users/rleigh-dundee/orgs","repos_url":"https://api.github.com/users/rleigh-dundee/repos","events_url":"https://api.github.com/users/rleigh-dundee/events{/privacy}","received_events_url":"https://api.github.com/users/rleigh-dundee/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/openmicroscopy/bioformats/labels/breaking","name":"breaking","color":"ffaacc"},{"url":"https://api.github.com/repos/openmicroscopy/bioformats/labels/develop","name":"develop","color":"02e10c"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":8,"created_at":"2015-09-25T15:04:26Z","updated_at":"2015-10-14T14:55:15Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/openmicroscopy/bioformats/pulls/1991","html_url":"https://github.com/openmicroscopy/bioformats/pull/1991","diff_url":"https://github.com/openmicroscopy/bioformats/pull/1991.diff","patch_url":"https://github.com/openmicroscopy/bioformats/pull/1991.patch"},"body":"This adds a DimensionSet class containing a list of Dimension objects.  It may be used to calculate offsets in arbitrary n-dimentional spaces:\r\n\r\n- calculate logical offsets (logical index ↔ logical coordinate)\r\n- calculate storage offsets (logical coordinate ↔ storage index)\r\n- separate logical dimension order from storage order and direction\r\n- optionally work on a relative window into an n-dimensional space (subrange)\r\n\r\nThis implements the basic logic used in Boost.MultiArray.  However, the number of dimensions is not fixed at compile-time, so this can handle calculations with an arbitrary number of dimensions.  The intent is that this will allow the replacement of the PixelBuffer class (which uses Boost.MultiArray) with a custom n-dimensional array type using this class as a basis for the index calculations.  This will make the PixelBuffer class handle arbitrary dimensions, and will also simplify the interface, making it more usable (it currently exposes the guts of the Boost.MultiArray implementation).  This also allows for this to be re-implemented in Java fairly trivially, so Java could potentially use an nDim pixel buffer in place of raw byte arrays at some future point, making the two implementations the same in this respect; the Boost.MultiArray logic is C++-specific and will not port to Java due to the heavy template usage.\r\n\r\nIn addition to PixelBuffer, this may also be used to replace the hardcoded getIndex/getCZTCoords methods and functions; this can be used directly in CoreMetadata to supplement/replace sizes and (eventually) modulo since it has an extent list of all dimensions.  This can be used to implement getIndex/getCZTCoords or it could be given directly to the user for their use, obsoleting these methods entirely.  It can also be used in any reader/writer needing to do format-specific calculations relating to dimensions.\r\n\r\nEverything is documented, and the internal logic is extensively commented.\r\n\r\nCI testing: Unit tests are provided for most of the functionality; the jobs should remain green\r\n\r\nReview:\r\n- Run `./bf-test ./cpp/test/ome-bioformats/dimension` and pipe to less or a file for review.\r\n- For each test you'll see something like this:\r\n\r\n```\r\nDimensionSet with storage order:\r\nLogical dimensions:      (X,Y)\r\nLogical extents:         (3,4)\r\nLogical ranges:          ([0,3),[0,4))\r\nLogical sizes:           (3,4)\r\nLogical strides:         (1,3)\r\nSize:                    2\r\nElements:                12\r\nStorage dimensions:      (X,Y)\r\nStorage extents:         (+3,+4)\r\nBase:                    0\r\nStrides:                 (1,3)\r\nDescending offsets:      (0,0)\r\n\r\nDimensionSet storage layout:\r\n\r\n    X\r\n    ┏━━┯━━┯━━┓\r\n    ┃ 0│ 1│ 2┃\r\nY ┏━╋━━┿━━┿━━┫\r\n  ┃0┃ 0│ 1│ 2┃\r\n  ┠─╂──┼──┼──┨\r\n  ┃1┃ 3│ 4│ 5┃\r\n  ┠─╂──┼──┼──┨\r\n  ┃2┃ 6│ 7│ 8┃\r\n  ┠─╂──┼──┼──┨\r\n  ┃3┃ 9│10│11┃\r\n  ┗━┻━━┷━━┷━━┛\r\n\r\n0,0 →  0 → 0,0\r\n1,0 →  1 → 1,0\r\n2,0 →  2 → 2,0\r\n0,1 →  3 → 0,1\r\n1,1 →  4 → 1,1\r\n2,1 →  5 → 2,1\r\n0,2 →  6 → 0,2\r\n1,2 →  7 → 1,2\r\n2,2 →  8 → 2,2\r\n0,3 →  9 → 0,3\r\n1,3 → 10 → 1,3\r\n2,3 → 11 → 2,3\r\n```\r\n\r\n- The first section is a description of the `DimensionSet`.  The first part is the user-specified details such as dimension names and sizes, and subranges.  The rest are the computed strides and offsets to handle the mapping from logical to storage coordinates.\r\n- The second section is is a visual representation of the mapping.  The row and column numbers are the logical coordinates; when there are more than two dimensions, higher dimensions are shown as multiple planes.  The grid values are the storage indices corresponding to these logical coordinates.\r\n- The last section is a linear list of the mappings, including round-trip back to verify the logic works both ways; this is checked by the unit tests so doesn't require manual validation.\r\n\r\nWhat to verify:\r\n- The logic in the `logical_index` and `storage_index` methods; also the `subrange` methods and stride/offset methods.\r\n- Could any of the performance-critical methods be improved; this is essentially `logical_index` and `storage_index`; the others such as subranges and stride/offset calculations only happen once so are not as critical.  The critical ones and their dependent bits should all be inlined.\r\n- That the storage indices are valid.  While this is easy for the simple 1D case (it's a 1:1 mapping) we also:\r\n  - use multiple dimensions\r\n  - vary dimension direction which will use negative strides and nonzero offsets\r\n  - use subranges to provide a window into the nD array; these can be verified by adding the begin value from the range to the logical index, and looking it up in the original layout (the tests display the original and subrange layouts for this purpose)--the number of combinations here is very large; verifying a small selection should be sufficient to verify the correctness of the algorithm"},"comment":{"url":"https://api.github.com/repos/openmicroscopy/bioformats/issues/comments/148075451","html_url":"https://github.com/openmicroscopy/bioformats/pull/1991#issuecomment-148075451","issue_url":"https://api.github.com/repos/openmicroscopy/bioformats/issues/1991","id":148075451,"user":{"login":"mtbc","id":2630707,"avatar_url":"https://avatars.githubusercontent.com/u/2630707?v=3","gravatar_id":"","url":"https://api.github.com/users/mtbc","html_url":"https://github.com/mtbc","followers_url":"https://api.github.com/users/mtbc/followers","following_url":"https://api.github.com/users/mtbc/following{/other_user}","gists_url":"https://api.github.com/users/mtbc/gists{/gist_id}","starred_url":"https://api.github.com/users/mtbc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mtbc/subscriptions","organizations_url":"https://api.github.com/users/mtbc/orgs","repos_url":"https://api.github.com/users/mtbc/repos","events_url":"https://api.github.com/users/mtbc/events{/privacy}","received_events_url":"https://api.github.com/users/mtbc/received_events","type":"User","site_admin":false},"created_at":"2015-10-14T14:55:15Z","updated_at":"2015-10-14T14:55:15Z","body":"It will be interesting to see all this work feed back into Java Bio-Formats someday!"}}