{"action":"created","issue":{"url":"https://api.github.com/repos/geotrellis/geotrellis/issues/1060","labels_url":"https://api.github.com/repos/geotrellis/geotrellis/issues/1060/labels{/name}","comments_url":"https://api.github.com/repos/geotrellis/geotrellis/issues/1060/comments","events_url":"https://api.github.com/repos/geotrellis/geotrellis/issues/1060/events","html_url":"https://github.com/geotrellis/geotrellis/issues/1060","id":68743578,"number":1060,"title":"Create map function that maps a (Int, Int, Int) => Int over a Tile in a way that doesn't Box.","user":{"login":"lossyrob","id":2320142,"avatar_url":"https://avatars.githubusercontent.com/u/2320142?v=3","gravatar_id":"","url":"https://api.github.com/users/lossyrob","html_url":"https://github.com/lossyrob","followers_url":"https://api.github.com/users/lossyrob/followers","following_url":"https://api.github.com/users/lossyrob/following{/other_user}","gists_url":"https://api.github.com/users/lossyrob/gists{/gist_id}","starred_url":"https://api.github.com/users/lossyrob/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lossyrob/subscriptions","organizations_url":"https://api.github.com/users/lossyrob/orgs","repos_url":"https://api.github.com/users/lossyrob/repos","events_url":"https://api.github.com/users/lossyrob/events{/privacy}","received_events_url":"https://api.github.com/users/lossyrob/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/geotrellis/geotrellis/milestones/6","html_url":"https://github.com/geotrellis/geotrellis/milestones/0.10","labels_url":"https://api.github.com/repos/geotrellis/geotrellis/milestones/6/labels","id":271794,"number":6,"title":"0.10","description":"","creator":{"login":"joshmarcus","id":241093,"avatar_url":"https://avatars.githubusercontent.com/u/241093?v=3","gravatar_id":"","url":"https://api.github.com/users/joshmarcus","html_url":"https://github.com/joshmarcus","followers_url":"https://api.github.com/users/joshmarcus/followers","following_url":"https://api.github.com/users/joshmarcus/following{/other_user}","gists_url":"https://api.github.com/users/joshmarcus/gists{/gist_id}","starred_url":"https://api.github.com/users/joshmarcus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/joshmarcus/subscriptions","organizations_url":"https://api.github.com/users/joshmarcus/orgs","repos_url":"https://api.github.com/users/joshmarcus/repos","events_url":"https://api.github.com/users/joshmarcus/events{/privacy}","received_events_url":"https://api.github.com/users/joshmarcus/received_events","type":"User","site_admin":false},"open_issues":28,"closed_issues":70,"state":"open","created_at":"2013-02-20T21:40:17Z","updated_at":"2015-04-15T19:53:50Z","due_on":null,"closed_at":null},"comments":4,"created_at":"2015-04-15T17:42:37Z","updated_at":"2015-04-15T20:32:37Z","closed_at":null,"body":"On `Tile`, we have a `map(f: Int => Int)` method that creates a new Tile by applying `f` to each cell of the Tile. This can be applied to something like `tile.map { z => z + 1 }` in a way that doesn't [box](http://docs.oracle.com/javase/1.5.0/docs/guide/language/autoboxing.html), because [Function1](https://github.com/scala/scala/blob/v2.11.6/src/library/scala/Function1.scala#L35) is [specialized](http://www.scala-notes.org/2011/04/specializing-for-primitive-types/).\r\n\r\nWhat we need is a `map(f: (Int, Int, Int) => Int)` so that we can map over a Tile in a way that allows the function that produces the new value to also take into account the column and row of the cell that's being operated on; for instance `tile.map { (col, row, z) => col + row + z }`.\r\n\r\nThe problem with this is, `Function3` is not specialized. So we get boxing and a lot of slowdown. See below for a code example, benchmark results, and bytecode inspection that illustrates this point.\r\n\r\nHow do we solve this? As we see below, we could create something like a \r\n```scala\r\ntrait Mapper { def apply(col: Int, row: Int, z: Int): Int }. \r\n```\r\nBut then our code would look like this,\r\n```scala\r\ntile.map(new Mapper { def apply(col: Int, row: Int, z: Int) = col + row + z })\r\n```\r\nUgly! Worse, we would have to name the trait differently for our `map` and `mapDouble` cases.\r\n\r\nIs there a way we can use Macros to solve this? Is there a way to define a macro so that my instances of calling this:\r\n\r\n```scala\r\ntile.map { (col, row, z) => col + row + z }\r\n```\r\nget translated to:\r\n```scala\r\ntile.map(new IntMapper { def apply(col: Int, row: Int, z: Int): Int = col + row + z })\r\n```\r\nat compile time?\r\n\r\n----\r\nHere is a more detailed explanation of why this is needed:\r\nLet's take a look at an code example to explain the consequences of this.\r\n\r\n```scala\r\ntrait IntFunc {\r\n  def apply(x: Int, y: Int, z: Int): Int\r\n}\r\n\r\ntrait IntFunc2[T] {\r\n  def apply(x: Int, y: Int, z: Int): T\r\n}\r\n\r\nclass ArrayWrapper(arr: Array[Int]) {\r\n  val size = arr.size\r\n\r\n  def function1MapToInt(f: Int => Int): Array[Int] = {\r\n    val result = arr.clone\r\n    var i = 0\r\n    while(i < size) {\r\n      val x = arr(i)\r\n      result(i) = f(x)\r\n      i += 1\r\n    }\r\n    result\r\n  }\r\n\r\n  def function1MapToT[T: ClassTag](f: Int => T): Array[T] = {\r\n    val result = Array.ofDim[T](size)\r\n    var i = 0\r\n    while(i < size) {\r\n      val x = arr(i)\r\n      result(i) = f(x)\r\n      i += 1\r\n    }\r\n    result\r\n  }\r\n\r\n  def function3MapToInt(f: (Int, Int, Int) => Int): Array[Int] = {\r\n    val result = arr.clone\r\n    var i = 0\r\n    while(i < size) {\r\n      val x = arr(i)\r\n      result(i) = f(x, x, x)\r\n      i += 1\r\n    }\r\n    result\r\n  }\r\n\r\n  def function3MapToT[T: ClassTag](f: (Int, Int, Int) => T): Array[T] = {\r\n    val result = Array.ofDim[T](size)\r\n    var i = 0\r\n    while(i < size) {\r\n      val x = arr(i)\r\n      result(i) = f(x, x, x)\r\n      i += 1\r\n    }\r\n    result\r\n  }\r\n\r\n  def traitMapToInt(f: IntFunc): Array[Int] = {\r\n    val result = arr.clone\r\n    var i = 0\r\n    while(i < size) {\r\n      val x = arr(i)\r\n      result(i) = f(x, x, x)\r\n      i += 1\r\n    }\r\n    result\r\n  }\r\n\r\n  def traitMapToT[T: ClassTag](f: IntFunc2[T]): Array[T] = {\r\n    val result = Array.ofDim[T](size)\r\n    var i = 0\r\n    while(i < size) {\r\n      val x = arr(i)\r\n      result(i) = f(x, x, x)\r\n      i += 1\r\n    }\r\n    result\r\n  }\r\n}\r\n```\r\n\r\nI wrote a [benchmark suite](https://gist.github.com/lossyrob/54a94d2b8e9ff310f99c) for testing these different variations as part of the geotrellis benchmarks, and here's a run result I get:\r\n\r\n```\r\n[info] Function1MapToInt   7.92 =\r\n[info]   Function1MapToT  73.42 =================\r\n[info] Function3MapToInt  91.50 =====================\r\n[info]   Function3MapToT 126.91 ==============================\r\n[info]     TraitMapToInt   7.68 =\r\n[info]       TraitMapToT  56.64 =============\r\n```\r\nTo get a better sense of these variations, we can take a look at the bytecode that is generated by the functions defined in our `ArrayWrapper`.\r\n\r\nFor the map that takes a `Int => Int`:\r\n```\r\n  public int[] function1MapToInt(scala.Function1<java.lang.Object, java.lang.Object>);\r\n    Code:\r\n       0: aload_0       \r\n       1: getfield      #21                 // Field arr:[I\r\n       4: invokevirtual #26                 // Method \"[I\".clone:()Ljava/lang/Object;\r\n       7: checkcast     #22                 // class \"[I\"\r\n      10: astore_2      \r\n      11: iconst_0      \r\n      12: istore_3      \r\n      13: iload_3       \r\n      14: aload_0       \r\n      15: invokevirtual #28                 // Method size:()I\r\n      18: if_icmpge     47\r\n      21: aload_0       \r\n      22: getfield      #21                 // Field arr:[I\r\n      25: iload_3       \r\n      26: iaload        \r\n      27: istore        4\r\n      29: aload_2       \r\n      30: iload_3       \r\n      31: aload_1       \r\n      32: iload         4\r\n      34: invokeinterface #34,  2           // InterfaceMethod scala/Function1.apply$mcII$sp:(I)I\r\n      39: iastore       \r\n      40: iload_3       \r\n      41: iconst_1      \r\n      42: iadd          \r\n      43: istore_3      \r\n      44: goto          13\r\n      47: aload_2       \r\n      48: areturn       \r\n```\r\nwe can see that line 34 calls the specialized method `Function1.apply$mcII$sp:(I)I`, and there is no boxing to be seen.\r\n\r\nFor the function that takes `Int => T`, \r\n```\r\n  public <T extends java/lang/Object> java.lang.Object function1MapToT(scala.Function1<java.lang.Object, T>, scala.reflect.ClassTag<T>);\r\n    Code:\r\n       0: getstatic     #47                 // Field scala/Array$.MODULE$:Lscala/Array$;\r\n       3: aload_0       \r\n       4: invokevirtual #28                 // Method size:()I\r\n       7: aload_2       \r\n       8: invokevirtual #51                 // Method scala/Array$.ofDim:(ILscala/reflect/ClassTag;)Ljava/lang/Object;\r\n      11: astore_3      \r\n      12: iconst_0      \r\n      13: istore        4\r\n      15: iload         4\r\n      17: aload_0       \r\n      18: invokevirtual #28                 // Method size:()I\r\n      21: if_icmpge     62\r\n      24: aload_0       \r\n      25: getfield      #21                 // Field arr:[I\r\n      28: iload         4\r\n      30: iaload        \r\n      31: istore        5\r\n      33: getstatic     #56                 // Field scala/runtime/ScalaRunTime$.MODULE$:Lscala/runtime/ScalaRunTime$;\r\n      36: aload_3       \r\n      37: iload         4\r\n      39: aload_1       \r\n      40: iload         5\r\n      42: invokestatic  #62                 // Method scala/runtime/BoxesRunTime.boxToInteger:(I)Ljava/lang/Integer;\r\n      45: invokeinterface #66,  2           // InterfaceMethod scala/Function1.apply:(Ljava/lang/Object;)Ljava/lang/Object;\r\n      50: invokevirtual #70                 // Method scala/runtime/ScalaRunTime$.array_update:(Ljava/lang/Object;ILjava/lang/Object;)V\r\n      53: iload         4\r\n      55: iconst_1      \r\n      56: iadd          \r\n      57: istore        4\r\n      59: goto          15\r\n      62: aload_3       \r\n      63: areturn       \r\n```\r\nmore byte code, and we can see boxing on line 42. We box the integer argument, because even though the argument is specialized on `Function1`, since the return type is a generic, it causes the version of `Function1` that takes all boxed generics to be invoked. This highlights a limitation of specialization.\r\n\r\nFor the function that takes `(Int, Int, Int) => Int`:\r\n```\r\n  public int[] function3MapToInt(scala.Function3<java.lang.Object, java.lang.Object, java.lang.Object, java.lang.Object>);\r\n    Code:\r\n       0: aload_0       \r\n       1: getfield      #21                 // Field arr:[I\r\n       4: invokevirtual #26                 // Method \"[I\".clone:()Ljava/lang/Object;\r\n       7: checkcast     #22                 // class \"[I\"\r\n      10: astore_2      \r\n      11: iconst_0      \r\n      12: istore_3      \r\n      13: iload_3       \r\n      14: aload_0       \r\n      15: invokevirtual #28                 // Method size:()I\r\n      18: if_icmpge     63\r\n      21: aload_0       \r\n      22: getfield      #21                 // Field arr:[I\r\n      25: iload_3       \r\n      26: iaload        \r\n      27: istore        4\r\n      29: aload_2       \r\n      30: iload_3       \r\n      31: aload_1       \r\n      32: iload         4\r\n      34: invokestatic  #62                 // Method scala/runtime/BoxesRunTime.boxToInteger:(I)Ljava/lang/Integer;\r\n      37: iload         4\r\n      39: invokestatic  #62                 // Method scala/runtime/BoxesRunTime.boxToInteger:(I)Ljava/lang/Integer;\r\n      42: iload         4\r\n      44: invokestatic  #62                 // Method scala/runtime/BoxesRunTime.boxToInteger:(I)Ljava/lang/Integer;\r\n      47: invokeinterface #80,  4           // InterfaceMethod scala/Function3.apply:(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;\r\n      52: invokestatic  #84                 // Method scala/runtime/BoxesRunTime.unboxToInt:(Ljava/lang/Object;)I\r\n      55: iastore       \r\n      56: iload_3       \r\n      57: iconst_1      \r\n      58: iadd          \r\n      59: istore_3      \r\n      60: goto          13\r\n      63: aload_2       \r\n      64: areturn       \r\n```\r\nHere we see a lot of boxing. Each of the 3 Int parameters are boxed before being shipped into a Function3 call, and then the resulting Int is unboxed on return. This causes some serious slowdown!\r\n\r\nHow about the function that takes `(Int, Int, Int) => T`?\r\n```\r\n  public <T extends java/lang/Object> java.lang.Object function3MapToT(scala.Function3<java.lang.Object, java.lang.Object, java.lang.Object, T>, scala.reflect.ClassTag<T>);\r\n    Code:\r\n       0: getstatic     #47                 // Field scala/Array$.MODULE$:Lscala/Array$;\r\n       3: aload_0       \r\n       4: invokevirtual #28                 // Method size:()I\r\n       7: aload_2       \r\n       8: invokevirtual #51                 // Method scala/Array$.ofDim:(ILscala/reflect/ClassTag;)Ljava/lang/Object;\r\n      11: astore_3      \r\n      12: iconst_0      \r\n      13: istore        4\r\n      15: iload         4\r\n      17: aload_0       \r\n      18: invokevirtual #28                 // Method size:()I\r\n      21: if_icmpge     72\r\n      24: aload_0       \r\n      25: getfield      #21                 // Field arr:[I\r\n      28: iload         4\r\n      30: iaload        \r\n      31: istore        5\r\n      33: getstatic     #56                 // Field scala/runtime/ScalaRunTime$.MODULE$:Lscala/runtime/ScalaRunTime$;\r\n      36: aload_3       \r\n      37: iload         4\r\n      39: aload_1       \r\n      40: iload         5\r\n      42: invokestatic  #62                 // Method scala/runtime/BoxesRunTime.boxToInteger:(I)Ljava/lang/Integer;\r\n      45: iload         5\r\n      47: invokestatic  #62                 // Method scala/runtime/BoxesRunTime.boxToInteger:(I)Ljava/lang/Integer;\r\n      50: iload         5\r\n      52: invokestatic  #62                 // Method scala/runtime/BoxesRunTime.boxToInteger:(I)Ljava/lang/Integer;\r\n      55: invokeinterface #80,  4           // InterfaceMethod scala/Function3.apply:(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;\r\n      60: invokevirtual #70                 // Method scala/runtime/ScalaRunTime$.array_update:(Ljava/lang/Object;ILjava/lang/Object;)V\r\n      63: iload         4\r\n      65: iconst_1      \r\n      66: iadd          \r\n      67: istore        4\r\n      69: goto          15\r\n      72: aload_3       \r\n      73: areturn       \r\n```\r\nHere we see expected results. The input parameters are boxed.\r\n\r\nWith the function that takes traits, we see a lot less boxing.\r\n```\r\n  public int[] traitMapToInt(geotrellis.benchmark.IntFunc);\r\n    Code:\r\n       0: aload_0       \r\n       1: getfield      #21                 // Field arr:[I\r\n       4: invokevirtual #26                 // Method \"[I\".clone:()Ljava/lang/Object;\r\n       7: checkcast     #22                 // class \"[I\"\r\n      10: astore_2      \r\n      11: iconst_0      \r\n      12: istore_3      \r\n      13: iload_3       \r\n      14: aload_0       \r\n      15: invokevirtual #28                 // Method size:()I\r\n      18: if_icmpge     51\r\n      21: aload_0       \r\n      22: getfield      #21                 // Field arr:[I\r\n      25: iload_3       \r\n      26: iaload        \r\n      27: istore        4\r\n      29: aload_2       \r\n      30: iload_3       \r\n      31: aload_1       \r\n      32: iload         4\r\n      34: iload         4\r\n      36: iload         4\r\n      38: invokeinterface #95,  4           // InterfaceMethod geotrellis/benchmark/IntFunc.apply:(III)I\r\n      43: iastore       \r\n      44: iload_3       \r\n      45: iconst_1      \r\n      46: iadd          \r\n      47: istore_3      \r\n      48: goto          13\r\n      51: aload_2       \r\n      52: areturn       \r\n```\r\nHere we see no boxing, and so it makes sense that the benchmarking result that it's the same as the specialized version of `Function1`\r\n\r\nHere is the bytecode for the call to `IntFunc2`:\r\n\r\n```\r\n  public <T extends java/lang/Object> java.lang.Object traitMapToT(geotrellis.benchmark.IntFunc2<T>, scala.reflect.ClassTag<T>);\r\n    Code:\r\n       0: getstatic     #47                 // Field scala/Array$.MODULE$:Lscala/Array$;\r\n       3: aload_0       \r\n       4: invokevirtual #28                 // Method size:()I\r\n       7: aload_2       \r\n       8: invokevirtual #51                 // Method scala/Array$.ofDim:(ILscala/reflect/ClassTag;)Ljava/lang/Object;\r\n      11: astore_3      \r\n      12: iconst_0      \r\n      13: istore        4\r\n      15: iload         4\r\n      17: aload_0       \r\n      18: invokevirtual #28                 // Method size:()I\r\n      21: if_icmpge     63\r\n      24: aload_0       \r\n      25: getfield      #21                 // Field arr:[I\r\n      28: iload         4\r\n      30: iaload        \r\n      31: istore        5\r\n      33: getstatic     #56                 // Field scala/runtime/ScalaRunTime$.MODULE$:Lscala/runtime/ScalaRunTime$;\r\n      36: aload_3       \r\n      37: iload         4\r\n      39: aload_1       \r\n      40: iload         5\r\n      42: iload         5\r\n      44: iload         5\r\n      46: invokeinterface #103,  4          // InterfaceMethod geotrellis/benchmark/IntFunc2.apply:(III)Ljava/lang/Object;\r\n      51: invokevirtual #70                 // Method scala/runtime/ScalaRunTime$.array_update:(Ljava/lang/Object;ILjava/lang/Object;)V\r\n      54: iload         4\r\n      56: iconst_1      \r\n      57: iadd          \r\n      58: istore        4\r\n      60: goto          15\r\n      63: aload_3       \r\n      64: areturn       \r\n```\r\n\r\nNo boxing! This gives explanation to why the benchmarks for this case are the fastest out of any methods returning `Option[Int]`.\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/geotrellis/geotrellis/issues/comments/93560065","html_url":"https://github.com/geotrellis/geotrellis/issues/1060#issuecomment-93560065","issue_url":"https://api.github.com/repos/geotrellis/geotrellis/issues/1060","id":93560065,"user":{"login":"lossyrob","id":2320142,"avatar_url":"https://avatars.githubusercontent.com/u/2320142?v=3","gravatar_id":"","url":"https://api.github.com/users/lossyrob","html_url":"https://github.com/lossyrob","followers_url":"https://api.github.com/users/lossyrob/followers","following_url":"https://api.github.com/users/lossyrob/following{/other_user}","gists_url":"https://api.github.com/users/lossyrob/gists{/gist_id}","starred_url":"https://api.github.com/users/lossyrob/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lossyrob/subscriptions","organizations_url":"https://api.github.com/users/lossyrob/orgs","repos_url":"https://api.github.com/users/lossyrob/repos","events_url":"https://api.github.com/users/lossyrob/events{/privacy}","received_events_url":"https://api.github.com/users/lossyrob/received_events","type":"User","site_admin":false},"created_at":"2015-04-15T20:32:37Z","updated_at":"2015-04-15T20:32:37Z","body":"That would be awesome @non, thank you!"}}