{"push_id":903167415,"size":117,"distinct_size":117,"ref":"refs/heads/master","head":"05392f9e7b2d19ee4bbbaa1434f7f5729ca43ab9","before":"2cd7816300fc14a79ca74ca566f6ac3023102aad","commits":[{"sha":"1959c04f6f29bf3ef488fd927634060b4c3d68ea","author":{"email":"6504ab7c5083d5070fa8749cea625821ca8a6c66@spotify.com","name":"Neville Li"},"message":"PARQUET-204: add parquet-schema directory support\n\nAuthor: Neville Li <neville@spotify.com>\n\nCloses #136 from nevillelyh/neville/PARQUET-204 and squashes the following commits:\n\n633829b [Neville Li] PARQUET-204: add parquet-schema directory support\n7aa8581 [Neville Li] PARQUET-203: consolidate PathFilter for hidden files","distinct":true,"url":"https://api.github.com/repos/garlicccbulb/parquet-mr/commits/1959c04f6f29bf3ef488fd927634060b4c3d68ea"},{"sha":"9a4c30d346300a7f351259dc694ca40bf3a12996","author":{"email":"3b0830a79602291f6a049708fe3ad223d893d220@intel.com","name":"Ferdinand Xu"},"message":"PARQUET-189: Support building parquet with thrift 0.9.0\n\n6cbed83f [Ferdinand Xu] PARQUET-189: Support building parquet with thrift 0.9.0","distinct":true,"url":"https://api.github.com/repos/garlicccbulb/parquet-mr/commits/9a4c30d346300a7f351259dc694ca40bf3a12996"},{"sha":"79af376995756d13025e21b159efebd9ff4e891d","author":{"email":"8baa999284405a44df6958c344ec0d51df537ce1@netflix.com","name":"Nezih Yigitbasi"},"message":"PARQUET-165: Add a new parquet-benchmark module\n\nPARQUET-165\n\nThis PR is an initial version of a new ``parquet-benchmark`` module that we can build upon. The module already contains some simple benchmarks for read/writes, we can discuss how we can make those more representative.\n\nWhen run, various statistics will be printed for all the benchmarks in this module. For example, for the read benchmarks the output will look like:\n\n```\n# Run complete. Total time: 00:03:16\n\nBenchmark                                                             Mode  Samples  Score   Error  Units\np.b.ReadBenchmarks.read1MRowsBS256MPS4MUncompressed                  thrpt        1  0.248 ±   NaN  ops/s\np.b.ReadBenchmarks.read1MRowsBS256MPS8MUncompressed                  thrpt        1  0.331 ±   NaN  ops/s\np.b.ReadBenchmarks.read1MRowsBS512MPS4MUncompressed                  thrpt        1  0.309 ±   NaN  ops/s\np.b.ReadBenchmarks.read1MRowsBS512MPS8MUncompressed                  thrpt        1  0.303 ±   NaN  ops/s\np.b.ReadBenchmarks.read1MRowsDefaultBlockAndPageSizeGZIP             thrpt        1  0.264 ±   NaN  ops/s\np.b.ReadBenchmarks.read1MRowsDefaultBlockAndPageSizeSNAPPY           thrpt        1  0.499 ±   NaN  ops/s\np.b.ReadBenchmarks.read1MRowsDefaultBlockAndPageSizeUncompressed     thrpt        1  0.360 ±   NaN  ops/s\np.b.ReadBenchmarks.read1MRowsBS256MPS4MUncompressed                   avgt        1  3.623 ±   NaN   s/op\np.b.ReadBenchmarks.read1MRowsBS256MPS8MUncompressed                   avgt        1  3.162 ±   NaN   s/op\np.b.ReadBenchmarks.read1MRowsBS512MPS4MUncompressed                   avgt        1  3.231 ±   NaN   s/op\np.b.ReadBenchmarks.read1MRowsBS512MPS8MUncompressed                   avgt        1  2.583 ±   NaN   s/op\np.b.ReadBenchmarks.read1MRowsDefaultBlockAndPageSizeGZIP              avgt        1  3.713 ±   NaN   s/op\np.b.ReadBenchmarks.read1MRowsDefaultBlockAndPageSizeSNAPPY            avgt        1  2.055 ±   NaN   s/op\np.b.ReadBenchmarks.read1MRowsDefaultBlockAndPageSizeUncompressed      avgt        1  2.904 ±   NaN   s/op\np.b.ReadBenchmarks.read1MRowsBS256MPS4MUncompressed                 sample        1  2.772 ±   NaN   s/op\np.b.ReadBenchmarks.read1MRowsBS256MPS8MUncompressed                 sample        1  2.538 ±   NaN   s/op\np.b.ReadBenchmarks.read1MRowsBS512MPS4MUncompressed                 sample        1  2.496 ±   NaN   s/op\np.b.ReadBenchmarks.read1MRowsBS512MPS8MUncompressed                 sample        1  2.416 ±   NaN   s/op\np.b.ReadBenchmarks.read1MRowsDefaultBlockAndPageSizeGZIP            sample        1  3.712 ±   NaN   s/op\np.b.ReadBenchmarks.read1MRowsDefaultBlockAndPageSizeSNAPPY          sample        1  1.772 ±   NaN   s/op\np.b.ReadBenchmarks.read1MRowsDefaultBlockAndPageSizeUncompressed    sample        1  2.819 ±   NaN   s/op\np.b.ReadBenchmarks.read1MRowsBS256MPS4MUncompressed                     ss        1  2.416 ±   NaN      s\np.b.ReadBenchmarks.read1MRowsBS256MPS8MUncompressed                     ss        1  2.564 ±   NaN      s\np.b.ReadBenchmarks.read1MRowsBS512MPS4MUncompressed                     ss        1  2.547 ±   NaN      s\np.b.ReadBenchmarks.read1MRowsBS512MPS8MUncompressed                     ss        1  3.094 ±   NaN      s\np.b.ReadBenchmarks.read1MRowsDefaultBlockAndPageSizeGZIP                ss        1  3.689 ±   NaN      s\np.b.ReadBenchmarks.read1MRowsDefaultBlockAndPageSizeSNAPPY              ss        1  1.983 ±   NaN      s\np.b.ReadBenchmarks.read1MRowsDefaultBlockAndPageSizeUncompressed        ss        1  2.928 ±   NaN      s\n\n```\n\nAuthor: Nezih Yigitbasi <nyigitbasi@netflix.com>\n\nCloses #104 from nezihyigitbasi/benchmark-module and squashes the following commits:\n\n90c72f5 [Nezih Yigitbasi] Add a new parquet-benchmark module","distinct":true,"url":"https://api.github.com/repos/garlicccbulb/parquet-mr/commits/79af376995756d13025e21b159efebd9ff4e891d"},{"sha":"a7528812b166d7cd260307dd36e1f7c6c54cd523","author":{"email":"8baa999284405a44df6958c344ec0d51df537ce1@netflix.com","name":"Nezih Yigitbasi"},"message":"PARQUET-165: Update parquet version in the benchmark module\n\nAuthor: Nezih Yigitbasi <nyigitbasi@netflix.com>\n\nCloses #155 from nezihyigitbasi/parquet-benchmarks-fix and squashes the following commits:\n\n3f43427 [Nezih Yigitbasi] Update parquet version","distinct":true,"url":"https://api.github.com/repos/garlicccbulb/parquet-mr/commits/a7528812b166d7cd260307dd36e1f7c6c54cd523"},{"sha":"a73e26290fbe0281487019c0038535c0fbadf786","author":{"email":"6504ab7c5083d5070fa8749cea625821ca8a6c66@spotify.com","name":"Neville Li"},"message":"PARQUET-210: add JSON support for parquet-cat\n\nJSON output with this patch:\n```\n{\"int_field\":99,\"long_field\":1099,\"float_field\":2099.5,\"double_field\":5099.5,\"boolean_field\":true,\"string_field\":\"str99\",\"nested\":{\"numbers\":[100,101,102,103,104],\"name\":\"name99\",\"dict\":{\"a\":100,\"b\":200,\"c\":300}}}\n```\n\nCurrent output format:\n```\nint_field = 99\nlong_field = 1099\nfloat_field = 2099.5\ndouble_field = 5099.5\nboolean_field = true\nstring_field = str99\nnested:\n.numbers:\n..array = 100\n..array = 101\n..array = 102\n..array = 103\n..array = 104\n.name = name99\n.dict:\n..map:\n...key = a\n...value = 100\n..map:\n...key = b\n...value = 200\n..map:\n...key = c\n...value = 300\n```\n\nAuthor: Neville Li <neville@spotify.com>\n\nCloses #140 from nevillelyh/neville/PARQUET-210 and squashes the following commits:\n\n45fd629 [Neville Li] PARQUET-210: add JSON support for parquet-cat","distinct":true,"url":"https://api.github.com/repos/garlicccbulb/parquet-mr/commits/a73e26290fbe0281487019c0038535c0fbadf786"},{"sha":"898542200fc96813eb4cbcc139be6552abcb6be1","author":{"email":"4c9a82ce72ca2519f38d0af0abbb4cecb9fceca9@apache.org","name":"Ryan Blue"},"message":"PARQUET-214: Fix Avro string regression.\n\nAt some point, parquet-avro converted string fields to binary without\nthe UTF8 annotation. The change in PARQUET-139 to filter the file's\nschema using the requested projection causes a regression because the\nannotation is not present in some file schemas, but is present in the\nprojection schema converted from Avro.\n\nThis reverts the projection change to avoid a regression in a release.\nFixing the projection as in PARQUET-139 will need to be done as a\nfollow-up.\n\nAuthor: Ryan Blue <blue@apache.org>\n\nCloses #142 from rdblue/PARQUET-214-fix-avro-regression and squashes the following commits:\n\n71e0207 [Ryan Blue] PARQUET-214: Add support for old avro.schema property.\n95148f9 [Ryan Blue] PARQUET-214: Revert Schema projection change from PARQUET-139.","distinct":true,"url":"https://api.github.com/repos/garlicccbulb/parquet-mr/commits/898542200fc96813eb4cbcc139be6552abcb6be1"},{"sha":"27145928b1516dbde6d9cea996afc7746d27cc63","author":{"email":"4c9a82ce72ca2519f38d0af0abbb4cecb9fceca9@apache.org","name":"Ryan Blue"},"message":"PARQUET-230: Add build instructions to README.\n\nAuthor: Ryan Blue <blue@apache.org>\n\nCloses #156 from rdblue/PARQUET-230-add-build-instructions-to-readme and squashes the following commits:\n\n896604a [Ryan Blue] PARQUET-230: Add build instructions to README.","distinct":true,"url":"https://api.github.com/repos/garlicccbulb/parquet-mr/commits/27145928b1516dbde6d9cea996afc7746d27cc63"},{"sha":"5ee0d3b50798d93d60603e20cc41807c15d00f16","author":{"email":"b98bbd1d47292cd44bd0779957153c7a22567181@sc.com","name":"Konstantin Shaposhnikov"},"message":"PARQUET-220: Remove unnecessary warnings initializing ParquetRecordReader\n\nRefactored to replace TaskInputOutputContext with TaskAttemptContext.\n\nParquetRecordReader used to check that the passed context is instance of\nTaskInputOutputContext however the functionality it uses doesn't rely on this\nfact.\n\nThis closes #152 when committed. It fixes the review feedback on that issue to include it in 1.6.0.\n\nAuthor: Konstantin Shaposhnikov <Konstantin.Shaposhnikov@sc.com>\n\nCloses #162 from rdblue/PARQUET-152-remove-counter-warning and squashes the following commits:\n\n0a7780f [Konstantin Shaposhnikov] PARQUET-220: do not log unnecessary warnings on initializing ParquetRecordReader","distinct":true,"url":"https://api.github.com/repos/garlicccbulb/parquet-mr/commits/5ee0d3b50798d93d60603e20cc41807c15d00f16"},{"sha":"d701a48962312fec5514eacdf8553861837c91bd","author":{"email":"4c9a82ce72ca2519f38d0af0abbb4cecb9fceca9@apache.org","name":"Ryan Blue"},"message":"Revert \"PARQUET-220: Remove unnecessary warnings initializing ParquetRecordReader\"\n\nThis reverts commit bfb314505469afcd5ea7b5bd15121acd50425318.","distinct":true,"url":"https://api.github.com/repos/garlicccbulb/parquet-mr/commits/d701a48962312fec5514eacdf8553861837c91bd"},{"sha":"154f3a5817aa416474f6079db336c50ac627b87a","author":{"email":"4c9a82ce72ca2519f38d0af0abbb4cecb9fceca9@apache.org","name":"Ryan Blue"},"message":"PARQUET-242: Fix AvroReadSupport.setAvroDataSupplier.\n\nThis should use the supplier class's name, rather than its toString\nrepresentation or else loading the class doesn't work.\n\nAuthor: Ryan Blue <blue@apache.org>\n\nCloses #161 from rdblue/PARQUET-242-fix-avro-data-supplier and squashes the following commits:\n\nff5b7f8 [Ryan Blue] PARQUET-242: Add Avro data supplier test.\n87a488b [Ryan Blue] PARQUET-242: Fix AvroReadSupport.setAvroDataSupplier.","distinct":true,"url":"https://api.github.com/repos/garlicccbulb/parquet-mr/commits/154f3a5817aa416474f6079db336c50ac627b87a"},{"sha":"cdaf22b2fd53410b7e738c5a31eec840e42bb77d","author":{"email":"4c9a82ce72ca2519f38d0af0abbb4cecb9fceca9@apache.org","name":"Ryan Blue"},"message":"PARQUET-234: Add ParquetInputSplit methods for compatibility.\n\nAuthor: Ryan Blue <blue@apache.org>\n\nCloses #159 from rdblue/PARQUET-234 and squashes the following commits:\n\nb09d34d [Ryan Blue] PARQUET-234: Add ParquetInputSplit methods for compatibility.","distinct":true,"url":"https://api.github.com/repos/garlicccbulb/parquet-mr/commits/cdaf22b2fd53410b7e738c5a31eec840e42bb77d"},{"sha":"f8a8c3f463b95c513ec8df8ec97660ccf08303b1","author":{"email":"4c9a82ce72ca2519f38d0af0abbb4cecb9fceca9@apache.org","name":"Ryan Blue"},"message":"PARQUET-235: Fix parquet.metadata compatibility.\n\nColumnPath and Canonicalizer were moved from parquet-hadoop to\nparquet-common in parquet.common.{internal,schema}, which broke\ncompatibility and would require bumping the major version. This moves\nthe classes back into parquet.hadoop.metadata and adds temporary\nexclusions for the move between modules. There are no breaking changes\nto the classes themselves, verified by copying them into parquet-hadoop\nand building.\n\nThis also changes the previous version back to 1.5.0 rather than an RC\n(which carries no compatibility guarantees, though this is compatible\nwith both version). It also adds an exclusions for a false positive in\nBinary.\n\nAuthor: Ryan Blue <blue@apache.org>\n\nCloses #166 from rdblue/PARQUET-235-fix-parquet-metadata and squashes the following commits:\n\nf56a57e [Ryan Blue] PARQUET-235: Fix parquet.metadata compatibility.","distinct":true,"url":"https://api.github.com/repos/garlicccbulb/parquet-mr/commits/f8a8c3f463b95c513ec8df8ec97660ccf08303b1"},{"sha":"f6ee5fea550badcac4c6df18da5406e94e27b1b5","author":{"email":"4c9a82ce72ca2519f38d0af0abbb4cecb9fceca9@apache.org","name":"Ryan Blue"},"message":"PARQUET-239: Make AvroParquetReader#builder static.\n\nFixes new API method added since 1.5.0.\n\nAuthor: Ryan Blue <blue@apache.org>\n\nCloses #158 from rdblue/PARQUET-239-fix-avro-builder and squashes the following commits:\n\nc8c64d7 [Ryan Blue] PARQUET-239: Make AvroParquetReader#builder static.","distinct":true,"url":"https://api.github.com/repos/garlicccbulb/parquet-mr/commits/f6ee5fea550badcac4c6df18da5406e94e27b1b5"},{"sha":"1d747cfa68a3f2b6c2553ef24c846ba9e914ad25","author":{"email":"4c9a82ce72ca2519f38d0af0abbb4cecb9fceca9@apache.org","name":"Ryan Blue"},"message":"PARQUET-211: 1.6.0 release changes\n\nAuthor: Ryan Blue <blue@apache.org>\n\nCloses #168 from rdblue/PARQUET-211-release-changes and squashes the following commits:\n\nd00aed7 [Ryan Blue] PARQUET-211: Add script for source release artifacts.\nd809d02 [Ryan Blue] PARQUET-211: Add release notes from JIRA 1.6.0.","distinct":true,"url":"https://api.github.com/repos/garlicccbulb/parquet-mr/commits/1d747cfa68a3f2b6c2553ef24c846ba9e914ad25"},{"sha":"f78fdf0cf420a2daeaef36607561cc27704ff4b8","author":{"email":"4c9a82ce72ca2519f38d0af0abbb4cecb9fceca9@apache.org","name":"Ryan Blue"},"message":"PARQUET-211: Set version to 1.6.0 for release.","distinct":true,"url":"https://api.github.com/repos/garlicccbulb/parquet-mr/commits/f78fdf0cf420a2daeaef36607561cc27704ff4b8"},{"sha":"bcb3b81addb7e6a0da5a62d9caf36690115185bb","author":{"email":"4c9a82ce72ca2519f38d0af0abbb4cecb9fceca9@apache.org","name":"Ryan Blue"},"message":"PARQUET-211: Set version for 1.7.0-incubating development.\n\nAuthor: Ryan Blue <blue@apache.org>\n\nCloses #176 from rdblue/PARQUET-211-set-dev-version and squashes the following commits:\n\n90e8fbd [Ryan Blue] PARQUET-211: Set version for 1.7.0-incubating development.","distinct":true,"url":"https://api.github.com/repos/garlicccbulb/parquet-mr/commits/bcb3b81addb7e6a0da5a62d9caf36690115185bb"},{"sha":"5f6a4565bcb940c7b28e07b6428bb2d60ef9a906","author":{"email":"f8a0b68fd75e68067624c0acb343bfb9a5ed4026@twitter.com","name":"Tianshuo Deng"},"message":"PARQUET-252 : Support nests container types for scrooge support\n\nparquet should support nested container type for scrooge, like list<list> or list<map>\n\nAuthor: Tianshuo Deng <tdeng@twitter.com>\n\nCloses #175 from tsdeng/support_nests_container_types_for_scrooge_support and squashes the following commits:\n\nbae3e68 [Tianshuo Deng] move set list and map inner elem name conversion to private static methods\n48b4342 [Tianshuo Deng] catch ClassCastException\nfc25bd0 [Tianshuo Deng] remove hack/fix for nested name\n429c61c [Tianshuo Deng] fix exception handling, use explicit imports\nf54e648 [Tianshuo Deng] comments\n815ee29 [Tianshuo Deng] support nested scrooge type","distinct":true,"url":"https://api.github.com/repos/garlicccbulb/parquet-mr/commits/5f6a4565bcb940c7b28e07b6428bb2d60ef9a906"},{"sha":"b73da413dbbf547ed7b07205ef624cab2d13fd60","author":{"email":"f8a0b68fd75e68067624c0acb343bfb9a5ed4026@twitter.com","name":"Tianshuo Deng"},"message":"Revert \"PARQUET-252 : Support nests container types for scrooge support\"\n\nThis reverts commit f28aa71041181867a720134e26b64b03cbccb6ec.","distinct":true,"url":"https://api.github.com/repos/garlicccbulb/parquet-mr/commits/b73da413dbbf547ed7b07205ef624cab2d13fd60"},{"sha":"628ae0d26f7f383a286bad5d49bba868a4a7cb8b","author":{"email":"4c9a82ce72ca2519f38d0af0abbb4cecb9fceca9@apache.org","name":"Ryan Blue"},"message":"PARQUET-23: Rename to org.apache.parquet.\n\nThis includes all of the code updates by module (so that we can use github for the review). I think this is currently all of the changes needed.\n\nAuthor: Ryan Blue <blue@apache.org>\n\nCloses #179 from rdblue/PARQUET-23-rename-to-org-apache and squashes the following commits:\n\n5018c94 [Ryan Blue] PARQUET-23: Change artifacts to org.apache.parquet group id.\n487b044 [Ryan Blue] PARQUET-23: POM fixes for move to org.apache.parquet.\nfe76459 [Ryan Blue] PARQUET-23: Move parquet-tools to org.apache.parquet.\n431bc6e [Ryan Blue] PARQUET-23: Move parquet-hive to org.apache.parquet.\n382db6f [Ryan Blue] PARQUET-23: Move parquet-scrooge to org.apache.parquet.\nf76b052 [Ryan Blue] PARQUET-23: Move parquet-scala to org.apache.parquet.\nc557515 [Ryan Blue] PARQUET-23: Move parquet-protobuf to org.apache.parquet.\n2a82366 [Ryan Blue] PARQUET-23: Move parquet-cascading to org.apache.parquet.\nf33db35 [Ryan Blue] PARQUET-23: Move parquet-thrift to org.apache.parquet.\n02399d2 [Ryan Blue] PARQUET-23: Move parquet-pig to org.apache.parquet.\ne0bb9ad [Ryan Blue] PARQUET-23: Move parquet-benchmarks to org.apache.parquet.\n55fb0c4 [Ryan Blue] PARQUET-23: Move parquet-avro to org.apache.parquet.\n8b7318f [Ryan Blue] PARQUET-23: Move parquet-hadoop to org.apache.parquet.\n666b3d7 [Ryan Blue] PARQUET-23: Move parquet-column to org.apache.parquet.\n3aaf772 [Ryan Blue] PARQUET-23: Move parquet-encoding to org.apache.parquet.\n9f6df3e [Ryan Blue] PARQUET-23: Move parquet-generator to org.apache.parquet.\ne34edab [Ryan Blue] PARQUET-23: Move parquet-common to org.apache.parquet.\n3ae3f7e [Ryan Blue] PARQUET-23: Use org.apache.parquet:parquet-format.","distinct":true,"url":"https://api.github.com/repos/garlicccbulb/parquet-mr/commits/628ae0d26f7f383a286bad5d49bba868a4a7cb8b"},{"sha":"44a0be8c7e127ab373e42661e207b18364759bb9","author":{"email":"4c9a82ce72ca2519f38d0af0abbb4cecb9fceca9@apache.org","name":"Ryan Blue"},"message":"PARQUET-211: Update version for 1.8.0 development.\n\nAuthor: Ryan Blue <blue@apache.org>\n\nCloses #182 from rdblue/PARQUET-211-update-to-1.8.0 and squashes the following commits:\n\n139e682 [Ryan Blue] PARQUET-211: Update version for 1.8.0 development.","distinct":true,"url":"https://api.github.com/repos/garlicccbulb/parquet-mr/commits/44a0be8c7e127ab373e42661e207b18364759bb9"}]}