{"push_id":747914915,"size":37,"distinct_size":0,"ref":"refs/heads/dev","head":"f5d072ae6c8186b58e38f73f9c906a715cbf3645","before":"6a82fba75f39f53b58c52ed3971d90a9bce7bd37","commits":[{"sha":"62dec0e2b75bc009d31b5540b7e820a2e1466529","author":{"email":"d7ad1f72016fc54d8b431aad856286e0cbe5a972@googlemail.com","name":"Adam Retter"},"message":"RemoveEmptyValueCompactionFilter - A compaction filter which removes entries which have an empty value","distinct":false,"url":"https://api.github.com/repos/norton/rocksdb/commits/62dec0e2b75bc009d31b5540b7e820a2e1466529"},{"sha":"3d00271e40ca909597559837b62e59cc385053be","author":{"email":"d7ad1f72016fc54d8b431aad856286e0cbe5a972@googlemail.com","name":"Adam Retter"},"message":"The ability to specify a compaction filter via the Java API","distinct":false,"url":"https://api.github.com/repos/norton/rocksdb/commits/3d00271e40ca909597559837b62e59cc385053be"},{"sha":"91bf1b80ef7d3a4f719e3f815c2d8c202855ebee","author":{"email":"d7ad1f72016fc54d8b431aad856286e0cbe5a972@googlemail.com","name":"Adam Retter"},"message":"Java facility to use the RemoveEmptyValueCompactionFilter","distinct":false,"url":"https://api.github.com/repos/norton/rocksdb/commits/91bf1b80ef7d3a4f719e3f815c2d8c202855ebee"},{"sha":"1bdfcef7bf37b4dfc171e0588bb2d0fa5385a8d0","author":{"email":"ed1215c0fddcf745750ccaf03b798589cd8f3fbe@dev854.prn2.facebook.com","name":"Poornima Chozhiyath Raman"},"message":"Fix when output level is 0 of universal compaction with trivial move\n\nSummary: Fix for universal compaction with trivial move, when the ouput level is 0. The tests where failing. Fixed by allowing normal compaction when output level is 0.\n\nTest Plan: modified test cases run successfully.\n\nReviewers: sdong, yhchiang, IslamAbdelRahman\n\nReviewed By: IslamAbdelRahman\n\nSubscribers: anthony, kradhakrishnan, leveldb, dhruba\n\nDifferential Revision: https://reviews.facebook.net/D42933","distinct":false,"url":"https://api.github.com/repos/norton/rocksdb/commits/1bdfcef7bf37b4dfc171e0588bb2d0fa5385a8d0"},{"sha":"221a94a5fde7df47692e8277ca2ff19b269edb26","author":{"email":"d7ad1f72016fc54d8b431aad856286e0cbe5a972@googlemail.com","name":"Adam Retter"},"message":"Another attempt at adding the Java API and tests to the travis build\n\nSummary: Adds the Java build and tests to Travis\n\nTest Plan: Make sure that Travis still runs (does currently)\n\nReviewers: igor, fyrz, sdong\n\nSubscribers: dhruba\n\nDifferential Revision: https://reviews.facebook.net/D43173","distinct":false,"url":"https://api.github.com/repos/norton/rocksdb/commits/221a94a5fde7df47692e8277ca2ff19b269edb26"},{"sha":"e95c59cd2f0eb0ac4885dc46e7529193d430a08d","author":{"email":"1ecdc35cf7b88536a46b917735c0b197cc73668f@fb.com","name":"Andres Notzli"},"message":"Count number of corrupt keys during compaction\n\nSummary:\nFor task #7771355, we would like to log the number of corrupt keys\nduring a compaction. This patch implements and tests the count\nas part of CompactionJobStats.\n\nTest Plan: make && make check\n\nReviewers: rven, igor, yhchiang, sdong\n\nReviewed By: sdong\n\nSubscribers: dhruba, leveldb\n\nDifferential Revision: https://reviews.facebook.net/D42921","distinct":false,"url":"https://api.github.com/repos/norton/rocksdb/commits/e95c59cd2f0eb0ac4885dc46e7529193d430a08d"},{"sha":"d06c82e477c7c1337d9cca8780e9e84c2893137f","author":{"email":"1ecdc35cf7b88536a46b917735c0b197cc73668f@fb.com","name":"Andres Notzli"},"message":"Further cleanup of CompactionJob and MergeHelper\n\nSummary:\nSimplified logic in CompactionJob and removed unused parameter in\nMergeHelper.\n\nTest Plan: make && make check\n\nReviewers: rven, igor, sdong, yhchiang\n\nReviewed By: sdong\n\nSubscribers: aekmekji, dhruba, leveldb\n\nDifferential Revision: https://reviews.facebook.net/D42687","distinct":false,"url":"https://api.github.com/repos/norton/rocksdb/commits/d06c82e477c7c1337d9cca8780e9e84c2893137f"},{"sha":"7bfae3a7237c293b84f85437b2f3c2a3143d5116","author":{"email":"485e2df9581649b6c8f873e000ad8bee4290c1b7@fb.com","name":"sdong"},"message":"tools/db_crashtest2.py should run on the same DB\n\nSummary:\nCrash tests are supposed to restart the same DB after crashing, but it is now opening a different DB. Fix it.\nIt's probably a leftover of https://reviews.facebook.net/D17073\n\nTest Plan: Run the test and make sure the same Db is opened.\n\nReviewers: kradhakrishnan, rven, igor, IslamAbdelRahman, yhchiang, anthony\n\nReviewed By: anthony\n\nSubscribers: leveldb, dhruba\n\nDifferential Revision: https://reviews.facebook.net/D43197","distinct":false,"url":"https://api.github.com/repos/norton/rocksdb/commits/7bfae3a7237c293b84f85437b2f3c2a3143d5116"},{"sha":"8161bdb5a0da1d65a1b5d8ac8226e1386003140f","author":{"email":"56f22f2f61c0e237aec3a15d31aad03bddd0f6a6@fb.com","name":"agiardullo"},"message":"WriteBatch Save Points\n\nSummary:\nSupport RollbackToSavePoint() in WriteBatch and WriteBatchWithIndex.  Support for partial transaction rollback is needed for MyRocks.\n\nAn alternate implementation of Transaction::RollbackToSavePoint() exists in D40869.  However, the other implementation is messier because it is implemented outside of WriteBatch.  This implementation is much cleaner and also exposes a potentially useful feature to WriteBatch.\n\nTest Plan: Added unit tests\n\nReviewers: IslamAbdelRahman, kradhakrishnan, maykov, yoshinorim, hermanlee4, spetrunia, sdong, yhchiang\n\nReviewed By: yhchiang\n\nSubscribers: dhruba, leveldb\n\nDifferential Revision: https://reviews.facebook.net/D42723","distinct":false,"url":"https://api.github.com/repos/norton/rocksdb/commits/8161bdb5a0da1d65a1b5d8ac8226e1386003140f"},{"sha":"4be6d44167377252f1c61928a27a258360a565f6","author":{"email":"615fd9e41b83a77c22e99dba07c2de55712cbaeb@devvm867.prn1.facebook.com","name":"Boyang Zhang"},"message":"Compression sizes option for sst_dump_tool\n\nSummary:\nAdded a new feature to sst_dump_tool.cc to allow a user to see the sizes of the different compression algorithms on an .sst file.\n\nUsage:\n./sst_dump --file=<filename> --show_compression_sizes\n./sst_dump --file=<filename> --show_compression_sizes --set_block_size=<block_size>\n\nNote: If you do not set a block size, it will default to 16kb\n\nTest Plan: manual test and the write a unit test\n\nReviewers: IslamAbdelRahman, anthony, yhchiang, rven, kradhakrishnan, sdong\n\nReviewed By: sdong\n\nSubscribers: dhruba\n\nDifferential Revision: https://reviews.facebook.net/D42963","distinct":false,"url":"https://api.github.com/repos/norton/rocksdb/commits/4be6d44167377252f1c61928a27a258360a565f6"},{"sha":"05d4265a28d98c7ca1f2c34ba64747050cfc09db","author":{"email":"615fd9e41b83a77c22e99dba07c2de55712cbaeb@devvm867.prn1.facebook.com","name":"Boyang Zhang"},"message":"Merge branch 'master' of github.com:facebook/rocksdb","distinct":false,"url":"https://api.github.com/repos/norton/rocksdb/commits/05d4265a28d98c7ca1f2c34ba64747050cfc09db"},{"sha":"193dc977e758a1378396ab1f73e69ddde0e9210e","author":{"email":"1ecdc35cf7b88536a46b917735c0b197cc73668f@fb.com","name":"Andres Notzli"},"message":"Fixing dead code in table_properties_collector_test\n\nSummary:\nThere was a bug in table_properties_collector_test that this patch\nis fixing: `!backward_mode && !test_int_tbl_prop_collector` in\nTestCustomizedTablePropertiesCollector was never true, so the code\nin the if-block never got executed. The reason is that the\nCustomizedTablePropertiesCollector test was skipping tests with\n`!backward_mode_ && !encode_as_internal`. The reason for skipping\nthe tests is unknown.\n\nTest Plan: make table_properties_collector_test && ./table_properties_collector_test\n\nReviewers: rven, igor, yhchiang, anthony, sdong\n\nReviewed By: sdong\n\nSubscribers: dhruba, leveldb\n\nDifferential Revision: https://reviews.facebook.net/D43281","distinct":false,"url":"https://api.github.com/repos/norton/rocksdb/commits/193dc977e758a1378396ab1f73e69ddde0e9210e"},{"sha":"544be638ab52adad3cc83607c83758b1ed090da2","author":{"email":"a27884a205c28e99df76dbb0d834084a9a704cba@gmail.com","name":"Andres Noetzli"},"message":"Fixing fprintf of non string literal\n\nSummary:\nsst_dump_tool contains two instances of `fprintf`s where the `format` argument is not\na string literal. This prevents the code from compiling with some compilers/compiler\noptions because of the potential security risks associated with printing non-literals.\n\nTest Plan: make all\n\nReviewers: rven, igor, yhchiang, sdong, anthony\n\nReviewed By: anthony\n\nSubscribers: dhruba, leveldb\n\nDifferential Revision: https://reviews.facebook.net/D43305","distinct":false,"url":"https://api.github.com/repos/norton/rocksdb/commits/544be638ab52adad3cc83607c83758b1ed090da2"},{"sha":"bd852bf118586f556dee660a6ce7ce80715fe4e8","author":{"email":"a27884a205c28e99df76dbb0d834084a9a704cba@gmail.com","name":"Andres Noetzli"},"message":"Fixed typos in db_stress\n\nSummary: Fixed typos.\n\nTest Plan: None\n\nReviewers: igor, yhchiang, sdong, anthony, rven\n\nReviewed By: rven\n\nSubscribers: dhruba, leveldb\n\nDifferential Revision: https://reviews.facebook.net/D43365","distinct":false,"url":"https://api.github.com/repos/norton/rocksdb/commits/bd852bf118586f556dee660a6ce7ce80715fe4e8"},{"sha":"47316c2d084a11790f50a1f5a88b306c2928e3c5","author":{"email":"485e2df9581649b6c8f873e000ad8bee4290c1b7@fb.com","name":"sdong"},"message":"dump_manifest supports DB with more number of levels\n\nSummary: Now ldb dump_manifest refuses to work if there are 20 levels. Extend the limit to 64.\n\nTest Plan: Run the tool with 20 number of levels\n\nReviewers: kradhakrishnan, anthony, IslamAbdelRahman, yhchiang\n\nReviewed By: yhchiang\n\nSubscribers: leveldb, dhruba\n\nDifferential Revision: https://reviews.facebook.net/D42879","distinct":false,"url":"https://api.github.com/repos/norton/rocksdb/commits/47316c2d084a11790f50a1f5a88b306c2928e3c5"},{"sha":"40c64434d41824a7e149a8a47ea2c03838fd8059","author":{"email":"5e18920209a4cd606e3313601e55b3e7d65ab4e9@fb.com","name":"Ari Ekmekji"},"message":"Parallelize L0-L1 Compaction: Restructure Compaction Job\n\nSummary:\nAs of now compactions involving files from Level 0 and Level 1 are single\nthreaded because the files in L0, although sorted, are not range partitioned like\nthe other levels. This means that during L0-L1 compaction each file from L1\nneeds to be merged with potentially all the files from L0.\n\nThis attempt to parallelize the L0-L1 compaction assigns a thread and a\ncorresponding iterator to each L1 file that then considers only the key range\nfound in that L1 file and only the L0 files that have those keys (and only the\nspecific portion of those L0 files in which those keys are found). In this way\nthe overlap is minimized and potentially eliminated between different iterators\nfocusing on the same files.\n\nThe first step is to restructure the compaction logic to break L0-L1 compactions\ninto multiple, smaller, sequential compactions. Eventually each of these smaller\njobs will be run simultaneously. Areas to pay extra attention to are\n\n  # Correct aggregation of compaction job statistics across multiple threads\n  # Proper opening/closing of output files (make sure each thread's is unique)\n  # Keys that span multiple L1 files\n  # Skewed distributions of keys within L0 files\n\nTest Plan: Make and run db_test (newer version has separate compaction tests) and compaction_job_stats_test\n\nReviewers: igor, noetzli, anthony, sdong, yhchiang\n\nReviewed By: yhchiang\n\nSubscribers: MarkCallaghan, dhruba, leveldb\n\nDifferential Revision: https://reviews.facebook.net/D42699","distinct":false,"url":"https://api.github.com/repos/norton/rocksdb/commits/40c64434d41824a7e149a8a47ea2c03838fd8059"},{"sha":"87df6295dd3b2e50eb6dd7becc6dd61be45d9b1d","author":{"email":"01b4858c232b891e1ad3edd1cd6314647466376f@fb.com","name":"Venkatesh Radhakrishnan"},"message":"Make SuggestCompactRangeNoTwoLevel0Compactions deterministic\n\nSummary:\nMade SuggestCompactRangeNoTwoLevel0Compactions by forcing\na flush after generating a file and waiting for compaction at the end.\n\nTest Plan: Run SuggestCompactRangeNoTwoLevel0Compactions\n\nReviewers: yhchiang, igor, sdong\n\nReviewed By: sdong\n\nSubscribers: dhruba, leveldb\n\nDifferential Revision: https://reviews.facebook.net/D43449","distinct":false,"url":"https://api.github.com/repos/norton/rocksdb/commits/87df6295dd3b2e50eb6dd7becc6dd61be45d9b1d"},{"sha":"20b244fccaf914f17b5c0f115fd04cc912becd9d","author":{"email":"01b4858c232b891e1ad3edd1cd6314647466376f@fb.com","name":"Venkatesh Radhakrishnan"},"message":"Fix CompactFiles by adding all necessary files\n\nSummary:\nThe compact files API had a bug where some overlapping files\nare not added. These are files which overlap with files which were\nadded to the compaction input files, but not to the original set of\ninput files. This happens only when there are more than two levels\ninvolved in the compaction. An example will illustrate this better.\n\nLevel 2 has 1 input file 1.sst which spans [20,30].\n\nLevel 3 has added file  2.sst which spans [10,25]\n\nLevel 4 has file 3.sst which spans [35,40] and\n        input file 4.sst which spans [46,50].\n\nThe existing code would not add 3.sst to the set of input_files because\nit only becomes an overlapping file in level 4 and it wasn't one in\nlevel 3.\n\nWhen installing the results of the compaction, 3.sst would overlap with\noutput file from the compact files and result in the assertion in\nversion_set.cc:1130\n\n // Must not overlap\n   assert(level <= 0 || level_files->empty() ||\n            internal_comparator_->Compare(\n                (*level_files)[level_files->size() - 1]->largest, f->smallest) <\n                0);\nThis change now adds overlapping files from the current level to the set\nof input files also so that we don't hit the assertion above.\n\nTest Plan:\nd=/tmp/j; rm -rf $d; seq 1000 | parallel --gnu --eta\n'd=/tmp/j/d-{}; mkdir -p $d; TEST_TMPDIR=$d ./db_compaction_test\n--gtest_filter=*CompactilesOnLevel* --gtest_also_run_disabled_tests >&\n'$d'/log-{}'\n\nReviewers: igor, yhchiang, sdong\n\nReviewed By: yhchiang\n\nSubscribers: dhruba, leveldb\n\nDifferential Revision: https://reviews.facebook.net/D43437","distinct":false,"url":"https://api.github.com/repos/norton/rocksdb/commits/20b244fccaf914f17b5c0f115fd04cc912becd9d"},{"sha":"26894303c1eb55650c9b2f5e5a8a3bfbd9f5f5ce","author":{"email":"e363a170056789af5295e3062579e8b8a22461d8@fb.com","name":"Yueh-Hsuan Chiang"},"message":"Add CompactOnDeletionCollector in utilities/table_properties_collectors.\n\nSummary:\nThis diff adds CompactOnDeletionCollector in utilities/table_properties_collectors,\nwhich applies a sliding window to a sst file and mark this file as need-compaction\nwhen it observe enough deletion entries within the consecutive keys covered by\nthe sliding window.\n\nTest Plan: compact_on_deletion_collector_test\n\nReviewers: igor, anthony, IslamAbdelRahman, kradhakrishnan, yoshinorim, sdong\n\nReviewed By: sdong\n\nSubscribers: maykov, dhruba\n\nDifferential Revision: https://reviews.facebook.net/D41175","distinct":false,"url":"https://api.github.com/repos/norton/rocksdb/commits/26894303c1eb55650c9b2f5e5a8a3bfbd9f5f5ce"},{"sha":"be8621ffafe34ac5e4560a70b0e795eee6f2bc6b","author":{"email":"e363a170056789af5295e3062579e8b8a22461d8@fb.com","name":"Yueh-Hsuan Chiang"},"message":"Fix compile warning in compact_on_deletion_collector in some environment\n\nSummary: Fix compile warning in compact_on_deletion_collector some environment\n\nTest Plan: make\n\nReviewers: igor, sdong, anthony, IslamAbdelRahman\n\nSubscribers: dhruba, leveldb\n\nDifferential Revision: https://reviews.facebook.net/D43467","distinct":false,"url":"https://api.github.com/repos/norton/rocksdb/commits/be8621ffafe34ac5e4560a70b0e795eee6f2bc6b"}]}