{"push_id":550043609,"size":1,"distinct_size":1,"ref":"refs/heads/feature/ku-optimize-mapred-listkeys","head":"8ab5fe64e542986e72af0f81f2df462ab54f08ed","before":"f31ad3881f474cb9b2e67ec67cec597034beab0c","commits":[{"sha":"8ab5fe64e542986e72af0f81f2df462ab54f08ed","author":{"email":"a7ead894957420aac12ad9f8253447fda897a7c1@basho.com","name":"UENISHI Kota"},"message":"Optimize mapreduce by replacing listkeys+get with 2i query\n\nSimple benchmark in CS showed 1.7x performance improvement:\n\n```\n(riak-cs@127.0.0.1)1> timer:tc(fun()->riak_cs_storage:sum_bucket(<<\"test\">>)end).\n{4673368,\n {struct,[{<<\"Objects\">>,98407},{<<\"Bytes\">>,4341828394}]}}\n(riak-cs@127.0.0.1)2> timer:tc(fun()->riak_cs_storage:sum_bucket2(<<\"test\">>)end).\n{2810982,\n {struct,[{<<\"Objects\">>,98407},{<<\"Bytes\">>,4341828394}]}}\n```\n\nThis performance gain stems from duplicate disk read from both\n`riak_kv_pipe_listkeys` and `riak_kv_pipe_get` .\n`riak_kv_pipe_listkeys` does not read from disk when the backend is\nbitcask, but when the backend is leveldb, its reads the whole block\nincluding not only keys but values (aka Riak objects). This pull\nrequest replaces them with modified `riak_pipe_index` with\n`return_body=true` and enable a vnode to just read from one time. This\nis inspired by `riak_kv_pb_csbucket`.","distinct":true,"url":"https://api.github.com/repos/basho/riak_kv/commits/8ab5fe64e542986e72af0f81f2df462ab54f08ed"}]}