Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:  Switch to Plain View
HBase, mail # user - Consider individual RSs performance when
writing records with random keys?  Copy link to this message
Consider individual RSs performance when writing records with
random keys? Alex Baranau 2012-05-17, 14:23  Not sure if you've
seen HBaseWD (https://github.com/sematext/HBaseWD) project. It
implements the "salt keys with prefix" approach when writing
monotonically increasing row key/timeseries data. If simplified,
the idea is to add random prefix to the row key so that writes
end up on different region servers (avoiding single RS hotspot).
When writing data to HBase with salted or random keys (so that
load is well distributed over cluster) the write speed per RS is
limited by the slowest RS in cluster (singe one Region is served
by one RS).  Given 1 & 2 I got this crazy idea to: * write in
multiple threads * each prefix (or interval of keys in case of
completely random