Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:  Switch to Threaded View
Copy link to this message One weird problem of my MR job upon
hbase table. Liu, Raymond 2013-01-05, 03:37  I encounter a weird
lag behind map task issue here :  I have a small hadoop/hbase
cluster with 1 master node and 4 regionserver node all have 16
CPU with map and reduce slot set to 24.  A few table is created
with regions distributed on each region node evenly ( say 16
region for each region server). Also each region has almost the
same number of kvs with very similar size. All table had
major_compact done to ensure data locality  I have a MR job
which simply do local region scan in every map task ( so 16 map
task for each regionserver node).  By theory, every map task
should finish within similar time.  But the real case is that
some regions on the same region server always lags behind a lot,
say cost 150 ~250% of the other map tasks average times.  If
this is happen to a single