Wednesday, August 6, 2008  scaling mnesia with local_content  so
we've been spending the last two weeks trying to scale our
mnesia application for a whole bunch of new traffic that's about
to show up. previously we had to run alot of ec2 instances since
we were using ets (RAM-based) storage; once we solved that we
started to wonder how many machines we could turn off.  initial
indications were disappointing in terms of capacity. none of
cpu, network, memory, or disk i/o seemed particularly taxed. for
instance, even though raw sequential performance on an ec2
instance can hit 100mb/s, we were unable to hit above 1mb/s of
disk utilization. one clue: we did get about 6mb/s of disk
utilization when we started a node from scratch. in that case,
100 parallel mnesia_loader processes grab table copies from
remote nodes. thus even under an unfavorable access pattern
(writing 100 different tables at once), the machines were
capable of more.  one problem we suspected was the registered
process mnesia_tm, since all table