In-memory cache VS. centralized cache in a distributed system
<p>We're currently looking for the most suitable solution for accessing critical data on a distributed system, and we're considering whether to use in memory caching, versus a centralized cache.</p>

<p>Some information about the data we wish to store/access:</p>

<ul>
<li>Very small data size</li>
<li>Data is very cold; meaning it barely changes, and only changes when a human changes something in our back office system</li>
<li>Has to be up to date when changed (a few 100ms delay is OK)</li>
<li>Very critical path of our application, requires very high SLA (both in reliability and in response times (no more than 20ms to access))</li>
<li>Data is read from frequently (up to thousands of times per second)</li>
</ul>

<p>The way we see it is as following -</p>

<p><strong>In memory cache</strong></p>

<p>Pros:</p>

<ul>
<li>Quicker than network access + serialization</li>
<li>Higher reliability in terms of distribution (if one instance dies, the data still exists on the other instances)</li>
</ul>

<p>Cons:</p>

<ul>
<li>Much more complex to code and maintain</li>
<li>Requires notifying instances once a change occurs and need to update each instance seperately + Need to load data on start of each server</li>
<li>Adds a high risk of data inconsistency (one instance having different or outdated data than others)</li>
</ul>

<p><strong>Centralized cache</strong></p>

<p>For the sake of conversation, we've considered using Redis.</p>

<p>Pros:</p>

<ul>
<li>Much simpler to maintain</li>
<li>Very reliable, we have a lot of experience working with Redis in a distributed system</li>
<li>Only one place to update</li>
<li>Assures data consistency</li>
</ul>

<p>Cons:</p>

<ul>
<li>Single point of failure (This is a big concern for us); even though if we go with this solution, we will deploy a cluster</li>
<li>What happens if cache is flushed for some reason</li>
</ul>