Understanding memory usage in Ubuntu, Buffer, Cached memory
<p>I apologise for big post, to explain my problem it is really required. Please bear with me.</p>

<p>I have been developing a <strong>C</strong> based commercial application in Linux (Ubuntu) for last 1+ year. My aim is to make it highly stable and must run for infinite time without any system error.</p>

<p>The major problem I face is memory shortage. The program runs for around 8 hours and after that I see it starts using <strong>swap</strong> memory. Then everything starts to execute bit slow.</p>

<p>The first thing that came to my mind is <strong>memory leak</strong>. I used Valgrind, code review to check for memory leak. I am putting heavy load on my application in Valgrind mode in a cyclic pattern for several hours, some tests pass, some fail, but I do not see memory leak or invalid access. I can confirm that there is no memory leak in my application. </p>

<p>I checked using <strong>top</strong> and <strong>free</strong> commands to see memory usage. The <strong>free -m</strong> command output looks as below.</p>

<pre><code>             total       used       free     shared    buffers     cached
Mem:          7858       2146       5712        426         11        759
-/+ buffers/cache:       1375       6483
Swap:         8062          0       8062
</code></pre>

<p>This the initial state, where memory consumed is less.
After I started the application, I did wait for 5 min. Then <strong>free -m</strong> output shows as below.</p>

<pre><code>             total       used       free     shared    buffers     cached
Mem:          7858       3611       4247        563         71       1892
-/+ buffers/cache:       1647       6211
Swap:         8062          0       8062
</code></pre>

<p>If you see the difference, it is really big. Initially used memory was 2146 MB, and after 5 min run, the memory used is 3611 MB. The difference is 1465 MB, which is around 1GB+. I am sure currently my application is not using 1GB+ RAM. Probably major part of the memory is in <strong>cached</strong> section.  </p>

<p>If you see <strong>cached</strong> column (last one), you can see the difference of 1133 MB.</p>

<p>Also you can see buffers (last but second column), buffers increased by 60.</p>

<p>These values keeps increasing and finally system starts using swap memory. Once it reaches that stage, then my application sustains for 1 more house and finally system speed becomes very slow, so I close the application.</p>

<p>My expectation is let system go and store allocated memory segments in cached section, but when my m/c RAM usage is say 90%, further OS should start using from cached memory, thereby 10% of RAM is always there for other applications. Is there any OS level configuration where I can put a cap beyond which cache/buffers should start yielding memory when RAM is about to exhaust. </p>

<p>Is there any configuration, where strictly I will say to OS do not use swap memory. Is it /proc/sys/vm/swappiness file, where I can change default value of 60 to 0, will it cause any issue.</p>

<p>Please guide me with your valuable input.</p>