Waiting Threads Resource Consumption
<p><strong>My Problem:</strong></p>

<p>Does large numbers of threads in JVM consume a lot of resources (memory, CPU), when the threads are <code>TIMED_WAIT</code> state (not sleeping) >99.9% of the time? When the threads are waiting, how much CPU overhead does it cost to maintain them if any are needed at all?</p>

<p>Does the answer also apply to non-JVM related environments (like linux kernels)?</p>

<p><strong>Context:</strong></p>

<p>My program receives a large number of space consuming packages. It store counts of similar attributes within the different packages.  After a given period of time after receiving a package(could be hours or days), that specific package expires and any count the package contributed to should be decremented.</p>

<p>Currently, I achieve these functionalities by storing all the packages in memory or disk. Every 5 minutes, I delete the expired packages from storage, and scan through the remaining packages to count the attributes. This method uses up a lot of memory, and has bad time complexity (<code>O(n)</code> for time and memory where n is the number of unexpired packages). This makes scalability of the program terrible.</p>

<p>One alternative way to approach this problem is to increment the attribute count every time a package comes by and start a <code>Timer()</code> thread that decrements the attribute count after the package expires. This eliminates the need to store all the bulky packages and cut the time complexity to <code>O(1)</code>. However, this creates another problem as my program will start having <code>O(n)</code> number of threads, which could cut into performance. Since most of the threads will be in the <code>TIMED_WAIT</code> state (Java’s <code>Timer()</code> invokes the <code>Object.wait(long)</code> method) the vast majority of their lifecycle, does it still impact the CPU in a very large way?</p>