Cassandra 3 Repair never finishes
<p>We have a cluster with 6 nodes in datacenters (3 nodes each). We are starting a repair on one node and shortly afterwords we can find something like this in the logs:</p>

<pre><code>ERROR [Repair#1:1] 2016-05-31 01:33:28,075 CassandraDaemon.java:195 -     Exception in thread Thread[Repair#1:1,5,RMI Runtime]
com.google.common.util.concurrent.UncheckedExecutionException: org.apache.cassandra.exceptions.RepairException: [repair #e8e21070-26be-11e6-aae8-77b20cefeee5 on ..... Validation failed in /xx.xxx.xx.xx
    at com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1525) ~[guava-18.0.jar:na]
    at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1511) ~[guava-18.0.jar:na]
    at org.apache.cassandra.repair.RepairJob.run(RepairJob.java:162) ~[apache-cassandra-3.0.4.jar:3.0.4]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_77]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[na:1.8.0_77]
    at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_77]
</code></pre>

<p>Afterwords nothing seems to happen anymore. We did not interrupt the repair for several days, but still nothing happens. We also tried it on two different clusters with the same result.</p>

<p>After searching through the web we stumbled upon <a href="https://support.datastax.com/hc/en-us/articles/205256895--Validation-failed-when-running-a-nodetool-repair" rel="nofollow">https://support.datastax.com/hc/en-us/articles/205256895--Validation-failed-when-running-a-nodetool-repair</a>. It says that we should run "nodetool scrub" and if it does not help "sstablescrub".</p>

<p>We tried the nodetool scrub but the repair does still not work. We now started a sstablescrub but it seems to take forever. It uses only one cpu at 100% and the data and index file is growing, but it now runs for over a day and the file now only has a size of 1.2GB.</p>

<p>Is it normal that "sstablescrub" is so slow?</p>

<p>The cluster is already running for some time, and we missed the GCGraceSeconds for the repair. Might that lead to the not working repair?</p>

<p>We currently do not know how to get the repair running hope someone can help.</p>