How to resolve a stuck major compaction in Apache Cassandra?

I've got a one-node Cassandra cluster which is currently stuck in a major compaction process. After executing a nodetool compactit started the compaction and I see the tmp-sstable files with a size of 0 bytes. But that's all. No progress for hours.

I already stopped the Cassandra service, but after a restart it continued the compaction again without progress. A nodetool compactionstats shows 0,00% progress and a remaining time of six minutes. But I already waited 24 hours. In the meantime I stopped all readers and writers without seeing any difference.

The version I use is 2.1.2. I already tried with OpenJDK 7 and with Oracle's Java 7, but no difference.

The logs show no indication of OOM or any other exception or warning.

I'm not sure if it is of any help, but I currently migrate away from Cassandra as it doesn't fit my needs. So I delete a lot of data which has been migrated. There are many rows with several hundred up to a few thousand columns. But there are a handful rows with a million of columns. They now have a lot of tombstones in them due to the migration (deletion) process.

Any help in how to debug the issue are welcome.

Solution 1:

I'am not a cassandra expert, but did you tried nodetool stop compaction and to set a threshold for compaction (setcompactionthreshold to 0), so the system will not retry it