How to make `rm` faster on ext3/linux?

Solution 1:

The most interesting answer was originally buried in a comment on the question. Here it is as a first class answer to make it more visible:

Basically no method from here worked, so we developed our own. Described it in here: http://www.depesz.com/index.php/2010/04/04/how-to-remove-backups/ – depesz Apr 6 '10 at 15:15

That link is an incredibly thorough analysis of the exploration for and discovery of a workable solution.

Note also:

The article says:

As you can see, I used -c2 -n7 options to ionice, which seem sane.

which is true, but user TafT says if you want no disruption then -c3 'idle' would be a better choice than -c2 'best-effort'. He has used -c3 to build in the background and has found it to work well without causing the build to wait for ever. If you really do have 100% io usage then -c3 will not let the delete ever complete but he doesn't expect that is what you have based on the worked test.

Solution 2:

Upgrade to ext4 or some other modern filesystem that uses extents. Since ext3 uses the indirect blocks scheme rather than extents, deleting large files inevitably entails lots of work.

Solution 3:

You can give ionice a try. It won't make it faster but it might make it less disruptive.