Redis eats up more and more memory
Turns out I couldn't see the wood for the trees, and one comment of Michael Hampton put me on the right track.
It didn't have anything to do with Redis itself after all but one of my apps was misbehaving. It uses Sidekiq for background processing, which uses Redis to store the jobs, error messages, backtraces etc. I started getting weird errors for some of my jobs but ignored them since I had to, you know, solve this issue with Redis first. :)
It seems like the background worker which is supposed to send a notification to Errbit any time an exception occurs in my app produced an exception itself, which generated extremely long backtraces and more and more jobs...
I stopped the background processing for now and now Redis consumes a normal amount of RAM.