How can I restrict RAM usage for a particular user or process?

I'm using a background service that leaks memory from time to time. To prevent using the Alt+SysRq REISUB[1], I'd like to restrict the RAM usage for one process or a system user.

I really would like to say: User x may use the maximum amount of RAM of 1 GB.

Is this feasible? Do I need kind of virtual machine for this? I'm using Ubuntu 9.10.

ulimit is your friend. Check out the -m flag. I think you set this before you launch your process and it applies from then on (in the current shell).

In addition to the shell commands ulimit (for bash) or limit (for csh), you can control this from inside your code with the ulimit() function.