Is there any way to limit FFmpeg’s memory usage?
ffmpeg does not use any RAM, that it not actually needs. Typically this is very little.
If you ffmpeg processes really use a lot of RAM (a.o.t a misinterpretation of some reading), then limiting its RAM allotment will just make it crash with an OOM condition.
If you tell us about your ffmpeg command line, we might be able to suggest alternatives, that use less RAM.