I need to create a lot of temporary files for which I need to have fast access, so I'm thinking about using a RAM disk. The problem is that the temporary files might get pretty big (1-4 GB) and in this case the disk-storage would be OK.

Is there any way to create a mix between a RAM disk and a physical HDD, so that the HDD gets used in the worst case (when I meet the big temporary files)?

---Edit---

I cannot afford faster SSD's, and I don't need persistent storage, and I don't need 100 GB of temporary data. I'm using my home computer for this (2 GHz AMD system...). I received a tip about UnionFS. What is the experience with it?


Solution 1:

Use tmpfs and a big swap partition or file. This file system will cache data in memory as long as it can, and swap them to disk if they don't fit into RAM.

Solution 2:

Why not try using the solid state SSD drives? You can get those with a capacity of 100 GB or more, and they are way faster than standard disks.

Solution 3:

I'd second the option to do RAID 0 (stripe) with a hardware RAID controller. I quoted a video encoding server and the scratch drives are 15k SAS in RAID 0 and it chews through video like nothing else.