Hardware Requirements & Tuning - Flash Media Server 3.5 Interactive
We run FMS 3.5 on a Dual Dual-Core Xeon with 4gb of ram, and so far it hasn't missed a beat. We run about 50-150 concurrent connections. The main issue you will have, from experience, is memory. FMS loves to cache, especially when stremaing oft-requested content.
There is only one thing I will advise you to stear absolutally clear of - do not run FMS under a virtualised environment. We initially had it running inside a virtual machine and it was absolutally pathetic. Switching it to a 'real' server (even though the server is shared with other services) was a thousand times better.
Update: It turns out that the virtual server we were running FMS in had the wrong NIC drivers installed. It was a Xen guest. We've moved to VMWare since then, and we haven't re-tried running FMS under the new hosts as it's still happilly chugging along on its own box.
Another thing to note is that we run Windows, which is probably less efficient than the *nix variants.
With some QoS between the server and the clients to maximise throughput for things that you really need (like streaming video) and throttle things you don't so much (webcam chats), I can't imagine you would have too many problems if you ran a server with two cores and 4gb of ram.