Security for university research lab systems

Being responsible for security in a university computer science department is no fun at all. And I explain: It is often the case that I get a request for installation of new hw systems or software systems that are really so experimental that I would not dare put them even in the DMZ. If I can avoid it and force an installation in a restricted inside VLAN that is fine but occasionally I get requests that need access to the outside world. And actually it makes sense to have such systems have access to the world for testing purposes.

Here is the latest request: A newly developed system that uses SIP is in the final stages of development. This system will enable communication with outside users (that is its purpose and the research proposal), actually hospital patients not so well aware of technology. So it makes sense to open it to the rest of the world. What I am looking for is anyone who has experience with dealing with such highly experimental systems that need wide outside network access. How do you secure the rest of the network and systems from this security nightmare without hindering research? Is placement in the DMZ enough? Any extra precautions? Any other options, methodologies?

This is entirely dependent on the "experimental system" in question -- Security is not something that comes in a box: it needs to be custom-tailored to every specific site, scenario, and application.

If you're talking about stuff written by students (who are pretty well notorious for having ZERO practical grasp of IT security), I would say that each project needs to be segregated into its own universe.
That means:

• Placed in its own (appropriate-sized) subnet, with its own router
  (it should go without saying that it should be in its own vLAN as well)
• Placed behind its own firewall, which should:
  • Restrict inbound access as much as practical
    (i.e. only the ports that the application needs to accept connections on)
  • Restrict outbound connections where sensible
  • Be completely separate from the rest of the university & department infrastructure
    (your externally exposed apps should not, for example, be able to find a way through your defenses to talk to the campus AD servers)
• Internal topology should follow best practices for containment of breaches
  This one is hard because it requires the software to be designed well, but where possible you want to set things up so if someone breaches a front-end system (like a web server) your back-end systems (like databases) may still be safe.
  If the software is designed for segmentation, for example something having an APP layer, a DBI layer, and a DB layer you can theoretically separate that into three internal networks. APP can talk to DBI, DBI can talk to DB, but APP can't talk to DB unless DBI handles the request.

If you're using good Cisco hardware this isn't hard to do (FWSM "PIX Blades" for the independent firewalls, some basic vLANs, and dropping a router into each subnet).

General Best Practices obviously apply as well - you should have everything documented (What needs to be open and why? What special network configuration (if any) is applied? etc.), and if you have an IDS/IPS in place you should look for ways to make sure these isolated environments are covered.

Not knowing your policies or regulatory requirements, we can't tell you what's "enough." A properly firewalled and monitored subnet generally suffices, as long as you

1. know what you're allowing
2. document it
3. ensure that this matches any security policies you have
4. and can justify it to your boss, his boss, the government, or a PCI auditor or whatever applies to your organization.