Pull network or power? (for contianing a rooted server)

Solution 1:

If you're facing an APT, then your best option is to set up a honeypot and thoroughly investigate all traffic that flows into and out of it, in addition to monitoring the server.

The measure of going through memory is so expensive in terms of time and effort that it's usually not worthwhile unless you've tried every other method, and if you determine that it's worthwhile, it's generally best setting up a honeypot that allows you to easily dump the memory and system state to another machine on the fly so you can do analysis with less threat of being detected while the machine is up and running.

I had one situation where the attacker kept everything in memory to the degree that, except for logs, the machine looked exactly like its image once powered off and back on. They would then hack back in and start using it again because the vulnerability was still there - they didn't need to leave any backdoors for themselves. A memory evaluation could have helped here, but watching the traffic was enough in this case to identify the vulnerability quickly.

Therefore:

The only reason to avoid pulling the power and doing offline disk evaluation is if you're going to go through the pain of doing a thorough memory analysis of the threat while it's in place and operating. If you've gotten to the point where this is necessary, then there is no reason to pull either plug.

If you're not doing a memory analysis, then pulling the power plug is your best bet - pulling the ethernet (or using a shutdown command) is only going to give the attacker's software advance notice - which does matter occasionally.

So:

Pull them both, unless you're doing a memory analysis, in which case, don't pull either.

Solution 2:

RAM forensics (e.g. /dev/shm) can be helpful.

But I prefer unplugging the power cable (but try to log-in and rsync /proc right before).

The reasons for going for the power cable are:

1. When you do forensics in a hacked system, you are "stepping all over the crime scene"
2. The root kit keeps running - not so hard for the malicious to execute something (e.g. system wipe-out) on Network Link Down event.

Kyle Rankin gave a nice Intro to Forensics talk - there he recommends pulling the power cable.