/etc/passwd was truncated

Today I received a call from security officer telling me that the /etc/passwd file of one machine was truncated (0 byte). The same happened with /etc/group a couple of days before.

I tried to trace the cause but I had no success... can someone please give me a clue where should I look for?

I'm running Red Hat 5.7 x86_64.

Solution 1:

If you think you've been hacked, I strongly recommend How do I deal with a compromised server? .

Otherwise, check root's .bash_history, looking for things like output redirection onto /etc/passwd. If you use sudo to control root access, check the sudo logs around the time (the modification time on the file will tell you when it was zeroed out, most likely).

And, for goodness' sake, get off Red Hat 5.7. You're running an OS that's nearly a year's worth of patches out-of-date, and there's no sane reason to. Red Hat's patching policy is that no version of any package is ever bumped inside a major version of the OS; patches are backported instead. The whole point of this is that you should be able to keep your RHEL5 constantly in patch without endangering any functionality.

In many ways, there isn't even any such thing as "Red Hat 5.7"; RHEL 5.7 isn't really an OS release version, it's simply a line drawn through the current state of patch of RHEL5 on 12/3/2012. When you say "I'm running RHEL 5.7" what you're really saying is "I'm running RHEL 5 and I'm a year out of date".

Solution 2:

It will be a lot easier to find out the cause, if you have auditd daemon properly configured to watch on critical files. In addition to the suggestions forwarded by others, you can try to look for hints in the /var/log/audit/audit.log file and use the aureport and ausearch commands.

For instance, you can run ausearch -f passwd to list all audit information related to the passwd file.

In the future, you might need to follow the security guidelines set in the CIS Red Hat Enterprise Linux 5 Benchmark on how to configure audit daemon to watch (-w ) certain files. Here are some lines you need to add to /etc/audit/audit.rules -

Things that affect identity:

   -w /etc/group -p wa -k identity
   -w /etc/passwd -p wa -k identity
   -w /etc/gshadow -p wa -k identity
   -w /etc/shadow -p wa -k identity
   -w /etc/security/opasswd -p wa -k identity

Solution 3:

@MadHatter offers a good start. Beyond that, here's a trick I keep in my toolbelt for days like these:

find /filesystem1 /filesystem2 -xdev -printf '%T@ %t %p\n' | sort

%T@ Modification time, epoch seconds (for sorting)

%t Modification time, human readable

%p Full file path

Season to taste. Run on one mounted filesystem at a time or as many as you think is relevant.

What this does is give you a timestamp sorted audit of the most recent modifications to files. This can occasionally give you insight into other things that were happening at the time of the unknown event. It's certainly not bulletproof, and may not reveal anything at all, but I occasionally find this extremely useful in identifying what work was being performed at the time that the unknown event happened. (say, a someone performing a rm -Rf on /lib instead of Tomcat's lib...)