Snmpd stop working without any changes

Solution 1:

"Cannot statfs" likely comes from the disk usage monitor in snmpd, that iterates the mounted file systems and asks for the amount of free space left.

If a statfs(2) call fails, that is a serious problem on the machine, this is one of the syscalls that basically just look up information in a shared structure and return it, the only way this can fail is in synchronizing access to that structure.

So, something is hanging there that is holding exclusive access to some structure in the kernel, and that is also what blocks file system access, which causes the restart timeout.

If this is a local file system, I'd reboot and force a file system check during boot. Before systemd, the mechanism to do that would have been shutdown -Fr now, but systemd requires you to set a kernel commandline parameter.

If this is on a SAN or similar, I'd find out what's wrong with the SAN first, and then do a file system check.

Three hosts at the same time can really be explained only by "this file system is on a SAN that failed."