Should security updates always be installed?

We are a small shop, with no real sysadmin. So developers (Java EE) also try to maintain Ubuntu server 12.04.

When I login a terminal and I see messages like:

6 packages can be updated.
6 updates are security updates.

Should all security updates always be installed? Can some be ignored? Should these be acted upon immediately? Or one could wait for 2:00 a.m. on the coming Sunday? Is there a way to know a "critical" update?

Is there a good sysadmin for dummies resource I should be reading?

Solution 1:

Generally security updates are about fixing serious BUGS in the current Ubuntu OS regarding Network, Security and all. If you are maintaining a web server or any web related things, then you must have them to be safe.

For example see here some issues: Ubuntu security notices | Ubuntu

Solution 2:

If you're going to hold back any updates, I wouldn't hold back security updates.

There's a reason the server install has an install-time option of "automatically download and install only security related updates"