No email notifications when Mail.app isn't running?

Solution 1:

I can only say that this is true. Mail.app has to be open to get notifications.

A workaround: Open Mail, and close it with the red dot (or command+w). So the blue point under the icon in the Dock is still there. So you have no window open but mail is still getting emails.

There is no app or anything there yet to fix this "problem".

Solution 2:

Unfortunately, there's currently no way to have Mail.app check for mail if it's not running. But it is possible to hide the badge!

In System Preferences, open Notifications and select Mail. There, deselect 'Badge app icon with notification count'.