Hide apps with closed windows from application switcher in OS X
Solution 1:
The automatic termination feature introduced in 10.7 should already do that, but it only works with some applications and only in some cases. If you for example open Preview, open at least one window in it, close all windows, and focus another application, Preview should be hidden from the Dock and the application switcher. The process is not actually terminated, but there might be some changes in how it is managed.
Witch and Switché have options to always hide applications with no open windows:
Solution 2:
Open the Info.plist of the application you intend to perform this on and add the following:
<key>NSUIElement</key>
<string >1</string>
The Info.plist can usually be found at /Applications/[app name].app/Contents/Info.plist
with the [app name]
obviously being the name of your app.