What folders should I be backing up? Are there any I should avoid?

Some people recommend backing up your entire ~/Library folder. But I find there are a lot of things in there I wouldn't want replaced if doing a clean install. So I selectively back up certain contents of that folder, along with certain contents of the system level /Library folder.

You should poke around in ~/Library and see what's in there that you need. Here's a list of what I backup to get you started:

~/Library/Application Support and ~/Library/Preferences are where most of your application settings live. You should go through each of those folders' subfolders and see which ones you need.

Notably, Address Book's database is stored in an Application Support subfolder while some other Mac OS preinstalled apps have their folders directly at the ~/Library level. It's a good idea to look through the folders at that level as well. For example:

~/Library/Mail has your Mail.app data.

~/Library/Calendars has your iCal data.

~/Library/iTunes has all your iOS device backups and iTunes specific scripts.

~/Library/Scripts is probably where you install scripts if you use AppleScripts as folder actions.

~/Library/ScriptingAdditions holds any AppleScript additions you've installed.

~/Library/Keychains is the default location using Keychain Access to save passwords.

~/Library/Fonts for all fonts installed for the current user.

~/Library/ColorSync for all color profiles installed for the current user.

~/Library/Safari/Bookmarks.plist has your Safari bookmarks. The other stuff in this folder is a good example of what I'd like to NOT be restored after a clean install.

The structure of the system level /Library folder closely matches that of the ~/Library folder, so if there are things in there (fonts, color profiles, scripts etc.) installed "for all users", you should back those up too.

Hope that helps you get started!

What not to miss really depends on your apps and the data you commonly interact with. Can't help without more info there...

It's good to at least take Apple's guidance into account when excluding backed up items, they've had a regularly updated exclusion list for TimeMachine, which it consults when setting up a backup:/System/Library/CoreServices/backupd.bundle/Contents/Resources/StdExclusions.plist

It includes pretty nice comments recently, and calls out per-user directories to avoid. When it comes to the rest of ~/Library, though, I'd leave everything except what Apple excludes in, as it's usually a relatively small amount of data besides mail.

With the speed of disk and internet being pretty luxurious, the times I edit large video being relatively few, and applications with large bundle formats being more intelligent about flagging their data as modified, I think you don't have much to explicitly avoid, per se.

If you think you can trust .zip, .pkg and .dmg extensions to be re-downloadable, you may want to exclude those file types from the Downloads folder, I find myself editing things I've downloaded and being lazy about moving them to a better location, so I leave the entire folder in.

I'd kill the .ipsw and .ipa file types (unless you're an iOS developer or have apps that are no longer available through the store, I guess), or exclude those sections from iTunes Media, as they can be quite large and are only a convenience to cache locally, and become out of date really quickly in most cases.

Depending on how much you trust cloud sync services like Box, Google Drive or Dropbox's restore capabilities you may decide to exclude those as well.

There are two library folders on OS X. One is your user level library folder which is ~/Library and the other is the system level library at /Library. Although most of the preferences will be in your user level library, some programs save their files in the system library. For example, Microsoft Office will save preferences in the system library. I recommend backing up both even though you will never restore most of the files from the library backup (but you never know which ones you might need).

With regard to the Applications folder, yes you should back it up too, as most Mac apps are standalone apps, installed by copying them to the Applications folder, so they will work normally if you restore them just like any other files. Let's say you downloaded an app 2 years ago and over these two years that app may have undergone several updates that the app on the original .dmg won't include, therefore it would be preferable to back up the current version rather than the two year old .dmg. You should leave out apps that come as part of OS X though, in case the version of OS X differs when you restore your backup.