What effect does happiness have?

Here is a list of all the effects from happiness level that I could find. To answer your specific question, most of the effects care only if your civilization is happy or unhappy, by one :) or :(. (I believe zero :) is fine.) But there are three effects that care about the amount your civilization is happy or unhappy:

• VERY unhappy occurs when you hit -10 happiness. (Thank you @Oak)
• Golden Age points accumulate (or deplete) equal to the amount your civilization is happy (or unhappy)
• Piety->Mandate of Heaven uses your exact excess happyness to calculate how much culture you get.

Full list of effects:

Happy

• Growth: Cities grow normally.
• Golden Ages: If you have excess happiness, each turn it accumulates into Golden Age points, and when you have enough, your civilization enters a Golden Age. After the golden age the process starts again, although I believe you need more Golden Age points for each subsequent Golden Age.

Unhappy

• Growth: Cities grow at 1/4 speed when your population is unhappy.
• Golden Ages: The amout your civilization is unhappy is subtracted from the Golden Age points.

VERY unhappy

• Growth: Cities do not grow.
• Units: Settlers may not be trained.
• Production: Cities generate -50% production.
• Unit are less effective in combat. They get -33% combat strength.

VERY VERY unhappy (-20 or lower)

• Rebels will spawn in your border and attack your city

Social Policies

• Piety->Mandate of Heaven allows you to add 50% of your excess happiness to your culture each turn.
• Rationalism->Sovereignty boosts science +15% when your empire is happy.
• There are numerous social policies that affect the happiness level, but these are the ones that get effect from the happiness level.

Sources:

• General info: mousing over the happiness listing on the top bar, left side, during various happy states.
• Golden age info: mousing over the golden age display on the top bar, left side, during various happy states.
• Combat info: Comparing happy (normal) combat strength to VERY unhappy combat strength, by mousing over an enemy unit. The VERY unhappy combat modifier is clearly listed in the calculations.
• Production info: mousing over a city's production listing in the top left when VERY unhappy.
• Social Policy info: The social policy screen, accessible by clicking on the culture listing on the top bar, left side.