Am I missing something with managing happiness?

Solution 1:

There aren't any strategies that have you sitting on significant unhappiness throughout the game - it is a serious detriment to your empire.

Conquest can produce some significant unhappiness. In my experience, it is almost always worse in the long run to annex a city. The happiness hit is overwhelmingly strong when you annex, and it's significantly less strong when you puppet. Puppeted cities will eventually build happiness buildings, although it can take a while.

Once you get past the "middle of the road" difficulty levels, happiness is kind of a key bellwether when it comes to expansion. I tend to only build a new city when I can take a happiness hit, and if I can gain access to at least one but preferably two new luxury resources. Strategic resources might trump this, in certain cases. It's very difficult to survive without Coal for Factories, for instance.

Razing a city can be a very legitimate strategy. If you conquer an enemy, and they have some tiny cities that are not near luxury/strategic resources, burn 'em to the ground. Expansion is the primary way you're going to get more powerful, and expansion is heavily limited by happiness. Hanging on to cities that aren't helping you become more powerful is a waste.

Unhappiness is a factor of both the number of cities and the population, so if you've gotten to a balancing point, you might consider setting your cities to "avoid growth" - this will up their production and down their food intake, in an attempt to keep them from getting bigger and producing more unhappiness.

Since happiness is generated by buildings you unlock through technology, science is an important key in maintaining your happiness balance. With Patronage, I'd expect you to be gifting/bribing city states to get access to their luxuries. You can also trade excess luxuries to other civs, if they've got something you need.

Solution 2:

The AI cheats to keep up.

Annexing is never a good idea unless you require immediate use to build something. You can purchase a courthouse immediately with gold to help if needed, but always have a courthouse be the first priority over anything except losing the city back to an opponent.

Hover your cursor over your happiness bar. It will tell you exactly where the deficits and gains are coming from. It sounds like your problem could be luxury resources. The games I have played have always been races for luxury goods in order to accumulate happiness and excess goods to both deny my opponents of them or to be able to trade them for unique resources they have and I do not. Sometimes it is important to get excess resources when playing computers to do 2 for 1 trades. Always only trade resources you have spares of or you lose the happiness benefit from goods traded away unless you have more.

I cannot remember if the skill changed in BNW (Not able to get in game right now), but the Merchant tree had an awesome benefit to provide extra happiness for each resource which has worked out really well for me in the past.

I typically do not raze cities that provide me any sort of long term benefit (even if it is small). Think of how long it takes to get a city going (really going, not just plopped down). Eventually my captures are most likely an asset that you can stage units from or provide me with resources/GPT. Puppet them until you can buy a courthouse OR if you have the spare happiness to let it slowbuild. Conquering multiple cities at a time will really hurt your happiness and sometimes there are just points in the game where you may need to dip negative in order to obtain a long term high positive.

I don't really mess with Patronage... I just conquer the City States as they are typically well developed with little downtime until courthouse is up. When you have enough money to be donating to City States you can typically afford it without the patronage benefits anyways. I usually play between Traditional/Liberty/Piety/Merchant/Autocracy Skill Trees.

Solution 3:

The AI gets a sizable happiness boost to keep them competitive against us humans. Happiness is indeed the expansion cap the designers devised to defeat infinite city sprawl (ICS): spamming out cities to quickly acquire land. As futurehermit says in his CivFanatics signature, "Land is power. Land is power. Land is power."

The Honor social policy branch has one or two happiness-related policies:

• +1 city :) and +2 culture for each garrisoned unit
• +1 city :) for each Barracks

I've been playing with the Communitas mod for quite some time, so forgive me if the second one isn't entirely correct. I think in the mod it's every military building including Armory and Military Academy, but maybe that is the same in the standard game.

Another option to razing cities is to trade them to rivals for gold or luxuries. This can be really fun if you can gift them to one rival who is an enemy of another that neighbors the city in the hopes that this will trigger mayhem. The AI is horrible at determining the actual value of a gifted city and will merrily accept a city they have no hope in defending or turning into a good city.