What attribute values should I strive for?

I know that part of the game is electing to trade off one thing for another - and this includes attribute values (e.g. higher magic level to gain access to spells and equipment or higher willpower for more mana). However, I don't know what the different attributes unlock for each character class and so don't know how to make a plan to get the necessary attributes when I need them.

So far I've been concentrating on the 2-3 attributes I think are most needed for each class (e.g. strength & constitution for warrior types and magic, willpower, and constitution for mage types).

Recently, I just discovered that some skills available for Alister also require fairly high (18) dexterity.

What are reasonable values that I should strive to get in each attribute for the various classes?

Strength increases damage with weapons except staves and crossbows. It also increases physical resistance. This is the bread and butter stat for warriors, especially 2 handed warriors. It also increases intimidation dialogue success.

Dex increases damage from piercing weapons, as well as increasing physical resistance and attack. You want this for your rogues BUT ALSO for your tanks, since it increases defense. It's also a pre-req for shield talents.

Cunning increases armor penetration. It also aids in persuasion dialogue, lock picking stealing, etc.

Willpower is mainly for mages since it increases their mana pool. It also increases mental resistance and stamina pool, so you can give it to anyone but honestly it's not worth it

Magic increases spell power. It increases mental resistance too and makes health potions more effective. You want to stack this on your mages.

Stack magic on mages. Strength and dex on warriors (mostly strength for damage, dex for tanks). Rogues get cunning and dex (You need 16 in cunning for your abilities. Past that, you can go full dex, full cunning, or equal. All are viable).

Constitution is more of a feel thing. Get constitution to where you're comfortable. You honestly don't need much since health potions are so easy to get.

You can find builds for the Warden and your companions on the Bioware forums if you need help with stats, abilities, and tactics.

The answer by K. Patterson is quite good. There are a few additional details to keep in mind though:

Resistances

Every point in Strength, Constitution, and Dexterity increase physical resistance by half a point while each point in Magic, Cunning, and Willpower increase mental resistance by half a point. (These translate very loosely to 1% reduced chance of related status effects per point of resistance.) Without going into the exact mechanics, it is worth stating that the benefits of resistances are disproportionately reduced when you have very little of that resistance. Thus, you are rewarded for focusing mental stats as a mage and neglecting physical stats because a trivial bonus to physical resistance will give you effectively no benefit.

Resource Pools

Constitution and Willpower (even for non-mages) can be important to a degree. Beyond a comfortable threshold they won't add much, but on harder difficulties you'll need to spend a little in each for most characters. Constitution will play a larger role for Blood Mages as they'll need to use a bit of their health pool to cast their spells while their special mode is active. A low Willpower can be made up for by a massive investment in mana potions for mages, but Warriors/Rogues won't have that option in the base game. (Deep Mushrooms restore a very small amount of stamina -- typically not enough to matter.) In the expansion stamina potions exist and make Willpower far less necessary for Warriors and Rogues.

Skills

Some skills require Cunning to unlock. This includes Persuasion, Stealing, and Survival. A cunning of 16 is sufficient to unlock the highest rank of all of these. (Note that Persuasion requires 16 cunning but when used as intimidation, will use the Strength attribute to increase your chances of success, not cunning.) Thus, a bit of Cunning can be helpful for a warrior that desires to learn survival.

Arcane Warrior Equipment

The Arcane Warrior mage specialization will use Magic for equipping all kinds of gear that normally require high strength scores. This is yet another reason why stacking magic on Mages is so useful.