What is a viable character build for using a whip?
First some generic comments:
a) Any character in Crawl can learn any skill. If you want to become good with a whip as a weapon, then spend a lot of time using a whip as a weapon. As your Maces & Flails skill increases, you will do more damage with the whip.
b) Any weapon is a viable weapon in the end game, provided your skill with that weapon type is high enough and the weapon has a good brand (i.e. of electrocution).
Back to your questions:
Whips are the daggers of the Maces & Flails family. They have limited damage but are extremely fast. As such, they are good weapons early on, but by the middle of the game, you will want to switch to a weapon in the same family that does more damage. Generally this is because as your skill increases, your speed increases, so you can start upgrading to slower, but higher damage weapons.
Whips are best used by the dextrous, which means that when you are leveling your character, you should ensure your dexterity is somewhat higher than your strength. The calculation between the two is a bit complicated and you never want to max out one at the expense of the other, but if you are using light weapons, you should strive to have a higher dexterity than strength.
Finally, while I'm not sure I would rely on a branded whip late in the game, the demon whip is probably one of the best weapons in the game. However, you don't have to spend the early game with a whip in order to use a demon whip in the late game - you just have to use something in the Maces & Flails category, such as, you know, a mace or a flail.