How do I obtain decorative gems?

There's a guaranteed spawn of 3 templars 10 o'clock from the Dwarfson's Pass camp. Go up the stony little path and they will be near the hut. The hut is just south of the cliffs/treeline on your map.

This is after I killed all the mage and templar quests.

I needed two extra gems and I got 1 drop each time I killed the group. So it's rinse and repeat depending how many you need. I visited Haven and went back for the re-spawn.

Even if you complete the quests, there will still be random spawns of bandits, mages, and templars. It might take some time before you can gather your quest materials though. Make sure you don't sell them to a vendor.