Which buildings have lots of ingredients?
Solution 1:
Finding ingredients in Skyrim is easy. Just go adventure and be perceptive. Ingredients can be found almost everywhere. I usually had to stash all my ingredients in a container next to an alchemy lab whenever I visited a place I owned because they weighted so much.
Below is a description where you will find the paralyze ingredients:
The Swamp fungal pods and Canis root ingredients grow in the Hjaalmarch. You can pick up a lot of ingredients by circling the region every once in a while. Imp stools are found throughout the caves in skyrim. There is a large amount of imp stools in Chillwind Depths, a cave to the south of Dragon Bridge near Hjaalmarch.
Human flesh and briar hearts are rare ingredients. Necromancers use the flesh and briar hearts can sometimes be found in alchemy rooms throughout skyrim. You can also kill Forsworn briar-hearts but they are uncommon and tough.
Gleamblossom can only be found in the Darkfall passage and Forgotten Vale but it's a long trek through an army of falmers.
Finally to acquire netch jelly, you need to fight the netch creatures which can be found along the western and southern coast on Solstheim.
Solution 2:
Just to add to Steinin's great answer:
If you have the hearthfire DLC, you can also plant imp stool and swamp fungal pod in a greenhouse or the garden. This makes collecting them MUCH easier. It also respawns quite fast (I think they respawn every 3 days).
Finally, if you don't mind mods that assist in this area, I suggest the Harvest Overhaul mod. It makes more sense to have the chance to collect 2 eyes from a sabertooth and 5-10 flowers from a clump of them than a single ingredient.
Also, keep in mind, to make your alchemy go up the fastest, you'll want to make the most expensive potion so this question may be of some help.