Having trouble getting beyond about 80 population in Banished
Solution 1:
Odd as it may seem, try starting on hard mode. Building all of your buildings where you want instead of in relation to the initial deployment can allow for a much more efficient setup. All the free stuff in easy/regular mode is nice, but poor initial placement of your free storage/houses can really hinder your game.
Keep several laborers free at all times, and don't build too much. You need laborers to haul goods around, and having a half dozen means your workers have more time to do their actual job instead of hauling. If I've got an ambitious building project I'll designate everything but then immediately pause all but one job at a time. As each individual building completes I'll evaluate whether I want to unpause the next one immediately or wait for resources to stock up, hauling jobs to complete, etc.
Solution 2:
Focus on your weaknesses - starvation and hypothermia. Make sure you have more than enough food and firewood - not just enough stored up, but enough produced so that you will never have a deficit. Once these two basic needs are met, make sure you are supplying enough tools to not disrupt the flow of food and firewood. Everything else, a shortage is only a minor setback.
Once you are confident you have enough food and firewood production, expand your population. If you have suddenly too many laborers, that's great! Build a school so your production will become more efficient over time.
As to your other question: The rabbit-hole professions (those that are not dependent on their surroundings, i.e. gatherers and foresters) do not seem to have diminishing returns. One worker in a quarry is not ineffective, other than the fixed cost of the building materials not being replaced very fast. However, if you are building these too soon, this could be the source of your food shortage.
Solution 3:
Beware this wall of text offers no TL;DR
Food... food. Food! > Fuel > Housing > Tools > Clothes
Set food max at half a million and never look back; this is what you trade and what keeps you alive almost no matter what (build more than enough barns to store it, and then some).
Farms should be 5x9 and worked by a single worker, paying special attention to orientate them so that when they randomly collect the harvest, they're only going back and forth on those 5 squares. Whatever dimensions you pick, should be a rectangle with this in mind.
Multiple fisheries interfere with each other, work best with the more water their 'circle' covers and with the maximum four workers. So just have one per area, keeping it fully employed throughout the year. They really help even-out the winter months.
Always have a wood cutter and foresters in operation. Always. IMO, there's only two things that will make you lose; lack of food and fuel. These two products are never something you want to be in a panic to get. Tasked, but not working anymore, workers revert into laborers, so don't worry that their times being wasted.
Try to keep a collect resources area in effect, whether or not you're employing actual laborers (of which there should always be a few, to automatically replace the deceased), so that there's always something to be done by the idlers.
Everything except tools and clothes gets set to at least 2000; those I set to 200~500 (anything higher takes up space unnecessarily). When their quota has been met, they're free to be re-tasked, just keep an eye on the stock.
Don't be afraid to send laborers way out there to get stone and iron, you don't have to carry it. One of the first things I do is build the four main roads out of town, as far as they'll go, dropping stockpiles along it every so often.
Any building that requires or produces products, should have a means of storage for those items directly adjacent to it; less walking. This is key for fisheries, as the best spot is usually a bit out of the way. Build a marketplace ASAP (employing one person per ~50 population), to keep everything flowing and your workers working, not walking around making deliveries.
Obviously you need housing and if your over-population dies from the lack of it, so be it; you weren't ready for them. Try not to accept nomads if you don't have a ridiculous amount of food stored and houses that are only half full. You need a buffer until they get settled and start contributing. Nomads are also the primary reason tools and clothes are set way above the initial 50; they'll bleed you dry of them.
The caveat being, don't ever run out of tools or clothes; people will die in the winter and work slow throughout the year. This leads to the poor production of everything. By the time you can make steel tools and warm coats, you'll have it made.
Don't be tempted to fully employ the mines even though you can throw lots of labor at them. You should have completely exhausted the map's resources before you really need them. One guy mining coal is enough for me and two or three getting stone or iron, if I can spare them. When the game gets going, this is where you can throw the fifty new people who just showed up, until you get them real jobs.
3~4 (5x9) farms is not enough, you need more like ten. I'd say at least a fifth of your population should be farming, on top of those who do husbandry, fishing, ect. Orchards should only be counted on for variety, not sustenance.
The 'outposts' consisting of a gatherer, hunter, forester and herbalist should also have one or two houses and a barn.
Solution 4:
Do you have markets up? I know your villagers will waste alot of time if they have to get resources to their house instead of resources being at their house. I usually have 1 Vendor assigned per 50 houses.