What's the most efficient way to select workers to build things in Starcraft 2?

So obviously selecting 1 is better than a group, but a little known fact:

Workers smart cast!

This means if you select a group of workers and try to build something, the nearest worker (and only him) will go to build the building. Additionally if you build more than one thing, the two nearest will go to do the task. This means you don't need to worry about selecting more than one (provided you don't issue any move orders), just select the group and build away!

Don't select all your workers and use smart cast, make sure you only grab workers that are on minerals. You don't want to pull a worker from gas and then later put him back on minerals. You'll be short a worker on gas, which is very easy to miss.

In the early game, when there isn't that much to do, you should get used to selecting a worker individually on his way back from mining, then watch him drop his resources off, and then tell him to build. This is better than smart casting. Later on when you've got a lot to do this'll be way too time intensive, so at that point you can switch to using smart casting.

Also, tell your worker to build the building, and then shift right click on your minerals. That way he'll go back to mining the moment he's done, which means no time is wasted.

No, don't have any builder idle if you can help it. Really I just single click, I know you tried this but I would guess that is the best way to get just one. Just spam click if you have to until you get one, eventually you'll get better at aiming anyways.

Another note: If you hit F1 it will select a worker who is doing nothing!

I do the shift-cntl f1 at game beginning to select all probes and send them to mine.

Then I select 1-2-3 probes and put them in groups 1 2 and 3, which serves as builder, expansion, scout as the game goes on.