Many computers, 1 set of headphones. What are the options?
Here's a 4 channel stereo passive (unpowered) mixer, USD 50 at Amazon:
If you can deal with switching between sources, there's a $5 option, 4 input stereo passive selector:
There's also the daisy-chain option. Since you'll need a bunch of 3.5mm-3.5mm stereo cables anyway, you could run them from one computer to the other and just plug your headphones into the last one.
This assumes you have a stereo line-in on your computers.