Effecient organization of spare cables and hardware
We always end up limited by either available space or budget, but I guess my ideal setup would be:
Big industrial racking (the metal struts and reconfigurable wood shelves kinda deal like you see in warehouses) for holding large hardware, servers, switches, monitors and workstations. This stuff is awesome if you have the cash and space to house it. IMO it should be standard setup for an infrastructure team to have some shelving like this available, but since some companies won't even give you a dedicated server room, getting a storeroom is sometimes unlikely. Try and keep your servers and the rails that fit them in the same location. Hunting around to find you're missing a single rail mount for a particular server is no fun, and pricey to re-buy.
Wall-mounted hooks from which cables are hung in groups - Coiling cables always leads to a mess and wasted time uncoiling them. If you have the space for it, hanging the cables straight down on a wall lets you see the length and keeps them relatively untangled for quick grabs.
Plastic storage bins for small components. Those modular dump-truck-bin shaped ones that will either standalone on a shelf or stack together to save space. Toolboxes are overrated and never seem to have the right size compartments for all those unique IT widgets. Save the sealable plastic baggies you get with new workstations and use them for stashing bits and pieces as needed.
I still haven't figured out a decent way of handling physical media that'll scale for a whole team. We had one of those fancy automated CD storage units at one point... useless. It required having software installed on a nearby PC to operate the thing and just made it harder to access the media you needed. Plastic storage wallets are pretty naff when you end up flicking through for 10 mins only to find that the particular version of smartstart you wanted hasn't been returned to the folder. I did find you can re-purpose the HP cd packs you get with each server by turning the sleeve inside-out and doubling up the number of sleeves inside it... then you can create a grab-bag of build disks and stash it in your drawer. At least that way you can maintain your own collection in a format that's easy to keep with you.
We have a ton of shelf space along the back of the server room - at least 12' wide and about 2' deep. Since manuals are largely a thing of the past, we use it for pretty much all our storage. Banker's boxes fit very nicely edge-wise on these shelves - it's easy to grab a box by the handle and pull it out - so we use them for lots of things.
Misc cables are sorted into three boxes by length - short, med, long. Whenever we buy misc. cables, they come in zip-lock bags, so we keep those and use them for any leftover cables.
We've also got a box of various serial cables (which we don't use much any more), a box of USB and firewire cables, a box of fiber patch cables, and a box of weird cables - e.g. a SCSI2 to right-angle VHDCI that cost a bundle when we got it so I can't bear to throw it out.
For each server, we keep all the odds and ends that came with it in magazine boxes. This includes all the little booklets, any special cables that we didn't need, screws, anything. This box (or multiple boxes, if necessary) is where we keep any files relating to the server, e.g. setup notes, printouts of license documents, CDs that came with the server. Basically, everything for each server is in one place. For one server we built ourselves, this included a spare power supply, a spare system disk, the spare RAID card we bought with it. Chris mentioned physical media, we deal with this by (mostly) having all the disks we need for each server stored with everything else for the server. So we buy extra media sets and/or burn CDs and DVDs to that we never have to search very far.
We have just two racks, 42U, and they came with a ton of caged nuts, extra name plates (one rack is Compaq, the other was bought after the merger and says HP, they both came with blank plastic plates you could put on if you wanted your own logo). All that stuff is in one box, along with a bunch of 10-32 machine screws that fit the couple older racks and the relay racks for our satellite patch rooms.
Spare switches and possibly useful equipment sits on the shelves. Extra rack rails are one thing that doesn't fit nicely, but there's a spot in the corner where we can stand them up. We avoid having to store too much oddly shaped stuff by ruthlessly throwing things out. (Ok, we still have an AlphaServer 2100 4/275 sitting under a desk, but that space isn't useful anyway.)