What is the most cost efficient way to increase storage on an Xbox 360?

The cheapest option is definitely going to be the USB route as mentioned by others.

Ignoring the fact that Microsoft's hardware prices are inflated and overpriced, if you go strictly off a cost per GB, then buying the console with the 250GB drive is your best bet for the money. Using the averages prices these days

GB    Total $    $ per GB
250   $100.00    $0.40    (w/console)
250   $130.00    $0.52    (standalone drive)
120   $100.00    $0.83    (standalone drive)
60     $70.00    $1.17    (standalone drive)
16     $20.00    $1.25    (USB Drive)

I would assume the standalone drives have a warranty of their own, and the one with the console would be part of the console warranty, but I'm not sure on those.

You can always use a USB hard-drive as a cost efficient solution to the lack of space. Two things to note with this. While this is the cheapest solution to upgrading hard-drive space consider these: You can only do a 16 gig thumb-drive for space (xbox will not recognize anything over this size), it often leads to space gaps where you have a 5 gig game to be downloaded but only 4 gigs available on each thumb-drive instead of one large hard-drive where continuity isn't an issue, and finally your xbox will look really hack-ish with all the thumb-drives sticking out of it and utilizing all your ports.

Or upgrade your console with a bigger hard-drive (though its important to note that the slim xbox uses a different hard-drive then old xbox making it harder to find cheap hard drives for it.)

I can speak from personal experience that my life was considerably improved upgrading from the Xbox Arcade to the Slim. If you have the money save yourself the headache and skip the low cost models, if you don't have the money yet it might be worth saving an extra month for it.

Replacing the hard drive has no effect on the warranty at all so if you choose at a later time to upgrade the hard-drive you can.

By far the cheapest way to get started (though it does not exactly scale) is to invest in 16 GB USB flash drives -- the maximum size the Xbox 360 will support.

You can use as many of them as you have USB ports, and your USB ports can be expanded with a hub. These go for under $20 on Amazon currently and will continue to drop in price over time.

So for about $35 you could have a 32 GB "hard drive" split across two devices. This compares to $110 ~ $130 for the 250 GB hard drive, ~$100 for the 120 GB hard drive, and ~$70 for the 60 GB hard drive.

The 4gb Slim isn't even considered a "valid Xbox 360 HDD." It's flash memory, just the same as adding the 16gb thumb drives. While you can download some arcade games, demo's, and DLC from the marketplace, some older games (eg. Mafia II w/ DLC disk) won't recognize either your console memory or thumb drive memory when you try to install the DLC disc. So I would just be careful because there may be some games you get with DLC disks that you won't be able to install.

I believe the only option at that point is buying an "S" hd. It's a pretty bad decision on Microsoft's part to do that, and even to the point of not pointing this out on the box that the console comes in. I'm pretty sure there's some legal issues there, but I'm sure they have it printed somewhere on the inside of the console just in case they ever got sued over it.