Does iPhoto need to be open in order to download Photo Stream photos?

iPhoto doesn't have to be open. There's a background process running called iCloud Photos in Mavericks that runs as a system daemon that's taking care of bringing down your Photostream photos.

The photos are downloaded to ~/Library/Application Support/iLifeAssetManagement/assets/sub on your Mac. You can press Cmd-Shift-G in Finder and enter that path to browse the location. You'll see a lot of funny named sub-directories and in them: your Photostream photos. I don't recommend messing with this structure though.

What doesn't happen without iPhoto running is your photos getting imported in to iPhoto and organized in to events. So when you open iPhoto it's discovering new Photostream photos that have downloaded since the last time it ran and then importing those photos to bring them in to your iPhoto library. This is where the delay is coming from.

If you just want to view your Photostream without having to endure iPhoto you can try MyPhotostream which gives you a simple, read-only way to see your Photostream photos without having to launch iPhoto. It doesn't make iPhoto any faster on start, but if you need to get at your Photostream photos often it can make the process a little less cumbersome.