How to browse a .zip file in IntelliJ (or .jar, etc.)

Jar files visible in your project structure that cannot be expanded have not yet been "added" as libraries. Right click the jar file and select Add as Library...

Also see Configuring Module Dependencies and Libraries.

And: Correct way to add lib/*.jar to an IntelliJ IDEA project

There is now a Intellij Plugin that solves that:

https://plugins.jetbrains.com/plugin/9491-archive-browser

Update: it works now also for nested archives. (tested 2019.1)

While Christopher Peisert's answer works for archives that aren't nested (you are out of luck if you have an EAR containing WARs) it is tedious that for most archive types (JARs being the exception), you have to add the file as a library or a dependency from the Project Structure area of Settings.

If you want to see archives be the default operation and work for nested archives, please vote for this YouTrack Issue 126376