How can I show the Cover Art window in iTunes 11?

To get the second window back, all you have to do is click the album artwork thumbnail in the Now Playing area, either in the full version:

Or in the Mini Player

Both will pop up the larger album art view you are looking for.

In iTunes 11.0, there currently isn't the sidebar album art view that used to be there.

My mistake, I've found that from mini-view, clicking the thumbnail of the current track opens the floating viewer I was looking for.. I was fooled because iTunes 10 had the little drawer that would allow selected tracks' covers to be shown. It seems that this view is only visible for the playing track..