What is a patch panel?
What is a patch panel? How it is different from (layer 2) switch?
EDIT: I think my primary question (and confusion) is this:
Why don't network connections go straight to the switch? Don't we still require 1-to-1 connections from switch to patch panel?
splattne has covered what a patch panel is, and why it's different to a switch.
To answer the last part of your question: the reason that host network connections don't go direct to switches is generally to do with ease of management. For example, desk locations on an office floor can be cabled back to a wiring closet patch panel which is labeled with the locations. You can then connect short patches ('tails' or 'whips') between the patch panel and the switch. This makes re-patching desk locations (for user moves etc) much simpler, as the desk->patch panel runs don't need to be touched at all.
In a data centre, a similar argument applies. If a server needs to be moved to a different subnet that is on a different physical switch to the one it's connected to, having intermediate patch panels is very useful. For example, many server rooms have an MDF (master distribution frame); all servers and all switch ports are cabled back to labeled patch panels on this frame. Then, creating a connection between a server and a switch is a simple case of a patch between two ports on the frame, rather than needing to have floor tiles lifted to run a new end-to-end patch.
EDIT: To add a few sample cabling topologies:
1) User floors.
[host]<<--patch-->>[floor port]<<--structured cabling-->>[wiring closet patch panel]<<--harnessed/bundled cabling-->>[wiring closet access switch]
2) Data centres, centralised access.
[host]<<--patch-->>[cabinet patch panel]<<--structured cabling-->>[master frame patch panel A]<<--patch-->>[master frame patch panel B]<<--harnessed/bundled cabling-->>[data centre access switch]
Note in the above, you could have another cabinet patch panel in the switch cabinet; however when using large modular switches (240+ ports per chassis), providing that many patch panel ports tends to use up valuable U-space in the cabinet; hence why these connections are often directly harnessed back to the master frame.
3) Data centres, distributed access (end of row).
[host]<<--patch-->>[cabinet patch panel]<<--harnessed/bundled cabling-->>[end of row access switch]
This kind of topology is often used with blade deployments, as the number of blade chassis you have deployed dictates precisely the number of ports you need to provision. Note the reduced physical flexibility, however - hosts must be cabled to switches in the same row. Your logical network design should take this into account.
4) Data centres, distributed access (top of rack).
[host]<<--patch-->>[top of rack access switch]
Potentially useful where you have a very homogeneous datacentre with lots of nodes with identical requirements.
Note these are just some examples - there are plenty of other approaches as well.
This is a patch panel:
From Wikipedia:
A patch panel or patch bay is a panel, typically rackmounted, that houses cable connections. One typically shorter patch cable will plug into the front side, whereas the back holds the connection of a much longer and more permanent cable.
The assembly of hardware is arranged so that a number of circuits, usually of the same or similar type, appear on jacks for monitoring, interconnecting, and testing circuits in a convenient, flexible manner.
Regarding your question about what's the difference between a patch panel and a switch: a patch panel has no "intelligence." It's simply used for connecting cables.
The counterpart to a network jack!
If you have big networks, you use CAT5e & CAT7 wires which are thicker than patch wires. From the Jack to your notebook, you've to use a patch wire.
To connect every network cable to the core switch which comes from the network jacks you need a big "panel" to patch the wires to the switches. That's the patch panel.
The inner life of a patch panel is similar to a network jack:
Patch panels are there to deal with big masses of cables!