How do I connect multiple devices with powerline adapters?

So I get how a switch would allow you to connect multiple devices in a one location to a single powerline adapter, but I'm not sure how to connect multiple devices in different locations. The part that I'm confused about is how these devices connect to the router. Presumably each device has its own adapter, including the router, so how can it route between the devices if they are all coming through a single adapter into a single LAN port on the router?

My scenario is that I have a TV in the bedroom, a TV in the living room, and some computers in a office. All of these are in different rooms from the router/modem. Is this scenario even compatible with powerline networking?

Yes, it's compatible with your scenario (designed for it, even).

The conceptual change you need to make in your thinking is that powerline networking is more like a hub, not a switch. That is, everyone is plugged into the same stream of data, and MAC addresses are used to tell which frame is going where. It can get chatty and data can collide, which is why a switched network is so much faster (and safer).

Before router/switches were so cheap, hubs were much more commonplace. Here's a good article explaining the difference.