How to configure HAProxy to route by port without using multiple frontend or listeners

Using a single frontend and multiplexing to the backends is indeed possible. You will be required to add one backend section per backend server, e.g.

backend JabberD1
    tcp-check connect
    tcp-check send PING\r\n
    tcp-check expect string +PONG
    server Server1ZoneD 10.60.30.209:80  check

backend JabberE1
    tcp-check connect
    tcp-check send PING\r\n
    tcp-check expect string +PONG
    server Server1ZoneE 10.60.47.4:80    check

etc.

In your frontend, split the traffic like this

    acl d1 dst_port 5222
    use_backend JabberD1 if d1

etc.