Installing SSH Server on RHEL 8.4
It turns out that Red Hat Universal Base Image repos for their Docker images simply do not include openssh-server. I wasn't able to find a statement from Red Hat as to why, but given how extreme an antipattern it is to run an ssh server in a Docker container, it does make some sense.
If you have a paid Red Hat subscription, you could open a case with Red Hat and ask about having it added. But it's probably better to get rid of the ssh server entirely.