Does a mountpoint have to be an empty directory?

No it does not have to be an empty directory. You can "overmount" it, but be aware, that you hide the files originally in that directory.

This can lead to some confusion since the space on the underlying filesystem - not the newly mounted one - stays occupied.