Does the Mac still only have one hardware video decoder for the first video, and then it will be software decoder that is used?

Yes, depending on the specs it often does have only one hardware media decoder. This is either integrated in the CPU (Intel processors) or on a graphic card. (Note that some graphic cards can have more than one hardware media decoder / encoder).

But that hardware decoder can play multiple streams of media, upto a limit, before the OS is forced to switch to the CPU (software rendering).

The limit though depends on the hardware decoders capability. For example, if a particular hardware media decoder claims it can decode a 1080p H.264 videos at 600 fps, then you should theoretically be able to play ten 1080p videos of 60fps (600/60 = 10) simultaneously on it.

Ofcourse, all this depends on the app playing the media and even on the OS.

For example, on Google's Chrome browser Youtube will often send you a VP8/9 encoded video or a H.264 encoded video depending on whether your laptop is running on battery or main power (H.264 consumes less power than VP8, so if you are running on battery, Youtube on Chrome will opt for H.264). If your hardware decoder can't decode VP8/9 the OS will switch to CPU software rendering. Many media players have an option to explcitly enable Hardware Acceleration (to decode videos through the hardware decoder), and if this is not enabled, then the media player app will use the CPU (for software decoding) even if your computer has a hardware decoder.

It's not that easy to test as there are many other such factors at play too - can you open 10+ videos? Is your hard disk or internet fast enough to all play 10+ videos? Will the OS hang? And so on ...