Registering multiple web sites as subfolders in the same azure app service

You can achieve this in Azure using tools like Azure Front Door or Application Gateway (I would recommend Front Door). This will allow you to configure your required URL's at the front end, and map those to path based rules at the back end.

You can use CNAMEs to point your clients to the right service, for example 'cms.client1.com CNAME supercmsservice.com' You of course need to configure the webserver at supercmsservice.com to listen for requests for cms.client1.com

As for the SSL certificate, you'll need to add the names of the services ('cms.client1.com' in my example) as Subject Alternative Names in your certificate when you request it. Some SSL certificate vendors have different pricing for Multi-SAN certificates, so look into that before buying, as it makes it a lot cheaper compared to some other vendors that charge you by the SAN regardless if you request 10 separate certificates or 1 with 10 SANs.

If you're using Let's Encrypt, it's just a matter of adding more domains to the request.