Creating different quotas for specific machines with Time Machine?

AFAIK there is no option to limit backup size on the client (macOS) side of things. For server-side limits two options come to mind:

• create individual partitions for each Mac to be backed up and size accordingly (on Macs you could also use sparseimages but I don‘t know whether these are available in your setup)
• create individual users on the server for each Mac to backup, and assign disk quotas with whatever Linux tools available

PS: I know that the options listed lack the specific commands. But AD is focused on Apple products so solving Linux side problems is out of scope. If you need help in implementing one of the options it might be better to ask a specific question on {askubuntu,unix}.stackexchange.com.