How to set Mac server to use different Gateway for internet bound traffic

Solution 1:

Give the second modem an address on the 192.168.1.0/24 network. You don't need a separate network for this.

On your Mac server, set the "default route" to the address of the second modem. With this configuration, your server will continue to talk to other systems on the network exactly as it does now, but for any off-network traffic it will use the second modem while other systems use the first modem.