ADSL modems directly connected

Is there a way to connect two ADSL routers together directly, to each other's ADSL port?

No, each ADSL modem/router that you have is a slave (or remote) unit. They cannot work together (as you already suspect).

Communication links like xDSL are point-to-point but have asymmetric topology, that is, the modem at your end (called the remote unit or CPE, customer-premises equipment) is a "slave" unit, and the modem/DSLAM at the ISP or central office end (called the line unit) is a "master" unit. (Do not confuse or conflate the asymmetric topology with the asymmetric connection speeds of ADSL.)

In order to use the ADSL link, the line and remote units first have to perform a predefined series transmissions and responses (called training) to evaluate line conditions and set transmission rates, power levels and equalization. Only after training is complete can the ADSL link be used to transmit/receive data.
Bottom line is that you cannot use an ADSL modem only built for use as the remote unit in place of the ADSL line unit.
You don't need a entire DSLAM (a multiplexer for many channels), but you would need a Line (or master) ADSL unit (e.g. Ethernet on one side, ADSL on the other).

Assuming a server is required, what kind of system can I build to provide an ADSL service to routers?

Rather than ADSL, you might be better off with a symmetrical connection (i.e. upstream and downstream speeds are the same), such as a point-to-point VDSL setup.
See How to network two locations through 130m of trees

BTW -- Your use of "server" and "client" are inappropriate. xDSL modems use the master/slave model, not the client/server model.

What kind of system can I build to provide an ADSL service to routers?

You need at least a DSLAM:

A digital subscriber line access multiplexer (DSLAM, often pronounced dee-slam) is a network device, often located in telephone exchanges, that connects multiple customer digital subscriber line (DSL) interfaces to a high-speed digital communications channel using multiplexing techniques.

There will be other bits of infrastructure required as well.

For example, you will something behind the DSLAM that will route traffic between your two devices.

What you are trying do is to become your own ISP ...