Direct ethernet link between two servers

Say I had two servers which needed super-low latency (Database, file etc.). Would it be possible to directly connect the two servers with 10GbE, so each server had 1 (in the real world it would have 2) connections to the 'main' network, but 1 network card with an ethernet cable that connected directly to the second server, no switches or routers, just a direct connection

                         Internet/Datacenter
                                 |
                                 |
                                 |
                                 |
                                 |
                                 |
                                 |
                        --------------------
                        |                  |
            ------------|      Switch      |-----------
            |           |                  |          |
            |           --------------------          |
            |                                         |
            |                                         |
            |                                         |
            |                                         |
            |                                         |
            |                                         |
            |                                         |
  Network Card 1 (eth0)                     Network Card 1 (eth0)
            |                                         |
  --------------------                      --------------------
  |                  |                      |                  |
  |     Server 1     |                      |     Server 2     |
  |                  |                      |                  |
  --------------------                      --------------------
            |                                         |
  Network Card 2 (eth1)                     Network Card 2 (eth1)
            |                                         |
            |                                         |
            |               Direct 10GbE              |
            -------------------------------------------

My first question is, would this even be possible? Would they need any unusual/special services configured to let them talk over this network other than a standard file in /etc/sysconfig/network-scripts/? They would both have static IPs on eth1 but how would things like routing work? I'm not an expert on networking so this is probably a n00b-ish question

Second question, is there any point? Would there be any advantages doing this over just letting them communicate over the standard network connection via the switch, or giving them a second dedicated network just for communicating intra-server (Since bandwidth would be used on the standard network by clients accessing the servers). Assuming latency was the priority.

I know there are some issues with this method, like when we came to add a 3rd server we'd ether have to give every server another network card and probably set up some very complicated replication triangle thingy but since this is hypothetical lets ignore that.

And since latency is the key issue, would fiber be better over ethernet (Speed isn't important so long as it can do a couple of Gb/sec)

I phrased this question from a linux POV, because that's my background, but it could apply to any server/device

Solution 1:

There's no reason why you technically can't do this.

I'd probably do something similar, under the circumstances, actually. From a purely linux point of view, it's really easy, just give the connection an IP address with a /30 bitmask, giving you 2 IP addresses, then it's a simple Point-to-Point link.

If you wanted to grow the network, you could get a 10GE switch, and then have a seperate VLAN for traffic between servers. There's some very shiny gear in the Force10 range of switches that can do line-rate 10GE switching, with enormous buffers.

Solution 2:

I cant comment a Linux point of view but I will just use my knowledge and ask some more questions.

Are you that dependant on low latency and that you need to keep these servers in sync? Are they both running a database or something? 10GbE should suffice most needs of 2 servers to keep them in sync. I would soon rather spend the money on a decent switch in the middle of the 2 instead of going the route you are looking at.

You could with a decent switch tag these ports to prioritise the traffic and even QoS the traffic that needs to be real time.

My thoughts.

Solution 3:

I have actually done this between two laptops. Most modern LAN adapters have auto negotiation between them so you can use a regular lan cable.

Set static IP addresses that are not in the same range as any other subnet you are using - for example, if my systems are on a 192.168.x.x subnet, i use a 10.0.0.x subnet between them. Otherwise, it should just work