Connect two networks to a single server using its two ethernet ports
I have two separated local networks (two routers, each is connected to its ISP for the internet connection) and a server with two ethernet ports. The server is running a local website service which I would like both networks to be able to access.
Can I just simply plug both ethernet ports of the server to each router port? Which ethernet port will be used for the internet connection on the server itself?
Note: the server is running Fedora 20 and apache.
You didn't specify IP version and if any NAT is in place. I am guessing each LAN is behind a NAT and has only IPv4.
Plugging the two Ethernet interfaces to the different networks will work as long as the two LAN are using different IP ranges. There is a significant risk that both LANs are using the same range of RFC 1918 addresses. (RFC 4193 introduce a method to avoid such conflicts, but that is for IPv6 only.)
If the two LANs are using the same IP addresses, you first need to reconfigure one of the routers, such that they no longer use conflicting addresses.
Once you have ensured that the two LANs use non-conflicting addresses, you can plug in the server. You should configure the server with static IP addresses on each Ethernet interface. The IP address obviously need to match the LAN it is connected to, and it need to be assigned an IP address which is not in the range used by DHCP.
Each LAN has a default gateway. On the server you only configure the default gateway on one of the interfaces, that will be the one it will use for internet connectivity. If you want to, you can configure individual routes, that will let the server reach some remote destinations through the other router.
One caveat to notice in this setup is that the clients on the two LANs will need to use different IP addresses to reach the server. You can get around that if you can configure a static route on one of the routers.
For example if the routers are 192.168.1.1 and 192.168.2.1, and the server has IP addresses 192.168.1.2 and 192.168.2.2, you can configure a static route on 192.168.1.1. That static route will say that in order to reach 192.168.2.2 the gateway to use is 192.168.1.2.
If configured like that, then clients on both LANs can reach the server using 192.168.2.2.
I can't add comments, but in addition to kasperd's response, you can also set your connections up in a failover or a bond them. The method of doing this will differ depending on OS. In the case of failover, you would ride your primary path to the internet and only use the failover if the primary is down (there are some other more advanced things you can do like scripts for latency checks to take down the primary path if it passes a threshold) or ride both concurrently in the case of a bonded channel which inherently has a failover as it continues to work when one link in the bond fails.