Routing traffic to two NICs on the same network

Check, there are replies (maybe replies are going out through other interface) or replies are missing.

Check the settings of the reverse path filter (check counters in the output of 'nstat -az' or 'netstat -S' - there is TcpExtIPReversePathFilter for packets dropped by rp_filter). Disable it or set in loose mode (see sysctl settins description). Lookup the reverse route for incoming packets to conmirm the assumption.

I think you should add routes for directly connected networks into route tables, because it required by arp resolve of corresponded gateways and for communication to other hosts in directly connected networks. These settings should be enough to solve your case:

ip route add 10.5.68.186/31 dev eth0 table 1
ip route 0/0 via 10.5.68.186 dev eth0 table 1

ip route add 10.5.68.188/31 dev eth1 table 2
ip route 0/0 via 10.5.68.188 dev eth1 table 2

ip rule add from 10.5.68.187 lookup 1
ip rule add from 10.5.68.189 lookup 2

Also, you should know, what this setup is only for case, where the ip addresses on these interfaces with overlapsed addressing is different. Otherwise you should use more complex scheme with CONNMARK and pbr by firewall marks.

If you're trying to ping the host from host itsels, you should use these commands:

ip route add local 10.5.68.187 dev eth0 table 1
ip route add 10.5.68.186/31 dev eth0 table 1
ip route 0/0 via 10.5.68.186 dev eth0 table 1

ip route add local 10.5.68.189 dev eth1 table 2
ip route add 10.5.68.188/31 dev eth1 table 2
ip route 0/0 via 10.5.68.188 dev eth1 table 2

ip rule add iif eth0 lookup 1 pref 101
ip rule add iif eth1 lookup 2 pref 102

ip rule add from 10.5.68.187 lookup 1 pref 201
ip rule add from 10.5.68.189 lookup 2 pref 202

ip rule add from all lookup local pref 300
ip rule del pref 0