haproxy 1.5 specific source IP address show <NOSRV> 503 SC in haproxy log

See the SC-- in the log entry?

This is a problem with your back-end server, or a network issue between HAProxy and the back-end... not on the front side, and not related to the IP address of the connecting client.

From "Session state at disconnect" in the docs:

SC The [back-end] server or an equipment between it and haproxy explicitly refused the TCP connection (the proxy received a TCP RST or an ICMP message in return). Under some circumstances, it can also be the network stack telling the proxy that the server is unreachable (eg: no route, or no ARP response on local network). When this happens in HTTP mode, the status code is likely a 502 or 503 here.

http://cbonte.github.io/haproxy-dconv/configuration-1.6.html#8.5