All connections from this network get stuck in SYN_RECV state, connections from my home or phone properly get ESTABLISHED
To this jaded eye, it looks like there is some kind of routing issue close to the server in question. Packets come in along one path, but seem to depart through a different path and something stateful is on that path and dropping the weird "ACK without a SYN" packets.
I had this happen to me once. What ended up being the case was that the server had a bad network mask, so when traffic from off the subnet came in, it would issue an ARP request to get the MAC address of the node. Unfortunately for me, both the router and our load-balancer were enabled for Proxy-ARP, and the load-balancer was a bit faster on the trigger than the router. So the SYN packets came in via the router, but were attempting to leave the subnet via the load-balancer. As the LB didn't have a connection for that ACk packet, it dropped it on the floor.
In your case some judicious trace-routes may illuminate the network-path issues. From the affected server, attempt to traceroute out to the IPs that cause the problem, and do the same from those same IPs. If you're getting different paths, that may be where it is.
I've been in a similar situation using Javalin as a server.
I was not using any firewalls, netstat was showing the local device trying to access local server with SYN RECV status, ping not reaching local server.
I had set the IPv4 manually in local machine, undoing that and setting ipv4 method to automatic (DHCP) again did the trick. After that I was able to access local server from other local devices.