AWS EC2: SSH security group inbound rules do not work

I have the following setup:

Instance A:
security group sgA
sgA inbound rules contains:
  Rule1: * All traffic - All protocols - All port ranges - Source: sgB
  Rule2: * SSH - TCP - port 22 - Source: 0.0.0.0/0

Instance B:
security group sgB

Both security groups have 'All traffic' enabled on 0.0.0.0/0 for outbound connections.

With this setup I can ssh from Instance B to Instance A. However, if I remove Rule 2 (the 0.0.0.0/0 rule), ssh now hangs and the machine does not respond to pings.

Am I misinterpreting how ssh access should work based on security groups?

When you specify a security group as the source or destination for a rule, the rule affects all instances associated with the security group. Incoming traffic is allowed based on the private IP addresses of the instances that are associated with the source security group (and not the public IP or Elastic IP addresses).

In your example Host A would allow Host B on port 22 from Host B's private IP address. If you are using Host A's public IP address when you attempt to SSH from B, Host B will use either a NAT Gateway IP or its public IP. Because those IP's are not the private IP address they will not be allowed access.

I believe to get your configuration to work, you'll need to either:

1. Use the private IP address when SSH'ing to Host A or
2. Add the public IP address of Host B (or the NAT Gateway IP) to the Security Group source instead of (or in addition to) B's security group.

Reference

EC2 Security Groups