What would cause SIP traffic to be seen going into a switch but not coming out?
Background
I have been struggling to get my SIP phones to register behind a brand new router and switch in our brand new office. Our PBX is hosted offsite. I have worked with our provider to attempt several different approaches. We have tried regular NAT to connect to their NAT-aware session border controller. We have tried using siproxd (the pfSense package) to intercept the SIP registration requests and register on the phones behalf. Finally, we have tried configuring the phones manually to register with the siproxd daemon on my local network.
Throughout testing we have seen the phones do all of the following successfully:
- Contact the hosted FTP server by IP address
- Download the configuration from said server
- Perform DNS queries to resolve the IP address of the NTP server
- Query the NTP server to set the time
- Perform DNS queries to resolve the IP address of the SIP server(s)
Symptoms
After the phones have done all of the pre-registration tasks successfully, we never see the registration attempt hit the pfSense box, or the provider's PBX. I have enabled the highest level of debugging in siproxd on my end and have seen nary a TCP connection or UDP packet. However, a simple telnet to port 5060 from a workstation will generate expected log messages. Performing a packet capture on the pfSense box showed absolutely no SIP traffic attempts.
What the heck?
My final troubleshooting step that thoroughly stumped me and brought me to ask this question was as follows. I first mirrored the switch port that a phone was plugged into to my workstation switch port. I performed a packet capture of all traffic on the interface. To my surprise I saw SIP registration packets coming from the phone. Here is an example:
Clearly the phone is trying to register with the PBXs (those are the correct IP addresses as well).
My next step was to mirror the switch port that feeds into the LAN side of the pfSense router. I saw all of the FTP, NTP, and DNS traffic from the 172.200.22.102 phone coming out of the switch, but not a trace of the SIP packets. This is completely baffling to me! What is causing only the SIP traffic to vanish within the switch?
Environment
- Hardware
- Router/Firewall: Netgate m1n1wall 2D2
- Switch: HP 1810G-24
- Phones: Polycom SoundPoint IP 501
- Software: pfSense 2.0-RC3
Switch Configuration
The phone with IP Address 172.22.200.102 is in port 4 of this switch, the router LAN link is in port 22.
I can share any more settings that may be needed.
I found the solution after spending roughly 40 hours on this problem.
There is a setting in the switch that enables "Auto DoS" protection. Apparently it considers TCP or UDP traffic that has matching source or destination ports to be a blat attack and drops the packet. This is ridiculously short-sighted since SIP traffic often (always?) relies on source and destination ports being 5060.
In case a textual description was insufficient: