PIX concurrent source IP addresses limit
One of my branch offices uses Cisco PIX 501 to create a site-to-site VPN with the main office. This branch is a DR site that run a Domain Controller, a file server and 10 Windows XP clients.
During my last visit to this site (which no one ever uses) I noticed that one of the PCs could not access the internet. After a quick review of the connection I found that all the local resources like file server or RDP are accessible, I could access shares, RDP or PCAnywhere to\from local machines and use the network printer.
My Documents redirection failed as the files are (still, I’m in a rebuilding process) in the main office.
I also noticed that PCAnywhere or RDP to\from the main office to this machine fail while other machines within this remote branch had no issues.
When I tried to access remote shares in the main office, browsing failed.
Internet connectivity was down too, I tried both IE and Firefox but both couldn’t connect. Testing to a local web page worked and other machines connected any web site easily. I verified that Proxy is not configured on IE or Firefox, this was not the issue.
One of the things that blew my mind was TCP/IP and RPC troubleshooting results:
Both ping and tracert worked both ways to\from the main office.
DNS had no issues, I could resolve both Internet addresses and main office names on the faulty machine.
Using gpresult command I verified that domain related features like Group Policy did apply to the local machine.
This machine like all PCs in the site uses DHCP. I verified the configuration using ipconfig /all but I knew it is identical since they all use the same DHCP server.
I have to admit that I did not suspect the firewall right away because ping did work but at this point I had nothing else to check and no one else to blame so I started poking around the PIX configuration and documentation. It took me a while but eventually I found the piece of information that solved the mystery:
The Cisco PIX 501 10-user license supports up to 10 concurrent source IP addresses from your internal network to traverse through the Cisco PIX 501
Doh! so simple, so obvious and yet I didn’t even think about this option…
The next step, confirming that this is the real issue was easy. I used the show local-host command to see which show all active connections:
show local-host
Interface inside: 10 active, 12 maximum active, 3967 denied
local host: <192.168.199.54>,
TCP connection count/limit = 1/unlimited
TCP embryonic count = 0
TCP intercept watermark = unlimited
UDP connection count/limit = 0/unlimited
AAA:
Xlate(s):
Conn(s):
TCP out 192.168.200.19:445 in 192.168.199.54:3383 idle 0:00:48 Bytes 288161 flags UIO
The complete output show the same details for ten different internal IPs. The PC that could not connect the internet was not on the list. It exceeded the limit and was denied.
The command show time xlate provided the other part of the answer, it shows the timeout settings:
timeout xlate 0:05:00
I used the clear local-host command to reset the PIX local-host table and immediately was able to browse from my machine. Problem solved (kind of).
The reason I did not notice this problem in the past is that no one uses this office. The empty office with few PCs that hardly communicate with the main office and never use the web could not raise an alarm. Being there and checking all the PCs (login, updates, etc) created traffic that did not get idle and prompted the issue.
Lesson #1 – When you support a low-cost DR site make sure you buy the right equipment. I use an old (very old) PIX as we tried to cut costs. We used an old PIX that had no use but what we saw as a great deal turn out to be a problem.
Lesson #2 – Make sure you visit your DR sites and test them. Try to simulate the real activity that your DR situation require and see if it is functioning as expected.
