Yesterday at one of my locations, my firewall/router shutdown, (came right back up and still trying to figure out what happened), but in trying to figure out what may have caused this shutdown or crash, I see hundreds, if not thousands of:
Jun 6 17:06:19 jcits arpwatch: bogon 10.92.3.176 0:1f:16:f7:5b:6b
Jun 6 17:06:21 jcits arpwatch: bogon 10.92.3.176 0:1f:16:f7:5b:6b
Jun 6 17:06:23 jcits arpwatch: bogon 10.92.3.176 0:1f:16:f7:5b:6b
Jun 6 17:06:25 jcits arpwatch: bogon 10.92.3.176 0:1f:16:f7:5b:6b
…in my firewall log… pretty much filling it up. The MAC address logged is the MAC of my FreePBX! I have since turned off bogon logging.
I am using FreePBX 14.0.11 with Asterisk 13.22.0
WHY is my FreePBX sending out so many bogus requests? I have one other on the exact same version of FreePBX and exact same hardware. Only difference is this one producing these bogons is full PJSIP to internal devices and to my SIP provider.
You’re going to have to provide more information. Like what are you seeing on the server? Do you see this traffic being sent out? Where from? Have you done anything at the server level to see where these requests are being generated from?
Thanks Dave! That’s true and that is what is exactly why this is confusing me… why my FreePBX is requesting/sending out these ARP bogon requests! And, how to find where and why my FreePBX is saying it is on a private network of 10.92.3.176 when it is set to a static address of 192.168.1.165. Arpwatch is running on my firewall with a static IP of 192.168.1.1. I have searched the logs on the firewall as well and sysconfig files and this is driving me nuts!
I have checked the ifconfig file, searched through everything in sysconfig and I am stymied!
I don’t know where to look!!! Again this is the only system that is doing this… something is whacky somewhere and I don’t know how to track it down.
Have you done what I asked and gotten traffic from the PBX server itself? Have you run a tcpdump on the PBX system looking for these requests and what might be sourcing them? The PBX doesn’t have anything in any of its logs with this IP?
Yes in one case. I know for a fact that my FreePBX is the source of these requests. My firewall, through arpwatch has reported these requests from the MAC of my FreePBX. I have yet to do a TCPDump. In my FreePBX box, I haven’t scoured all of the logs yet… Wish I could focus in on one or two logs in particular.
But I continue to move forward as time permits. Just seeking time saving advice! Thanks!
A quiet Sunday AM so I VPNed into my remote network.
@Stewart1, I previously did a an ifconfig as you suggested and found nothing. I did do a netstat per your request and nothing found. Argh.
@BlazeStudios, Tom I did a TCPDump on the Freepbx and nothing at all was found going to or from that ghost IP. I rechecked the ArpWatch log and confirmed the reported MAC was in fact my FreePBX MAC address.
…arpwatch: bogon 10.92.3.176 0:1f:16:f7:5b:6b
I did a TCPDump from my firewall and it sees it: Here’s the output.
[[email protected] ~]# tcpdump -i any -c5 -nn -v src 10.92.3.176
tcpdump: listening on any, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
08:57:23.365952 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.92.0.1 tell 10.92.3.176, length 46
08:57:25.365947 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.92.0.1 tell 10.92.3.176, length 46
10.92.0.1, some mystery gateway I suppose, was not found anywhere. This IP range is in the private allocation range as I recall.
I am at a loss. Thanks for your help and suggestions. Any other thoughts on how to find this?
The PC that this FreePBX is running on is a Dell Optiplex 160 Tiny with an Atom 330. My network scan, as does your info, shows that the NIC is a Wistrom device!!!
My other FreePBX, local to me, is also a Dell Optiplex 160, same processor, but both both are supposed to have Broadcom NICs according to Dell specs and my local FreePBX shows as a Broadcom when I do my scan. However, the one giving me the bogons was ‘refurbished’ by who knows… or where. I’m cheap! Been running fine, but it is also interesting that after 3 years I am finding this weirdness out. The NICs are onboard not add-in cards.
I just researched it and read about it. At least the IP is not an Internet address. Must be trying to find a Winstron adapter in a router or a Winstron made router. Interesting… But why can’t it be foound with a local on the freepbx box using TCPDump. Would nmap find it?
Looks like a new piece of hardware and upgrade from 14 and try out 15.
arp is layer 2 which tcpdump wont see but wireshark could, the trap is sprung when it gets an ethernet response which triggers a layer 3 conversation,apparently you will find them in directtv boxes, alarm panels !, apart from routers also.
These ARP requests are broadcast packets and IMO are likely emanating from another device that is ‘spoofing’ the PBX’s MAC. It’s not necessarily malicious; something may be trying to perform some task on behalf of the PBX, but a misconfiguration is preventing anything from happening.
Try running tcpdump on another system on the 192.168.1.0/24 LAN. After you see these bogons, pull the Ethernet cable from the PBX (when no calls are in progress) and see if they keep coming.
Since they are being sent every two seconds, a 5-second interruption should be enough to confirm an external source; this is short enough to not drop registrations, etc.
Possibly your router/firewall is the source, e.g. 10.92.x.x. might be left over from an old VPN setup.