In theory, but in the real world admins (for example) set up trunks with from-internal context or test extensions with very weak passwords, or simply make mistakes. Numerous cases of SIP exploits have been documented in this forum.
There are many ways to gain additional protection, including avoiding SIP over UDP altogether, filtering by domain name (trivial but unpopular here, for reasons that make no sense to me), using various static or dynamic block lists (complex and IMO not very secure) and using nonstandard ports.
To estimate the ‘performance’ of running SIP over UDP on a random port > 10000, I spun up a test server on Vultr. Except for the SSH connection and a few spiders and monitoring sites, all of the incoming traffic is malicious. To amplify the traffic, I set up BGP and routed a presently unused /23 to the VPS. I naively thought that this would give me ~500x more attacks, but it turns out that many attackers scan addresses in numerical order, so you see the same attack 512 times. A little analysis showed that the useful gain is only ~10x over a single IP address, but that’s still a lot better than nothing.
I started running
tcpdump -w foo.cap udp dst portrange 5000-65535
and after a couple of days have a little over 100 MB of malicious high-port UDP traffic. Further filtering on ‘sip’ with Wireshark yields ~80,000 SIP attacks. The packet summary is exported as .csv for further analysis with simple scripts. One of the first was to see how many unique IP addresses attack various destination ports. Results:
5050 3
5051 2
5052 2
5053 2
5054 2
5060 347
5061 42
5062 4
5070 3
5080 4
5081 1
5082 1
5083 1
5084 1
5085 1
5086 1
5087 1
5088 1
5089 1
5090 2
5091 1
5092 1
5101 1
5102 1
5103 1
5104 1
5105 1
5106 1
5107 1
5108 1
5109 1
5110 1
5160 1
5285 1
5960 2
6050 1
6060 5
6070 1
6090 1
8050 1
8066 1
9060 1
9066 1
9340 1
15060 2
16392 1
25050 1
25060 2
35050 1
35060 1
36060 1
39490 1
43354 1
45060 1
55060 1
56060 1
61000 1
62597 1
65060 1
65476 2
Looking at the hits on ports > 10000, we see 15060, 25060, etc. I guess some people choose easy-to-remember alternate ports, and the attackers know that. There are 7 hits on ‘random’ ports. If we assume that those attackers are scanning all ports and we choose one at random from 10000-49999, the hit rate relative to 5060 can be estimated at 7 / 40000 / 347 = ~ 0.00005%.
Note that to get anything close to this performance, you must start with a ‘fresh’ IP address. If an attacker has ever seen a response to a SIP probe, even if subsequently blocked by fail2ban, they know you have a PBX and will scan every port on your system (from different IP addresses) to find it.
If a normal UDP SIP request does not get a reply, it is retransmitted several times, in case of lost packets. However, scanning all ports of the IPv4 space is very expensive and I believe that attackers doing that generally make only one attempt. We measured the number of packets for each combination of destination port, source address and destination address. Except to ports 5060 and 5080, there were zero retransmissions. Dropping the first packet received may be a useful defense. It would only slow down the initial connection, as you would allow established/related packets to always be accepted.