Toll Free Misc. Dest. Not Working?

Server: FreePBX 13.0.25 with Asterisk 13.6.0 (and have tried with version 11)
VSP: AnveoDirect
Network: Cisco ASA 5510, site-to-site VPN’s to SonicWall’s

Strange problem recently started occurring for us. We have several DID’s that have inbound routes that go to Misc. Destinations which are set to outside DID’s (people’s cell phones, other offices, etc.). Well, what started happening sometime over the past week was that certain ones give a busy sound when dialed. I’ve noticed that only toll free numbers are having the issue.

Below is in reference to me dialing our DID of 19099291802, with an inbound route and misc destination of 8003356500.

Here’s what Asterisk says with “-vvr”:

[2015-12-22 12:30:32] WARNING[5298]: func_cdr.c:352 cdr_write_callback: CDR requires a value (CDR(variable)=value)
== Using SIP RTP TOS bits 184
== Using SIP RTP CoS mark 5
== Using SIP RTP TOS bits 184
== Using SIP RTP CoS mark 5
[2015-12-22 12:30:32] WARNING[5298]: func_cdr.c:352 cdr_write_callback: CDR requires a value (CDR(variable)=value)
== Using SIP RTP TOS bits 184
== Using SIP RTP CoS mark 5
== Spawn extension (macro-dialout-trunk, s, 23) exited non-zero on ‘SIP/anveo_in_4-000000e0’ in macro ‘dialout-trunk’
== Spawn extension (from-internal, 8003356500, 8) exited non-zero on ‘SIP/anveo_in_4-000000e0’
== Spawn extension (from-internal, h, 1) exited non-zero on ‘SIP/anveo_in_4-000000e0’
== Everyone is busy/congested at this time (1:1/0/0)
== Spawn extension (macro-dialout-trunk, s-BUSY, 3) exited non-zero on ‘SIP/201-000000de’ in macro ‘dialout-trunk’
== Spawn extension (from-internal, 9291802, 8) exited non-zero on ‘SIP/201-000000de’
== Spawn extension (from-internal, h, 1) exited non-zero on ‘SIP/201-000000de’

And here is what the sip trace on Anveo says:

/*>>>|199.91.70.167:5060 @ 2015-12-22 20:30:32 */
INVITE sip:[email protected]:5060 SIP/2.0
Via: SIP/2.0/UDP 50.22.101.14:5060;branch=z9hG4bK90da30b6e34a022cf221b32b123f2a4a;rport
Max-Forwards: 70
From: “VERTICAL COMPUT” sip:[email protected];tag=3e8d70ce40fa265005f9d4d1eee0a7b1
To: sip:[email protected]
Call-ID: [email protected]_1
CSeq: 200 INVITE
Contact: Anonymous sip:[email protected]:5060
Expires: 300
User-Agent: Anveo Callcontrol
cisco-GUID: 2922397999-1129949192-1286311730-3412150297
h323-conf-id: 2922397999-1129949192-1286311730-3412150297
P-Asserted-Identity: “VERTICAL COMPUT” sip:[email protected]:5060
X-anveo-e164: 19099291802
Content-Type: application/sdp
Content-Length: 281

v=0
o=Sonus_UAC 130722 196253 IN IP4 67.231.1.112
s=SIP Media Capabilities
c=IN IP4 67.231.1.79
t=0 0
m=audio 17048 RTP/AVP 0 18 101
a=rtpmap:0 PCMU/8000
a=rtpmap:18 G729/8000
a=fmtp:18 annexb=no
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-15
a=sendrecv
a=ptime:20

/*<<<|199.91.70.167:5060 @ 2015-12-22 20:30:32 */
SIP/2.0 100 Trying
Via: SIP/2.0/UDP 50.22.101.14:5060;branch=z9hG4bK90da30b6e34a022cf221b32b123f2a4a;received=50.22.101.14;rport=5060
From: “VERTICAL COMPUT” sip:[email protected];tag=3e8d70ce40fa265005f9d4d1eee0a7b1
To: sip:[email protected]
Call-ID: [email protected]_1
CSeq: 200 INVITE
Server: FPBX-13.0.25(13.6.0)
Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, SUBSCRIBE, NOTIFY, INFO, PUBLISH, MESSAGE
Supported: replaces, timer
Contact: sip:[email protected]:5060
Content-Length: 0

/*>>>|199.91.70.167:5060 @ 2015-12-22 20:30:41 */
CANCEL sip:[email protected]:5060 SIP/2.0
Via: SIP/2.0/UDP 50.22.101.14:5060;rport;branch=z9hG4bK90da30b6e34a022cf221b32b123f2a4a
Max-Forwards: 70
From: “VERTICAL COMPUT” sip:[email protected];tag=3e8d70ce40fa265005f9d4d1eee0a7b1
To: sip:[email protected]
Call-ID: [email protected]_1
CSeq: 200 CANCEL
User-Agent: Anveo Callcontrol
Content-Length: 0

/*<<<|199.91.70.167:5060 @ 2015-12-22 20:30:41 */
SIP/2.0 487 Request Terminated
Via: SIP/2.0/UDP 50.22.101.14:5060;branch=z9hG4bK90da30b6e34a022cf221b32b123f2a4a;received=50.22.101.14;rport=5060
From: “VERTICAL COMPUT” sip:[email protected];tag=3e8d70ce40fa265005f9d4d1eee0a7b1
To: sip:[email protected];tag=as4af7fa4e
Call-ID: [email protected]_1
CSeq: 200 INVITE
Server: FPBX-13.0.25(13.6.0)
Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, SUBSCRIBE, NOTIFY, INFO, PUBLISH, MESSAGE
Supported: replaces, timer
Content-Length: 0

/*>>>|199.91.70.167:5060 @ 2015-12-22 20:30:41 */
ACK sip:[email protected]:5060 SIP/2.0
Via: SIP/2.0/UDP 50.22.101.14:5060;rport;branch=z9hG4bK90da30b6e34a022cf221b32b123f2a4a
Max-Forwards: 70
From: “VERTICAL COMPUT” sip:[email protected];tag=3e8d70ce40fa265005f9d4d1eee0a7b1
To: sip:[email protected];tag=as4af7fa4e
Call-ID: [email protected]_1
CSeq: 200 ACK
User-Agent: Anveo Callcontrol
Content-Length: 0

/*<<<|199.91.70.167:5060 @ 2015-12-22 20:30:41 */
SIP/2.0 200 OK
Via: SIP/2.0/UDP 50.22.101.14:5060;branch=z9hG4bK90da30b6e34a022cf221b32b123f2a4a;received=50.22.101.14;rport=5060
From: “VERTICAL COMPUT” sip:[email protected];tag=3e8d70ce40fa265005f9d4d1eee0a7b1
To: sip:[email protected];tag=as4af7fa4e
Call-ID: [email protected]_1
CSeq: 200 CANCEL
Server: FPBX-13.0.25(13.6.0)
Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, SUBSCRIBE, NOTIFY, INFO, PUBLISH, MESSAGE
Supported: replaces, timer
Content-Length: 0

And because I believe it’s a VSP issue, I submitted a ticket.Here’s the ticket thus far: (Note that they referenced a different CDR, but it is the same issue as above)

12/22/2015 9:50:39 AM
by Steven Sedory Hello, please see our CDR from today, the 22nd, with calls from 19092276828 to 19099291709.

They’re getting the status of “Trying”. Can you look at the sip trace and help us understand why?

Note that we have our firewall set to allow 5060 from your five servers, and 10,000-20,000 udp from all addresses, per your FAQ instructions.

We had this issue before, but after we allowed the RTP ports to all IP’s, it started working again. This new problem has been recently discovered.
12/22/2015 9:59:28 AM
by Steven Sedory Please note two things:

What it is we’re doing is setting one of our DID’s to hit our PBX, and then send the user to a different number by having our PBX forward the call. You will see some completed calls on the CDR too, but those are when we have our PBX answer the call first, and then dial back out. THE PROBLEM SEEMS TO ONLY HAPPEN WITH TOLL FREE NUMBERS. Other numbers will ring through just fine.
12/22/2015 10:07:20 AM
by Steven Sedory Again, I would like to confirm that this seems to only be for toll free numbers, that it is affecting all of our servers, and has recently become noticed as an issue. We have made no changes to our servers, firewall, or network that would have caused this.

Has something changed on the Anveo side recently?
12/22/2015 10:43:05 AM
by MFonk I checked SIP session for the following call

IN DIDIN 22-Dec-2015 10:05:51 22-Dec-2015 10:05:51 19099291800 +15594185003

You can download the trace from http://siptraces.anveo.com/sip_1178872370_22535835672311112b2b_1.log

The call was canceled because your server was not responding to the call (other than 100 Trying code) within 10 seconds. Such cases are considered as destination not reachable and the call would failover to the backup SIP URI. Because you do not have failover SIP URI configured the call was canceled.

To avoid such issues you need to make sure that your server is not overloaded can can respond with call progress codes (either 180 Ringing, 183 Session Progress, 200 OK or can termination code).

Thoughts? I see their advice on at the end of the ticket, but am unsure how to assess it. I know our servers are not overloaded, but I’m unsure as how I’m supposed to check those progress codes. Again, this all work fine and dandy last week with toll free numbers.