IAX Truck One way does not work

I have two boxes setup at two locations with a iax trunk between then. today one side just stopped working.

When dialing out i get all circuits are busy

asterisk -rvvvv
Asterisk 13.9.1, Copyright © 1999 - 2014, Digium, Inc. and others.
Created by Mark Spencer [email protected]
Asterisk comes with ABSOLUTELY NO WARRANTY; type ‘core show warranty’ for detail s.
This is free software, with components licensed under the GNU General Public
License version 2 and other licenses; you are welcome to redistribute it under
certain conditions. Type ‘core show license’ for details.

Connected to Asterisk 13.9.1 currently running on apdpbx (pid = 21513)
[2016-09-22 15:34:33] NOTICE[21561]: chan_sip.c:28223 handle_request_subscribe: Received SIP subscribe for peer without mailbox: 689
[2016-09-22 15:34:33] NOTICE[21561]: chan_sip.c:28223 handle_request_subscribe: Received SIP subscribe for peer without mailbox: 649
[2016-09-22 15:34:34] NOTICE[21561]: chan_sip.c:28223 handle_request_subscribe: Received SIP subscribe for peer without mailbox: 689
[2016-09-22 15:34:34] NOTICE[21561]: chan_sip.c:28223 handle_request_subscribe: Received SIP subscribe for peer without mailbox: 649
– Channel SIP/621-0000b68a left ‘simple_bridge’ basic-bridge
– Channel IAX2/freepbx-130 left ‘simple_bridge’ basic-bridge
== Extension Changed 621[ext-local] new state Idle for Notify User 678
== Extension Changed 621[ext-local] new state Idle for Notify User 639
== Extension Changed 621[ext-local] new state Idle for Notify User 668
== Extension Changed 621[ext-local] new state Idle for Notify User 622
== Extension Changed 621[ext-local] new state Idle for Notify User 642
== Spawn extension (macro-dial, s, 17) exited non-zero on ‘IAX2/freepbx-130’ in macro ‘dial’
== Extension Changed 621[ext-local] new state Idle for Notify User 683
== Spawn extension (followme-sub, 621, 38) exited non-zero on ‘IAX2/freepbx-130’
== Extension Changed 621[ext-local] new state Idle for Notify User 660
== Extension Changed 621[ext-local] new state Idle for Notify User 646
– Hungup ‘IAX2/freepbx-130’
== Extension Changed 621[ext-local] new state Idle for Notify User 643
== Extension Changed 621[ext-local] new state Idle for Notify User 664
== MixMonitor close filestream (mixed)
== End MixMonitor Recording IAX2/freepbx-130
== Extension Changed 621[ext-local] new state Idle for Notify User 623
== Extension Changed 621[ext-local] new state Idle for Notify User 624
== Extension Changed 621[ext-local] new state Idle for Notify User 662
== Extension Changed 621[ext-local] new state Idle for Notify User 663
== Extension Changed 621[ext-local] new state Idle for Notify User 621
== Extension Changed 621[ext-local] new state Idle for Notify User 634
[2016-09-22 15:34:40] NOTICE[21561]: chan_sip.c:28223 handle_request_subscribe: Received SIP subscribe for peer without mailbox: 692
[2016-09-22 15:34:41] NOTICE[21561]: chan_sip.c:28223 handle_request_subscribe: Received SIP subscribe for peer without mailbox: 692
== Using SIP RTP TOS bits 184
== Using SIP RTP CoS mark 5
== Extension Changed 621[ext-local] new state InUse for Notify User 678
== Extension Changed 621[ext-local] new state InUse for Notify User 639
== Extension Changed 621[ext-local] new state InUse for Notify User 668
== Extension Changed 621[ext-local] new state InUse for Notify User 622
== Extension Changed 621[ext-local] new state InUse for Notify User 642
– Executing [203@from-internal:1] Macro(“SIP/621-0000b68b”, “user-callerid,LIMIT,EXTERNAL,”) in new stack
== Extension Changed 621[ext-local] new state InUse for Notify User 683
== Extension Changed 621[ext-local] new state InUse for Notify User 660
== Extension Changed 621[ext-local] new state InUse for Notify User 646
== Extension Changed 621[ext-local] new state InUse for Notify User 643
– Executing [s@macro-user-callerid:1] Set(“SIP/621-0000b68b”, “TOUCH_MONITOR=1474576481.543350”) in new stack
== Extension Changed 621[ext-local] new state InUse for Notify User 664
== Extension Changed 621[ext-local] new state InUse for Notify User 623
– Executing [s@macro-user-callerid:2] Set(“SIP/621-0000b68b”, “AMPUSER=621”) in new stack
== Extension Changed 621[ext-local] new state InUse for Notify User 624
– Executing [s@macro-user-callerid:3] GotoIf(“SIP/621-0000b68b”, “0?report”) in new stack
== Extension Changed 621[ext-local] new state InUse for Notify User 662
– Executing [s@macro-user-callerid:4] ExecIf(“SIP/621-0000b68b”, “1?Set(REALCALLERIDNUM=621)”) in new stack
== Extension Changed 621[ext-local] new state InUse for Notify User 663
== Extension Changed 621[ext-local] new state InUse for Notify User 621
– Executing [s@macro-user-callerid:5] Set(“SIP/621-0000b68b”, “AMPUSER=621”) in new stack
== Extension Changed 621[ext-local] new state InUse for Notify User 634
– Executing [s@macro-user-callerid:6] GotoIf(“SIP/621-0000b68b”, “0?limit”) in new stack
– Executing [s@macro-user-callerid:7] Set(“SIP/621-0000b68b”, “AMPUSERCIDNAME=J Lando”) in new stack
– Executing [s@macro-user-callerid:8] GotoIf(“SIP/621-0000b68b”, “0?report”) in new stack
– Executing [s@macro-user-callerid:9] Set(“SIP/621-0000b68b”, “AMPUSERCID=621”) in new stack
– Executing [s@macro-user-callerid:10] Set(“SIP/621-0000b68b”, “__DIAL_OPTIONS=Ttr”) in new stack
– Executing [s@macro-user-callerid:11] Set(“SIP/621-0000b68b”, “CALLERID(all)=“J Lando” <621>”) in new stack
– Executing [s@macro-user-callerid:12] GotoIf(“SIP/621-0000b68b”, “0?limit”) in new stack
– Executing [s@macro-user-callerid:13] ExecIf(“SIP/621-0000b68b”, “1?Set(GROUP(concurrency_limit)=621)”) in new stack
– Executing [s@macro-user-callerid:14] ExecIf(“SIP/621-0000b68b”, “0?Set(CHANNEL(language)=)”) in new stack
– Executing [s@macro-user-callerid:15] GotoIf(“SIP/621-0000b68b”, “1?continue”) in new stack
– Goto (macro-user-callerid,s,29)
– Executing [s@macro-user-callerid:29] Set(“SIP/621-0000b68b”, “CALLERID(number)=621”) in new stack
– Executing [s@macro-user-callerid:30] Set(“SIP/621-0000b68b”, “CALLERID(name)=J Lando”) in new stack
– Executing [s@macro-user-callerid:31] GotoIf(“SIP/621-0000b68b”, “0?cnum”) in new stack
– Executing [s@macro-user-callerid:32] Set(“SIP/621-0000b68b”, “CDR(cnam)=J Lando”) in new stack
– Executing [s@macro-user-callerid:33] Set(“SIP/621-0000b68b”, “CDR(cnum)=621”) in new stack
– Executing [s@macro-user-callerid:34] Set(“SIP/621-0000b68b”, “CHANNEL(language)=en”) in new stack
– Executing [203@from-internal:2] Gosub(“SIP/621-0000b68b”, “sub-record-check,s,1(out,203,force)”) in new stack
– Executing [s@sub-record-check:1] GotoIf(“SIP/621-0000b68b”, “0?initialized”) in new stack
– Executing [s@sub-record-check:2] Set(“SIP/621-0000b68b”, “__REC_STATUS=INITIALIZED”) in new stack
– Executing [s@sub-record-check:3] Set(“SIP/621-0000b68b”, “NOW=1474576481”) in new stack
– Executing [s@sub-record-check:4] Set(“SIP/621-0000b68b”, “__DAY=22”) in new stack
– Executing [s@sub-record-check:5] Set(“SIP/621-0000b68b”, “__MONTH=09”) in new stack
– Executing [s@sub-record-check:6] Set(“SIP/621-0000b68b”, “__YEAR=2016”) in new stack
– Executing [s@sub-record-check:7] Set(“SIP/621-0000b68b”, “__TIMESTR=20160922-153441”) in new stack
– Executing [s@sub-record-check:8] Set(“SIP/621-0000b68b”, “__FROMEXTEN=621”) in new stack
– Executing [s@sub-record-check:9] Set(“SIP/621-0000b68b”, “__MON_FMT=wav”) in new stack
– Executing [s@sub-record-check:10] NoOp(“SIP/621-0000b68b”, “Recordings initialized”) in new stack
– Executing [s@sub-record-check:11] ExecIf(“SIP/621-0000b68b”, “0?Set(ARG3=dontcare)”) in new stack
– Executing [s@sub-record-check:12] Set(“SIP/621-0000b68b”, “REC_POLICY_MODE_SAVE=”) in new stack
– Executing [s@sub-record-check:13] ExecIf(“SIP/621-0000b68b”, “0?Set(REC_STATUS=NO)”) in new stack
– Executing [s@sub-record-check:14] GotoIf(“SIP/621-0000b68b”, “3?checkaction”) in new stack
– Goto (sub-record-check,s,17)
– Executing [s@sub-record-check:17] GotoIf(“SIP/621-0000b68b”, “1?sub-record-check,out,1”) in new stack
– Goto (sub-record-check,out,1)
– Executing [out@sub-record-check:1] NoOp(“SIP/621-0000b68b”, “Outbound Recording Check from 621 to 203”) in new stack
– Executing [out@sub-record-check:2] Set(“SIP/621-0000b68b”, “RECMODE=force”) in new stack
– Executing [out@sub-record-check:3] ExecIf(“SIP/621-0000b68b”, “0?Goto(routewins)”) in new stack
– Executing [out@sub-record-check:4] ExecIf(“SIP/621-0000b68b”, “1?Goto(routewins)”) in new stack
– Goto (sub-record-check,out,7)
– Executing [out@sub-record-check:7] Gosub(“SIP/621-0000b68b”, “recordcheck,1(force,out,203)”) in new stack
– Executing [recordcheck@sub-record-check:1] NoOp(“SIP/621-0000b68b”, “Starting recording check against force”) in new stack
– Executing [recordcheck@sub-record-check:2] Goto(“SIP/621-0000b68b”, “force”) in new stack
– Goto (sub-record-check,recordcheck,5)
– Executing [recordcheck@sub-record-check:5] Set(“SIP/621-0000b68b”, “__REC_POLICY_MODE=FORCE”) in new stack
– Executing [recordcheck@sub-record-check:6] GotoIf(“SIP/621-0000b68b”, “1?startrec”) in new stack
– Goto (sub-record-check,recordcheck,16)
– Executing [recordcheck@sub-record-check:16] NoOp(“SIP/621-0000b68b”, “Starting recording: out, 203”) in new stack
– Executing [recordcheck@sub-record-check:17] Set(“SIP/621-0000b68b”, “AUDIOHOOK_INHERIT(MixMonitor)=yes”) in new stack
– Executing [recordcheck@sub-record-check:18] Set(“SIP/621-0000b68b”, “__CALLFILENAME=out-203-621-20160922-153441-1474576481.543350”) in new stack
– Executing [recordcheck@sub-record-check:19] MixMonitor(“SIP/621-0000b68b”, “2016/09/22/out-203-621-20160922-153441-1474576481.543350.wav,ai(LOCAL_MIXMON_ID),”) in new stack
== Begin MixMonitor Recording SIP/621-0000b68b
– Executing [recordcheck@sub-record-check:20] Set(“SIP/621-0000b68b”, “__MIXMON_ID=0x7ff474aeb380”) in new stack
– Executing [recordcheck@sub-record-check:21] Set(“SIP/621-0000b68b”, “__RECORD_ID=SIP/621-0000b68b”) in new stack
– Executing [recordcheck@sub-record-check:22] Set(“SIP/621-0000b68b”, “__REC_STATUS=RECORDING”) in new stack
– Executing [recordcheck@sub-record-check:23] Set(“SIP/621-0000b68b”, “CDR(recordingfile)=out-203-621-20160922-153441-1474576481.543350.wav”) in new stack
– Executing [recordcheck@sub-record-check:24] Return(“SIP/621-0000b68b”, “”) in new stack
– Executing [out@sub-record-check:8] Return(“SIP/621-0000b68b”, “”) in new stack
– Executing [203@from-internal:3] ExecIf(“SIP/621-0000b68b”, “0 ?Set(CDR(accountcode)=)”) in new stack
– Executing [203@from-internal:4] Set(“SIP/621-0000b68b”, “INTRACOMPANYROUTE=YES”) in new stack
– Executing [203@from-internal:5] Set(“SIP/621-0000b68b”, “MOHCLASS=default”) in new stack
– Executing [203@from-internal:6] Set(“SIP/621-0000b68b”, “_NODEST=”) in new stack
– Executing [203@from-internal:7] Macro(“SIP/621-0000b68b”, “dialout-trunk,2,203,off”) in new stack
– Executing [s@macro-dialout-trunk:1] Set(“SIP/621-0000b68b”, “DIAL_TRUNK=2”) in new stack
– Executing [s@macro-dialout-trunk:2] GosubIf(“SIP/621-0000b68b”, “0?sub-pincheck,s,1()”) in new stack
– Executing [s@macro-dialout-trunk:3] GotoIf(“SIP/621-0000b68b”, “0?disabletrunk,1”) in new stack
– Executing [s@macro-dialout-trunk:4] Set(“SIP/621-0000b68b”, “DIAL_NUMBER=203”) in new stack
– Executing [s@macro-dialout-trunk:5] Set(“SIP/621-0000b68b”, “DIAL_TRUNK_OPTIONS=Ttr”) in new stack
– Executing [s@macro-dialout-trunk:6] Set(“SIP/621-0000b68b”, “OUTBOUND_GROUP=OUT_2”) in new stack
– Executing [s@macro-dialout-trunk:7] GotoIf(“SIP/621-0000b68b”, “1?nomax”) in new stack
– Goto (macro-dialout-trunk,s,9)
– Executing [s@macro-dialout-trunk:9] GotoIf(“SIP/621-0000b68b”, “1?skipoutcid”) in new stack
– Goto (macro-dialout-trunk,s,12)
– Executing [s@macro-dialout-trunk:12] GosubIf(“SIP/621-0000b68b”, “1?sub-flp-2,s,1()”) in new stack
– Executing [s@sub-flp-2:1] ExecIf(“SIP/621-0000b68b”, “1?Return()”) in new stack
– Executing [s@macro-dialout-trunk:13] Set(“SIP/621-0000b68b”, “OUTNUM=203”) in new stack
– Executing [s@macro-dialout-trunk:14] Set(“SIP/621-0000b68b”, “custom=IAX2/ToCityHall”) in new stack
– Executing [s@macro-dialout-trunk:15] ExecIf(“SIP/621-0000b68b”, “0?Set(DIAL_TRUNK_OPTIONS=M(setmusic^default)Ttr)”) in new stack
– Executing [s@macro-dialout-trunk:16] ExecIf(“SIP/621-0000b68b”, “0?Set(DIAL_TRUNK_OPTIONS=TtrM(confirm))”) in new stack
– Executing [s@macro-dialout-trunk:17] Macro(“SIP/621-0000b68b”, “dialout-trunk-predial-hook,”) in new stack
– Executing [s@macro-dialout-trunk-predial-hook:1] MacroExit(“SIP/621-0000b68b”, “”) in new stack
– Executing [s@macro-dialout-trunk:18] GotoIf(“SIP/621-0000b68b”, “0?bypass,1”) in new stack
– Executing [s@macro-dialout-trunk:19] ExecIf(“SIP/621-0000b68b”, “1?Set(CONNECTEDLINE(num,i)=203)”) in new stack
– Executing [s@macro-dialout-trunk:20] ExecIf(“SIP/621-0000b68b”, “1?Set(CONNECTEDLINE(name,i)=CID:621)”) in new stack
– Executing [s@macro-dialout-trunk:21] ExecIf(“SIP/621-0000b68b”, “0?Set(CONNECTEDLINE(name,i)=CID:(Hidden)621)”) in new stack
– Executing [s@macro-dialout-trunk:22] GotoIf(“SIP/621-0000b68b”, “0?customtrunk”) in new stack
– Executing [s@macro-dialout-trunk:23] Dial(“SIP/621-0000b68b”, “IAX2/ToCityHall/203,300,Ttr”) in new stack
– Called IAX2/ToCityHall/203
– Hungup ‘IAX2/ToCityHall-16994’
== Everyone is busy/congested at this time (1:0/0/1)
– Executing [s@macro-dialout-trunk:24] NoOp(“SIP/621-0000b68b”, “Dial failed for some reason with DIALSTATUS = CHANUNAVAIL and HANGUPCAUSE = 50”) in new stack
– Executing [s@macro-dialout-trunk:25] GotoIf(“SIP/621-0000b68b”, “0?continue,1:s-CHANUNAVAIL,1”) in new stack
– Goto (macro-dialout-trunk,s-CHANUNAVAIL,1)
– Executing [s-CHANUNAVAIL@macro-dialout-trunk:1] Set(“SIP/621-0000b68b”, “RC=50”) in new stack
– Executing [s-CHANUNAVAIL@macro-dialout-trunk:2] Goto(“SIP/621-0000b68b”, “50,1”) in new stack
– Goto (macro-dialout-trunk,50,1)
– Executing [50@macro-dialout-trunk:1] Goto(“SIP/621-0000b68b”, “continue,1”) in new stack
– Goto (macro-dialout-trunk,continue,1)
– Executing [continue@macro-dialout-trunk:1] NoOp(“SIP/621-0000b68b”, “TRUNK Dial failed due to CHANUNAVAIL HANGUPCAUSE: 50 - failing through to other trunks”) in new stack
– Executing [continue@macro-dialout-trunk:2] ExecIf(“SIP/621-0000b68b”, “1?Set(CALLERID(number)=621)”) in new stack
– Executing [203@from-internal:8] Macro(“SIP/621-0000b68b”, “outisbusy,”) in new stack
– Executing [s@macro-outisbusy:1] Progress(“SIP/621-0000b68b”, “”) in new stack
– Executing [s@macro-outisbusy:2] GotoIf(“SIP/621-0000b68b”, “0?emergency,1”) in new stack
– Executing [s@macro-outisbusy:3] GotoIf(“SIP/621-0000b68b”, “1?intracompany,1”) in new stack
– Goto (macro-outisbusy,intracompany,1)
– Executing [intracompany@macro-outisbusy:1] Playback(“SIP/621-0000b68b”, “all-circuits-busy-now&pls-try-call-later, noanswer”) in new stack
– <SIP/621-0000b68b> Playing ‘all-circuits-busy-now.ulaw’ (language ‘en’)
> 0x7ff48efdc5d0 – Probation passed - setting RTP source address to 10.50.20.33:38080
[2016-09-22 15:34:43] NOTICE[21561]: chan_sip.c:28223 handle_request_subscribe: Received SIP subscribe for peer without mailbox: 683
– <SIP/621-0000b68b> Playing ‘pls-try-call-later.ulaw’ (language ‘en’)
[2016-09-22 15:34:44] NOTICE[21561]: chan_sip.c:28223 handle_request_subscribe: Received SIP subscribe for peer without mailbox: 683
– Executing [intracompany@macro-outisbusy:2] Congestion(“SIP/621-0000b68b”, “20”) in new stack
[2016-09-22 15:34:46] WARNING[2063][C-0000561f]: channel.c:4883 ast_prod: Prodding channel ‘SIP/621-0000b68b’ failed
== Spawn extension (macro-outisbusy, intracompany, 2) exited non-zero on ‘SIP/621-0000b68b’ in macro ‘outisbusy’
== Spawn extension (from-internal, 203, 8) exited non-zero on ‘SIP/621-0000b68b’
– Executing [h@from-internal:1] Macro(“SIP/621-0000b68b”, “hangupcall”) in new stack
– Executing [s@macro-hangupcall:1] GotoIf(“SIP/621-0000b68b”, “1?theend”) in new stack
– Goto (macro-hangupcall,s,3)
– Executing [s@macro-hangupcall:3] ExecIf(“SIP/621-0000b68b”, “0?Set(CDR(recordingfile)=)”) in new stack
– Executing [s@macro-hangupcall:4] Hangup(“SIP/621-0000b68b”, “”) in new stack
== Spawn extension (macro-hangupcall, s, 4) exited non-zero on ‘SIP/621-0000b68b’ in macro ‘hangupcall’
== Spawn extension (from-internal, h, 1) exited non-zero on ‘SIP/621-0000b68b’
== Extension Changed 621[ext-local] new state Idle for Notify User 678
== Extension Changed 621[ext-local] new state Idle for Notify User 639
== MixMonitor close filestream (mixed)
== End MixMonitor Recording SIP/621-0000b68b
== Extension Changed 621[ext-local] new state Idle for Notify User 668
== Extension Changed 621[ext-local] new state Idle for Notify User 622
== Extension Changed 621[ext-local] new state Idle for Notify User 642
== Extension Changed 621[ext-local] new state Idle for Notify User 683
== Extension Changed 621[ext-local] new state Idle for Notify User 660
== Extension Changed 621[ext-local] new state Idle for Notify User 646
== Extension Changed 621[ext-local] new state Idle for Notify User 643
== Extension Changed 621[ext-local] new state Idle for Notify User 664
== Extension Changed 621[ext-local] new state Idle for Notify User 623
== Extension Changed 621[ext-local] new state Idle for Notify User 624
== Extension Changed 621[ext-local] new state Idle for Notify User 662
== Extension Changed 621[ext-local] new state Idle for Notify User 663
== Extension Changed 621[ext-local] new state Idle for Notify User 621
== Extension Changed 621[ext-local] new state Idle for Notify User 634

all passwords on trunks are correct all settings are the same.

I have check outbound routes and trunk routes with no luck

I figured it out it was the incoming user settings was not matching the remote end