Some outgoing calls fail for no reason

Freepbx distro 6.12.65-26, asterisk 13.2.0
we have a PRI trunk, 30 channels.

sometimes certain outgoing calls return “all circuits are busy” even though they are not (there are rarely more that 9 call at once).

asterisk logs look like this:

[2015-03-24 10:29:04] VERBOSE[31190][C-00001d26] pbx.c: Executing [s@macro-dialout-trunk:22] Dial("SIP/311-00002022", "DAHDI/G0/0544972786,300,wW") in new stack
[2015-03-24 10:29:04] VERBOSE[31190][C-00001d26] sig_pri.c: Requested transfer capability: 0x00 - SPEECH
[2015-03-24 10:29:04] VERBOSE[31190][C-00001d26] app_dial.c: Called DAHDI/G0/0544972786
[2015-03-24 10:29:04] VERBOSE[31190][C-00001d26] app_dial.c: DAHDI/i1/0544972786-e32 is proceeding passing it to SIP/311-00002022
[2015-03-24 10:29:04] VERBOSE[2103][C-00001d26] sig_pri.c: Span 1: Channel 0/31 got hangup request, cause 34
[2015-03-24 10:29:04] VERBOSE[31190][C-00001d26] app_dial.c: DAHDI/i1/0544972786-e32 is circuit-busy
[2015-03-24 10:29:04] VERBOSE[31190][C-00001d26] chan_dahdi.c: Hungup 'DAHDI/i1/0544972786-e32'
[2015-03-24 10:29:04] VERBOSE[31190][C-00001d26] app_dial.c: Everyone is busy/congested at this time (1:0/1/0)
[2015-03-24 10:29:04] VERBOSE[31190][C-00001d26] pbx.c: Executing [s@macro-dialout-trunk:23] NoOp("SIP/311-00002022", "Dial failed for some reason with DIALSTATUS = CONGESTION and HANGUPCAUSE = 34") in new stack

any ideas?

The cause 34 is a congestion:

define AST_CAUSE_NORMAL_CIRCUIT_CONGESTION 34

Enable the PRI debug and see the response from your provider.

this is what the log looks like with PRI debug on:

    -- Executing [s@macro-dialout-trunk:22] Dial("SIP/770-000065b6", "DAHDI/g0/0547702683,300,wW") in new stack
PRI Span: 1 -- Making new call for cref 32865
    -- Requested transfer capability: 0x00 - SPEECH
PRI Span: 1
PRI Span: 1 > DL-DATA request
PRI Span: 1 > Protocol Discriminator: Q.931 (8)  len=42
PRI Span: 1 > TEI=0 Call Ref: len= 2 (reference 97/0x61) (Sent from originator)
PRI Span: 1 > Message Type: SETUP (5)
PRI Span: 1 TEI=0 Transmitting N(S)=79, window is open V(A)=79 K=7
PRI Span: 1
PRI Span: 1 > Protocol Discriminator: Q.931 (8)  len=42
PRI Span: 1 > TEI=0 Call Ref: len= 2 (reference 97/0x61) (Sent from originator)
PRI Span: 1 > Message Type: SETUP (5)
PRI Span: 1 > [04 03 80 90 a3]
PRI Span: 1 > Bearer Capability (len= 5) [ Ext: 1  Coding-Std: 0  Info transfer capability: Speech (0)
PRI Span: 1 >                              Ext: 1  Trans mode/rate: 64kbps, circuit-mode (16)
PRI Span: 1 >                                User information layer 1: A-Law (35)
PRI Span: 1 > [18 03 a1 83 81]
PRI Span: 1 > Channel ID (len= 5) [ Ext: 1  IntID: Implicit  Other(PRI)  Spare: 0  Preferred  Dchan: 0
PRI Span: 1 >                       ChanSel: As indicated in following octets
PRI Span: 1 >                       Ext: 1  Coding: 0  Number Specified  Channel Type: 3
PRI Span: 1 >                       Ext: 1  Channel: 1 Type: CPE]
PRI Span: 1 > [6c 0b 21 80 37 33 32 35 35 30 33 39 39]
PRI Span: 1 > Calling Number (len=13) [ Ext: 0  TON: National Number (2)  NPI: ISDN/Telephony Numbering Plan (E.164/E.163) (1)
PRI Span: 1 >                           Presentation: Presentation permitted, user number not screened (0)  '732550399' ]
PRI Span: 1 > [70 0b 80 30 35 34 37 37 30 32 36 38 33]
PRI Span: 1 > Called Number (len=13) [ Ext: 1  TON: Unknown Number Type (0)  NPI: Unknown Number Plan (0)  '0547702683' ]
PRI Span: 1 > [a1]
PRI Span: 1 > Sending Complete (len= 1)
PRI Span: 1 q931.c:6036 q931_setup: Call 32865 enters state 1 (Call Initiated).  Hold state: Idle
    -- Called DAHDI/g0/0547702683
PRI Span: 1
PRI Span: 1 < Protocol Discriminator: Q.931 (8)  len=10
PRI Span: 1 < TEI=0 Call Ref: len= 2 (reference 97/0x61) (Sent to originator)
PRI Span: 1 < Message Type: CALL PROCEEDING (2)
PRI Span: 1 < [18 03 a9 83 81]
PRI Span: 1 < Channel ID (len= 5) [ Ext: 1  IntID: Implicit  Other(PRI)  Spare: 0  Exclusive  Dchan: 0
PRI Span: 1 <                       ChanSel: As indicated in following octets
PRI Span: 1 <                       Ext: 1  Coding: 0  Number Specified  Channel Type: 3
PRI Span: 1 <                       Ext: 1  Channel: 1 Type: CPE]
PRI Span: 1 Received message for call 0x7f56c3455f40 on link 0x7f56707f53c0 TEI/SAPI 0/0
PRI Span: 1 -- Processing IE 24 (cs0, Channel Identification)
PRI Span: 1 q931.c:8454 post_handle_q931_message: Call 32865 enters state 3 (Outgoing Call Proceeding).  Hold state: Idle
Span 1: Processing event PRI_EVENT_PROCEEDING(13)
    -- DAHDI/i1/0547702683-85 is proceeding passing it to SIP/770-000065b6
PRI Span: 1
PRI Span: 1 < Protocol Discriminator: Q.931 (8)  len=9
PRI Span: 1 < TEI=0 Call Ref: len= 2 (reference 97/0x61) (Sent to originator)
PRI Span: 1 < Message Type: DISCONNECT (69)
PRI Span: 1 < [08 02 80 a2]
PRI Span: 1 < Cause (len= 4) [ Ext: 1  Coding: CCITT (ITU) standard (0)  Spare: 0  Location: User (0)
PRI Span: 1 <                  Ext: 1  Cause: Circuit/channel congestion (34), class = Network Congestion (resource unavailable) (2) ]
PRI Span: 1 Received message for call 0x7f56c3455f40 on link 0x7f56707f53c0 TEI/SAPI 0/0
PRI Span: 1 -- Processing IE 8 (cs0, Cause)
PRI Span: 1 -- Found active call: 0x7f56c3455f40 cref:32865
PRI Span: 1 q931.c:8707 post_handle_q931_message: Call 32865 enters state 12 (Disconnect Indication).  Hold state: Idle
Span 1: Processing event PRI_EVENT_HANGUP_REQ(15)
    -- Span 1: Channel 0/1 got hangup request, cause 34
    -- DAHDI/i1/0547702683-85 is circuit-busy
PRI Span: 1 q931.c:6837 q931_hangup: Hangup other cref:32865
PRI Span: 1 q931.c:6594 __q931_hangup: ourstate Disconnect Indication, peerstate Disconnect Request, hold-state Idle
PRI Span: 1 q931.c:5703 q931_release: Call 32865 enters state 19 (Release Request).  Hold state: Idle
PRI Span: 1
PRI Span: 1 > DL-DATA request
PRI Span: 1 > Protocol Discriminator: Q.931 (8)  len=9
PRI Span: 1 > TEI=0 Call Ref: len= 2 (reference 97/0x61) (Sent from originator)
PRI Span: 1 > Message Type: RELEASE (77)
PRI Span: 1 TEI=0 Transmitting N(S)=80, window is open V(A)=80 K=7
PRI Span: 1
PRI Span: 1 > Protocol Discriminator: Q.931 (8)  len=9
PRI Span: 1 > TEI=0 Call Ref: len= 2 (reference 97/0x61) (Sent from originator)
PRI Span: 1 > Message Type: RELEASE (77)
PRI Span: 1 > [08 02 81 a2]
PRI Span: 1 > Cause (len= 4) [ Ext: 1  Coding: CCITT (ITU) standard (0)  Spare: 0  Location: Private network serving the local user (1)
PRI Span: 1 >                  Ext: 1  Cause: Circuit/channel congestion (34), class = Network Congestion (resource unavailable) (2) ]
    -- Hungup 'DAHDI/i1/0547702683-85'
  == Everyone is busy/congested at this time (1:0/1/0)

YUP, your provider has a congestion in the network according both traces. You need to check what happen with them.

Then I guess I’ll have a talk with the provider.
Thank you for the help.