The issue occurs randomly once a month propably
after applying changes (adding Extension or adding DDI) asterisk stops responding.
(we are applying changes several times per day)
but I can still connect to asterisk CLI asterisk -rx so probably it is not crashing
2- When Asterisk is still accessible but there is no core dump file I assume it did not crash but just hung. In this case I would need to use this command?
Incorrect, Asterisk can be compiled to provide backtraces on any version of Debian (by my experience since Wheezy (7) ). That Sangoma have not done yet is a different story.
I’m thinking that if this issue occurs when clicking “Apply Changes”, the cause might be related to how we use func_odbc with a local MySQL database to determine the correct route path.
For example, with 50 concurrent calls, each call queries MySQL through the ODBC driver, generating 50 simultaneous database queries.
At the same time, when we click “Apply Changes” in FreePBX, it also executes multiple SQL queries to retrieve configurations and regenerate config files.
This makes me wonder if we might be overloading MySQL or ODBC due to the combined load of call-related queries and FreePBX configuration updates.
Although I haven’t looked at the actual transactions, my understanding is that apply changes is just a report generation operation, so should only do read accesses, on the SQL database. I think is only things like CDRs that involve write locks at run time, and they wouldn’t be read, so shouldn’t get read locks.
The actual working configuration, in Asterisk, is held in memory, an constructed from flat files, output by the report generation step, so done after access to the FreePBX database has ended.
Consider that “Apply changes” also involves re-writing sqlite3 tables, I have long suspected that the asterisk ‘database’ code is not as rubust as it could be since moving from Berkeley to sqlite3, Asterisk itself has no awareness of mysql/mariadb outside of it’s add-on cdr odbc driver.
I suspect it has the same locking granularity as the Berkeley one, so difficult to deadlock. I don’t know if FreePBX uses non-trivial transactions, though, when loading it.
It is easy to ‘deadlock’, open a connection to /var/lib/asterisk/astdb.sqlite3 by any method that is not the ongoing asterisk user one and continually even just read data, in short order the database will be locked. BTDT
