I agree with @billsimon that saturated uplink is likely.
To confirm or refute, capture all traffic at the PBX, wait for a report of a bad call, find the captured call and analyze the RTP with Wireshark, looking for lost packets and excessive jitter. See
You mentioned TLS and if all traffic is encrypted, it makes troubleshooting more difficult. In this case, I would recommend also turning pjsip logger on. You can then see unencrypted SIP for the bad call with accurate timestamps in the Asterisk log, so you can find the RTP in Wireshark. Although the RTP is also encrypted (you can’t listen to it), you can still analyze it for loss and jitter.