This is one of the most complicated subjects in VoIP and very difficult to troubleshoot. This is a bit of a dissertation however I hope it gives a better understanding of why talk off occurs and that it is not easy to eliminate when we converge the analog and digital worlds.
First to clarify a a point, DTMF would never be transcoded unless the inband method was used.
The symptom the OP is describing is called “talk off”, it happens when the DTMF receiver code (it’s all DSP’s) “falses” and regenerates the tone on the distant end. SNOM’s are horrific for this issue, Aastra’s are next.
In order to understand why the phenomena occurs a review of how VoIP handles DTMF is useful.
DTMF is as pointed out dual frequency. If you visualize the two sine waves of the tones they will cross at a certain point. In the early day this was done with filters. When the tones crossed the filters would conduct and decode the tone. There are 8 tones total arranged in a matrix of 4x4, this gives 16 tones (the ABCD tones are not used anymore). Here is illustrations of both:
If you look at the basis function where the purple and green waveforms cross, that is the point of the tones detection.
As was pointed out this is useless technology in the digital age. It was originally designed to transmit the data (2^4) over an audio link.
However we are stuck with it.
Compressed CODEC’s often used in voice do a terrible job of reproducing sine waves such as DTMF. In order to get around this limitation the DTMF is detected by the endpoint (phone) and converted to out of band data.
Two popular formats exist:
1 - RFC2883
This uses specific data words inserted into the RTP stream. Software at each end recognizes the data pattern and reproduces the DTMF tone.
2 - SIP Info
A SIP info message is utilized to send the DTMF
The data is never transcoded, the key to the two methods is they must be configured the same on both ends of the session.
Asterisk has a method called auto that will try and figure it out. It is less than perfect in practice.
You can use one method on your trunks and one method on your phones. Asterisk has it’s own internal DTMF architecture that is used as the common denominator.
VoIP providers redirect traffic to gateways that are located close to the rate center you are calling to reduce toll charges. Keeping track of thousands of gateways owned by many different providers is a challenge. If these gateways do not use the same DTMF method you will have intermittent DTMF recognition and transmission.
Lastly, if you are using a non-compressed linear CODEC (ulaw or alaw) you can choose to leave the DTMF as audio. This method is called inband and the tones are digitized along with the speech. This is the least reliable DTMF transmission method.
The talk off often occurs when the digital audio level is very high (loud speech patterns).
I have found that using RFC2833 to your endpoints is the most effective way of reducing talk off.