Very Bad Echo

We are experiencing a significant echo issue…

here’s the setup.

IBM x3650 Quad-Core Xeon 2.5GHz with 4GB RAM
Digium TE121 w/ hardware echo cancel
SNOM 370 and Polycom SoundStation IP 7000

it doesn’t happen all the time, however, we can hear our own voice with about a half to full second delay

The most noticeable echo happens on the Polycom when dialed into a conference bridge.
And, it doesn’t start right away… the users have said the call seems very good for about ten minutes,
and then all of a sudden they hear their own voices echo back.

I’ve also noticed it on the SNOM devices, and most of the time on these the echo is always there.

Any ideas would be appreciated.