Trouble with Queues and penalties

I have three permanent agents in a queue. They all have penalties as there is a prefered agent for taking tech support calls but if they are busy or out of the office the calls need to go elsewhere.

I had ringall with the penalties but it would still ring the first agent when he is on the phone and never try any others.

I have changed it to roundrobin with behaves better but seems to ignore the penalty values when choosing where to start ringing.

Is there a way to make this work the way I want without having to limit the calls per phone to 1 in the sip.conf?