Connecting to the same DNS/IP through different gateways (maybe some rewrite/iptables stuff is needed?)

Hi there,

my first post, all other things I could solve just searching existing topics but now I have I think a not so common setup here:
I have two routers for to different ISP-connections to the same ISP. This ISP also services VOIP-services which are bound to the connections to the ISP. But the DNS to connect to is the same for both. So let’s describe some networkstuff here:
RouterA: VLAN 102
RouterB VLAN 103
RouterMain: VLAN 1 this router connects to the internet through router A and B (OPNSense running)
As I have things virtualised it is no problem to give FreePBX networkcards which go into the RouterA and B VLANS.
problem is now that the trunks register to and for some numbers it has to be routed through RouterA and for some through RouterB.
The problem here for me is how to distinguish the connection it comes from to route it.
Eventually through the transport?

Would be great if someone has an idea to get this working. Unfortunately seems to resolve to one IP-address, otherwise a workaround could be set one specific route for IP1 and another for IP2.