I have done this routinely for years, but my thinking on this has evolved recently. While you are doing setup and testing, by all means while list local subnets in fail2ban, but once you’re in production I would argue that you want to be notified if any local IP address is abusing the PBX services. There was a router/firewall exploit recently that would could have been detected sooner had local subnets not been white listed in fail2ban.