FreePBX AllowList Caller Module -- to block all but a list of allowed callers

freepbx
Tags: #<Tag:0x00007f70289ad6f8>

(Mitch Mitchell) #41

I have the route by route allow list checking feature in place now – not a whole lot of testing so far but its working here ok. Let me know if you give it a try.

Since turning on for routes is now the switch that enables the module, I have changed the behavior when no destination specified to send the caller to a black hole again just like the blacklist module. I really don’t want to have two switches that need to be thrown to make the module active.

I’ve updated the github repo with the latest code.


(Lorne Gaetz) #42

Really good stuff here. I only found one cosmetic issue on the settings tab when you select a destination with 2 drop down menus (like voicemail), they are displayed side by side instead of over/under.

When you auto whitelist an outbound dialed number, it looks like you’re acting on the initial dialed digits, but I wonder if it wouldn’t be better to use the resulting dial string after it’s gone thru the outbound routes?

Example, I dial 555-1212, 7 digits only
The outbound route has a rule that prepends 1444 with country and area code to allow the call to terminate, but the string whitelisted is 5551212

When that user calls me back, the Caller ID is 1-444-555-1212, so it won’t match the value that’s currently whitelisted.

I think the best value for whitelisting is the number after outbound routes rules apply but before trunk dial rules apply, but I don’t see a channel variable we can reliably use for this purpose. The channel var DIAL_NUMBER is a string after both route and trunk rules have been applied, which is prob best.

Have you tested what happens when an inbound PSTN call is forwarded off the system to another PSTN number with the “Automatically Allow Called Numbers” option enabled?


#43

You need to .gitignore your module.sig file so the module can be installed directly from github with the GUI Module Admin’s “Download (from Web)” or via cli:

fwconsole ma downloadinstall https://github.com/mitchmitchell/allowlist/archive/master.zip

#44

Looks like that was inherited from the blacklist module.


(Lorne Gaetz) #45

Indeed, which makes me wonder if this might be a framework issue. I have a ticket for it.


(Mitch Mitchell) #46

Can you check and see if you get the same horizontal layout on the blacklist module destination selection? I don’t see that in Chrome and on my phone.
Nevermind, I saw your posts above.

What browser are you using?

I’ll dig into the other things this weekend


(Mitch Mitchell) #47

OK I changed the variable to DIAL_NUMBER for the outbound checks – I have that inserted right above “gocall” in macro-dialout-trunk – does that look ok to you guys?

I removed module.sig and added it to .gitignore – I hope that works. I’ll try installing the module from the web tonight.

I checked when forwarding out:

  1. custom destination – the outbound allowlist check is done
  2. trunk destination – the outbound allowlist check is not done.
  3. DISA destination – the outbound the allowlist check is done.

Can you think of some other outbound configurations worth checking?


(Mitch Mitchell) #48

I’m working to add the commands to fwconsole – an interesting exercise, I’m learning about about Symfony now too.


(Mitch Mitchell) #49

I submitted a pull request into the unsupported srteam. Hopefully the module will be available to folks soon!


(Mitch Mitchell) #50

Some more anti-spam thoughts here:

I realized tonight that my allowlist is not taking advantage of CID look ups at all since it is acting right after the blacklist routines in the dialplan (label ‘did’). I’m seriously rethinking that approach.

I’ve moved the allowlist checking down below the call to CID Superfecta (right after label ‘did-cid-hook’). This will allow taking advantage of some of the anti-spam capabilities built into CID Superfecta.

I’ve got my IVR for non-allowlisted calls set up as some folks described earlier in this thread to prompt the caller to select ‘1’ or ‘2’ depending upon who they want to talk to. So far the only spam to get past it has been a live caller (a fundraiser).


#51

Unless you marked a number ‘allowed’ by mistake, I don’t see how that could help. But it could certainly hurt if a legitimate contact ended up on a spam list (as a result of malice or someone else’s mistake).

Unfortunately, I don’t have good stats on that. In ~3 years, I have had two spam calls ring through. In one case, the spammer spoofed a legitimate contact. In the other, the number was allowlisted because I was curious about a previous blocked call from the same number and I had called the spammer back. Your reordering would likely have blocked that call. However, I don’t track whether any wanted allowlisted calls would have been blocked by the reordering, so can’t do a useful analysis.


(Mitch Mitchell) #52

The way the dial plan flows, if a number is allowed it is processed normally so it would still have to pass the superfecta spam check even if I leave the allowlist checking where it was. The non-allowed call get kicked out to a destination specified in the settings and skip superfecta processing. Moving it down means superfecta gets a shot at the non-allowed calls before allowlist does its check. So in both cases, the allowlist is not a guarantee that the call gets through. I’m not sure if there is a channel variable that can be set to tell superfecta to allow the call through regardless of spam checking. I need to look into that. Probably it should be set up such that being on the allowlist guarantees the call will get past spam checking.


(Jared Busch) #53

I think you don’t need to bother. CID Superfecta does not just have a “block option” that you can always check against.

For example, what would you be flagging in this scenario?

image


(Mitch Mitchell) #54

Yep, I’ve considered and reconsidered a couple of times now. I’m going to run some more tests over the next few days to see what makes more sense. I’m definitely leaning toward putting it back earlier in the dial plan.


(Mitch Mitchell) #55

I moved the check back to where it was – early – I also noticed I was incorrect before when I stated that the blacklist will win, actually the allowlist will win if a number is on both lists.

I’m wondering now if I should tie the auto add feature to outbound routes the way the checking is controlled by inbound routes – any thoughts/preferences?


#56

I’m glad to see that people are finally thinking about ways to improve the blacklist and call screening capabilities. I have mixed feelings about the whitelist approach, though. Yes, it guarantees that those who call from recognized numbers will get through. But what it doesn’t do is give us a good way to solve the real root problem, which is how to we reject calls from telemarketers and spam callers while allowing calls from people we may wish to talk to even if they are calling from an unknown number.

Here is the nightmare scenario - your teenage son or daughter is in a car wreck and when they realize what has happened, they also realize you can’t find their phone - maybe it was ejected from the car, or hidden under a mangled seat or whatever. A passerby or first responder offers the use of their phone and they try to call home but the call isn’t from a recognized number and so is rejected. They can barely speak coherently so any kind of voice recognition fails. Eventually they are taken to a hospital but if the hospital tries to call the same thing happens because their number isn’t recognized either. You get the point. This is why I think a whitelist is a great idea as a preliminary check, but failing the whitelist should probably not mean that the call gets summarily dropped. And you really have to give some thought as to how hard you want to make it to put a call through, for example relying on a code that has to be memorized could cause a problem if a person can barely remember their name. But if you make it too easy then any telemarketer, survey taker, political caller, donation solicitor, or outright fraudster can get through.

Of all the things I have seen suggested to stop annoyance calls, there are two that have given me the biggest bang for the buck, so to speak. One is blocklisting certain repeat callers by Caller ID name. There are certain companies that will call from different numbers but they always use the same Caller ID name, so that traps them. But that only works with annoying repeat callers.

The other thing is something I first read about on another forum a few years ago, and the idea is to block ANY call coming from an exchange or “thousands block” owned by one of two specific telephone companies. These two companies together own millions of phone numbers in the USA, and I don’t recall ever getting a wanted call from one of their numbers, but most of the unwanted calls I get come from one of their numbers. I’m not going to say the name of those two companies (they might be litigious types) but if you get a lot of “bad” calls, take a look at the numbers they come from and go to a site such as https://www.telcodata.us/search-area-code-exchange-detail and put in the area code and first three digits of the number, then scroll down if necessary to find the correct “thousands block” (4th digit of the 7 digit number). After you do that for several different unwanted calls you should begin to see a pattern develop, and if you are like me you will find the vast majority come from one company in particular with a smaller but significant minority coming from another.

What I would wish is that we had a quick and easy way to do a real time lookup of the first seven digits of a ten digit number to determine which company it belongs to, and then reject the call if the number is owned by a company known to host pretty much nothing but annoyance callers. I have tried building lists of area codes and exchanges and “thousands blocks” owned by those companies but they keep acquiring more “thousands blocks” all the time, and tend to give annoyance callers numbers from their newest blocks, so it is very much a moving target. I don’t know if telcodata.us is the only site where you can get this information but in any case I don’t know of any way to do a lookup in real time on a given number, even though I know the Superfecta module queries there to lookup a city and state associated with the exchange.

The only other thing I have wished forever is that the blocklist would recognize patterns and not just numbers. So if you wanted to reject calls from area codes outside the USA and Canada, you could do that. Or if you wanted to reject calls from invalid area codes (that start with 0 or 1, for example), by simply including a pattern such as [01]XXXXXXXXX.

Just some things to consider down the road. I do think the whitelist module is a great first step but hope that more tools for call screening will be developed in time. It has gotten to the point where many of us get as many or more annoyance calls in a day as wanted calls!


(Lorne Gaetz) #57

Have you tried the module yet? There is a user config option to choose a destination for unrecognized CID numbers, you don’t have to drop them. There’s a corresponding option for the blacklist module.

I implemented a deny-by-default inbound call strategy on New Year’s day, 2019. I estimate I’ve blocked about 1800-2000 calls over the last two years, and something like 4-5 unsolicited calls have managed to get thru. You might think I have a difficult challenge, but all I have is a recording telling the caller who they’re calling and to press a number to continue. ANY DTMF tone will get thru my challenge and it happens so rarely I’ve not investigated improving it. The scammers will evolve, but the state of the art right now (in my experience) is blocked by minimal effort. The recording also weeds out the wrong numbers, I literally haven’t had one in 2 years.

The real problem in my experience, is accepting unexpected but desirable robocalls such as utility outage announcements, suspicious bank activity notifications and appointment reminders. I’ve basically had to go without.


#58

I find that recording all incoming calls that fail the immediate testing and STT’ing them gives you the opportunity to post-process them by any means you want to, SMS messages of any plausible ones that fail immediate testing stops the phone ringing yet stills give you an immediacy of response of only a few seconds if necessary.

How exactly to post-process is my ongoing challenge, by number of words, regexing ‘hot’ words time of day etc. If the STT is inconclusive then you always have the raw recording . . .


(Mitch Mitchell) #59

The way I look at it, the blocklist/blacklist eliminates the numbers you know are bad, and the allowlist gets the numbers you know are good through immediately. In between are the numbers that need to pass through some sort of spam detection algorithm.

In my case, all I have is an IVR that asks you to press 1 for Tresa and 2 for Mitch. If you don’t respond it drops the call (I may change that later). That has stopped all but one paid fund raiser who was a human from getting through. If you press 1 or 2, you are taken back to a dedicated inbound route that has superfecta spam checking turned on then you are sent to either a set of extensions for me or my wife. If those go unanswered or we are not at home then you are sent to our mobile phone depending upon who you chose.

My home automation system updates three global variables MITCH_PRESENCE, TRESA_PRESENCE, and MAGNOLIAMANOR_UNOCCUPIED to allow the system to know if there is anyone home to answer a call or should the call go straight to our mobile phones. I use the dynamic routes module at the end of the two dedicated inbound routes to make that decision.

I do have google speech to text and text to speech working but haven’t taken the time to integrate it into the spam checker yet.

Is anyone maintaining the blacklist module at the moment? If not. I’d be willing to take a crack at pattern recognition implementation for it.

Mitch


#60

Asterisk’s blacklist hasn’t changed in donkey’s years, it is what it is but you could, with effort, embellish it. Currently the mere existence of a number will be operative, supporting regexes and processing supplemental ‘values’ would be a good addition. As would expanding it to ‘whitelists’ or any other ident.

When it was written it used the Berkeley database schema, now it uses sqlite3 by inheritance which is much more efficient and expandable