FreePBX AllowList Caller Module -- to block all but a list of allowed callers

FreePBX Applications / Modules
1

[This module is now installed from the FreePBX mirror repo with FreePBX 15 and up - mod]

We’ve been getting lots of spam calls lately so I decided to try and see if I could create a module that would let me block all callers not in a allowed list.

I started with the Blacklist module and essentially reversed the logic so that callers on the list are allowed through and all others blocked or sent to the destination selected. I did change one aspect of the logic so that if a non-allowlisted destination is not defined, the call is allowed to proceed. This prevents shutting down all incoming calls when you first install the module.

I also took care to make sure that the Allowlist module would play well with the Blacklist module. The Blacklist rules come first in the dial plan so if a number is blacklisted it will be blocked regardless of its presence in the whitelist.

The one thing I really wasn’t sure of was how to locate unused feature codes to be sure it wasn’t stomping on some other modules codes – I used one unused on my system but that’s no guarantee for anyone else that tries it.

I put it out on github if anyone is interested. Its self signed so you’ll have to resign the module locally if you decide to try it.

https://github.com/mitchmitchell/allowlist

I don’t have the language skills to update the message translations so they will all be incorrect, and may not even work if the system language is something other than English.

1 Like
2

As you develop your code, be aware that in blacklist , the mere existence of the /family/key entity triggers the blacklist* stuff unconditional of any value .

Being conditional on the value you ‘put’ to your /family/key/value will hopefully expand your horizon.

Just my suggested ‘food for thought’.

3

Yes, and mine follows the same logic right now. I’d have to think about what would be reasonable ‘values’ to store and act on. I suppose that one could be a destination but that smacks of the dynamic routes package (which could easily implement a whitelist function of course).

It could also store priorities, actions, and perhaps no-answer destinations on a per caller basis. I’ve not thought much further beyond the sound of silence the absence of telemarketing calls has brought to the house for now.

4

Another thing I want to do is figure out how to change the ‘whitelist last caller’ function into a ‘whitelist voicemail caller’ function. I’ve not had a chance to did into that yet.

5

The database grew from the flatform berkleyDb to a full rdbms sqlite3, you will find that the ‘family’ can grow from family /a key b value c to family /a/b key c value d and still work in legacy asterisk talk

1 Like
6

Thanks for the post. I hope that you, with community help, can grow this into a first class spam filter.

If the rest of the system is working properly, manual additions or deletions should rarely be needed. I’ve not made a manual edit in more than a year.

My (somewhat simplified) flow: If whitelisted, ring phone. Else give IVR challenge and if passed, add to whitelist and ring phone. Else, check number at TrueCNAM and if passed, send to voicemail. Else hang up call. Also, outbound calls add the called number to the whitelist.

See

Of course, like any filter, it’s less than perfect. It’s nearly 100% effective at keeping unwanted callers from ringing the phone (only 2 in 3 years, caused by spoofs of legitimately whitelisted numbers). And AFAIK, it has never hung up on a legitimate caller. However, a few wanted calls are sent to voicemail (false positives), and some spam ends up in voicemail (false negatives).

I don’t see a good way of greatly reducing the number of false negatives, but the time they waste could be nearly eliminated. Presently, I spend ~20 seconds each, listening to and deleting them. What would help a lot is a combination of voicemail transcription and a visual voicemail system that groups together all messages resulting from the spam filter. A few seconds looking at the screen, you click one button “Filtered messages are all spam” and they disappear.

Most false positives are not a big deal. The caller leaves a message and you call back. Even if you don’t get through, they’re now whitelisted and their next attempt will ring your phone. However, some wanted robocalls are a serious problem, for example, a notification of a possibly fraudulent charge on a credit card. This can be mitigated by requesting that such notifications come by SMS. But if you call a big company, are on hold for a long time and are offered a callback, it often comes from a different number, unknown to you. If that goes to voicemail, you’re screwed and have to start over. My present crude workaround is a BLF key on the phone that temporarily disables spam filtering. When active, it lights up red, so I’m reminded to turn it off after receiving the callback. I hope that someone has a database that could be used to dynamically auto-whitelist these numbers.

1 Like
7

Good stuff @mitchmitchell. I haven’t tried it yet, but some stray thoughts:

  • Consider auto whitelisting all outbound numbers or adding an option to do so. This should largely eliminate the need for a “whitelist last call” feature
  • Consider that different providers can provide CID in different formats, and even the same provider may not be consistent. I’m not sure there is a way to deal with this in astdb without separate db entries.
  • My own whitelist effort has the ability to whitelist callers that successfully navigate an IVR, which eliminates maintenance. I would consider this feature important.
  • Something to keep in mind, but would prob require major restructuring. There is a recurring request from the community to provide “per extension” blacklists, so each user maintains their own list independently, and can’t affect call flow to other extensions.
1 Like
8

I prefer to ask an unknown caller who they want to speak to, then send the audio to STT and act on the answer, this prevents the more sophisticated systems that themselves use STT to traverse IVR’s , no dtmf’s allowed here and the robocallers and telemarketers are at a loss for what to do :slight_smile: , extract the 10 least significants put in astdb /whitelist/number/date and /whitelist/number/destination where destination is ‘mum’, ‘voicemail’, ‘cell’, ‘desk’, ‘sales’ , ‘hangup’ , ‘lenny’, ‘SIT’ etc

2 Likes
9

My experience is much different. The anti-spam IVR on my personal DID is simply 1 second of silence, followed by “For Stew, please press 7, or stay on the line to leave a message.” I have never had even one spammer get past that. Unfortunately, there are quite a few false positives because callers aren’t listening at the beginning, or don’t understand the request. Repeating the prompt would help, but that would leave me open to live telemarketers (who come on the line after the first prompt is over).

I agree that STT and other AI could be used to build a much better filter, but it needs to be carefully thought out and tested. Asking for the called party’s name would likely reject both the credit card fraud alert and the customer service callback examples in my earlier post.

1 Like
10

Replacing any IVR was the kicker for me , you can repeat the question as often as needed, whether man or machine, if they don’t know who they are calling then chances of passing my test are slim, they can also press 0 or 7 until they are blue in the face.

I find the responses to be mostly one or two words, For an individual, John, Smith , John Smith or Mr. Smith, For a Company, Sales, Accounts (receivable,Payable) , Operator, or a name . All of which are easily consumed by Google or Amazon

If there is no response or lots of words, almost always a machine

Added bells and whistles are soundex if your company name is ZYZZYX Widgets Incorporated, Additionally asking for the number they called or a random question like ’ what month it is’ or ‘what weekday is it’ also tricks machines , adding an in-call feature codes to wrap transfer around a function to also change /whitelist/3235551234/destination to that destination.

Can’t say it’s well thought out because it was me who did the thinking :wink: but it is tested all the time, no complaints yet except for one girl who’s announcement was “You have reached the Whitehead family, who do you want to call?” better that with “You have reached the number you dialed, who do you want to speak to?” for Companies a more conventional “Hi there you have reached Biggies Wholesale, do you want sales or customer service?”

1 Like
11

Lots of excellent thoughts from you guys here – thanks! I definitely will look at an auto white list outbound callers in the short term. Longer term I need to learn more about the FreePBX IVR capabilities. My thinking is I don’t want to replicate functionality from other modules but provide some way of allowing other modules to access and update the whitelist.

I do think adding some additional data about when and how (auto or manual) a number was whitelisted will be good. Not really sure what to do about providers CID formatting though. One of the issues I ran into on another project is that these hooks seem to be available too early in the dial plan so Superfecta and such have not had a chance to manipulate the CID iinformation.

I think an API interface to enable/disable the whitelist could be tied to a home automation system so the controls don’t necessarily have to be on a phone.

Last night just for grins I was able to build a WHITELIST() function for asterisk but its not really necessary for this application to work with all the powerful tools available in dialplans now.

The one thing I’ve been stymied too is getting a full FreePBX build environment up and running. I can rebuild asterisk from source but FreePBX instructions are not working for me right now – more stuff to dig into.

12

The last 10 digits would be identical for all providers because that is always a unique phone number at least in NANPy land (non digit characters , which although ‘illegal’ are possibly seen in the wild, should be stripped first just in case )

I would suggest an agi script as the hook as you can do your Superfecta database lookup stuff in that.

If you want to manipulate anything in astdb use a wrapper around rasterisk -x 'database (whatever)' as directly working on the sqlite3 astdb while asterisk is running will sooner or later cause a crash.

Perhaps use one of the recipes in the wiki to build freepbx for your OS of choice on Vultr or DO or . . . That allows an easy snapshot of a basic system and then a clone or restore is only a few minutes away.

There are quite a few MQTT recipes also here and on your HA forum for interacting bidirectionally between Asterisk and the HA system.

1 Like
13

Hmm, I have MQTT messages coming out of my FreePBX system regularly but nothing going into it – I do have a home occupied flag that is maintained by my HA system that would probably be useful to send to Asterisk.

I’m looking at the AGI scripts Lorne Gaetz posted in the robocalls thread mentioned above. Those might help with looking up contacts in the Contact Manager as well.

Ultimately if you are using contact manager then the actual whitelist is a bit redundant, I’m thinking of adding a switch to treat all contact manager entries as whitelisted. That doesn’t sound too complicated.

Any thoughts on the best place to hook into the outbound dialplan to automatically add called numbers to the whitelist?

14

Particularly "How do I do X on every outbound call?"

I would start with

[macro-dialout-trunk-predial-hook]
exten => s,1,NoOp(Entering user defined context macro-dialout-trunk-predial-hook in extensions_custom.conf)
exten => s,n,DumpChan()
exten => s,n,Set(DB(whitelist/${DIAL_NUMBER:-10}/date)=${EPOCH}) 
exten => s,n,Set(DB(whitelist/${DIAL_NUMBER:-10}/destination)=We_called

; additional lines here 
exten => s,n,MacroExit

which will show all the channel variables you can read (and sometimes re-write ) in your agi script.

1 Like
15

I found macro-dialout-trunk-predial-hook but it appears to be intended for admins to customize their own systems rather than for FreePBX Modules to overwrite. I’m looking where it is called, but for now I am working on the whitelist from contact manager – I shamelessly copied Lorne Gaetz’s lgaetz-cmcheck.php to do the contact manager lookup – reading up on how to properly package agi scripts with modules.

16

Well I got whitelisting of users in the contact manager working. I packed the agi script following what I found in the superfecta module, seemed pretty reasonable. I’m not checking on whether the contact manager is installed or not so that’s on the todo list.

But now I’m off to figure out automatically adding outbound callers to the whitelist.

17

I got auto adding called numbers working at a prototype stage.

Also I tested the bulk import on a file that I pulled of our dialed numbers out from our CDR info and bulk uploaded it to a CNAME site. So that whitelist pretty much anyone we’d want to get a call from now.

I’m going to have to relook at the Contact Manager whitelisting – I think that script should check asterisk phonebook as well, and handle the case where Contact Manager is not installed.

18

This module splices into the macro-dialout-trunk context which might be useful for you:

I see you’re reusing the returnhere channel var from the blacklist module, it might be wise to change the name to something different in your module.

There appears to be a recentish realization that the name “blacklist” is culturally insensitive, and should be renamed to “BlockList”. That being the case, if there is not already a corresponding preferred name for whitelist, I suggest ‘AllowList’, with the added benefit that they will be listed together alphabetically. I would only just rename the GUI elements without changing the underlying rawname/code.

1 Like
19

Good point about the ‘sensitive’ wording – that will be some sed scripting time, I’ll change the module name and all the references to the word ‘allow’ – I must have missed the channel var there, I’ll fix that as well.

The name of the module has been changed to allowlist. I’ve also fixed the channel var collision. Anyone who starred or followed the repository will have to do it again.

I also edited the title of this thread as well.

1 Like
20

How did you compute the number (-4) to use for $spliceposition ?