Possible new source to block "bad" numbers (telemarketers, etc.)

Ok I’ve got a beta version of the lookup going. Here is the address if you want to give it a try

http://www.everycall.us/query?502-447-4497

Simply enter http://www.everycall.us/query? and then the phone number. It will return a score if it is in the database and -1 if the number is not in the database. Please feel free to give it a test and let me know if you think any changes need to be made.

We are going to look into the verification of username/secret code to prevent abuse. Any comments on that would be helpful too.

Thanks.

Okay, here’s what I did. I warn you up front this DOES NOT WORK, but I can’t figure out where I am going wrong. First I created a file /share/var/lib/asterisk/agi-bin/check-everycall-us.agi which contains this:

BADCALLSCORE="0"
RETRIEVEDSCORE=`/usr/bin/curl -s -m 2 -A Mozilla/4.0 http://www.everycall.us/query\?${1}`
BADCALLSCORE=`echo ${RETRIEVEDSCORE} | grep score | sed -e 's/.*score=//' -e 's/ <br\/>//'`

if [ -z $BADCALLSCORE ]
then
	BADCALLSCORE=0
fi

echo "SET VARIABLE badcallscore ${BADCALLSCORE}"

This works fine if I run it from the command line (it gets the score of a number I put in). Of course you do have to remember to change the permissions and ownership so they match the other agi files in that directory (must be executable, and user and group are asterisk, etc). Also be careful not to lose any of the backticks when copying the code.

Now the part that is oh so close, but giving me fits. I added this to /etc/asterisk/extensions_custom.conf (if YOU do this make sure there’s not already a from-pstn-custom context - if there is you’ll have to add this code to it):

[from-pstn-custom]
exten => _.,1,NoOp(Checking Everycall.us for bad caller)
exten => _.,n,AGI(check-everycall-us.agi,${CALLERID(number)})
exten => _.,n,NoOp(Everycall.us returned score of ${badcallscore})
exten => _.,n,GotoIf($["${badcallscore}" < "10"]?notbadcaller:badcaller)
exten => _.,n(badcaller),NoOp(Bad caller score above threshold, blocking call)
exten => _.,n,Congestion(20)
exten => _.,n,Macro(hangupcall,)
exten => _.,n(notbadcaller),NoOp(Bad caller score below threshold - continuing)
exten => h,1,Hangup()

The above may be a bit longer than it needs to be but keep in mind I’m still trying to debug it. The real issue seems to be the AGI call. I’m not doing it correctly, apparently, but I’m not sure what the correct format is. When a call comes in, I get an error like this (phone numbers have been munged for privacy reasons):

-- Executing [8005551212@from-trunk:1] NoOp("SIP/trunk1-08c47fa0", "Checking Everycall.us for bad caller") in new stack
-- Executing [8005551212@from-trunk:2] AGI("SIP/trunk1-08c47fa0", "check-everycall-us.agi|8005552368") in new stack
-- Launched AGI Script /var/lib/asterisk/agi-bin/check-everycall-us.agi

== check-everycall-us.agi|8005552368: Failed to execute ‘/var/lib/asterisk/agi-bin/check-everycall-us.agi’: Exec format error
!!! NOTE THE ABOVE LINE !!!
– Executing [8005551212@from-trunk:3] NoOp(“SIP/trunk1-08c47fa0”, "Everycall.us returned score of ") in new stack
– Executing [8005551212@from-trunk:4] GotoIf(“SIP/trunk1-08c47fa0”, “1?notbadcaller:badcaller”) in new stack
– Goto (from-trunk,2124007595,8)
– Executing [8005551212@from-trunk:8] NoOp(“SIP/trunk-1-08c47fa0”, “Bad caller score below threshold - continuing”) in new stack

So everything is working EXCEPT the actual AGI call. I’m 99% sure it’s just a syntax error but I’ve been looking at other examples and cannot figure out what I am doing wrong. I need to take a break from this so I’ll throw what I’ve done so far out there, in case anyone spots the obvious flaw in my code. By the way, suggestions for making it shorter/more efficient are also appreciated!

exten => _.,n,AGI(check-everycall-us.agi|${CALLERID(number)})

Actually I tried the bar instead of the comma first, and got the same error. Nevertheless I tried it again, but still it doesn’t work. I know, it looks like it SHOULD work, but it doesn’t.

DON’T USE THIS - KEEP SCROLLING DOWN!!!

/share/var/lib/asterisk/agi-bin/check-everycall-us.agi contains this:

#!/bin/bash

BADCALLSCORE="0"
RETRIEVEDSCORE=`/usr/bin/curl -s -m 2 -A Mozilla/4.0 http://www.everycall.us/query\?${1}`
BADCALLSCORE=`echo ${RETRIEVEDSCORE} | grep score | sed -e 's/.*score=//' -e 's/ <br\/>//'`

if [ -z $BADCALLSCORE ]
then
	BADCALLSCORE=0
fi

echo "SET VARIABLE badcallscore ${BADCALLSCORE}"

The from-pstn-custom context added to /etc/asterisk/extensions_custom.conf:

[from-pstn-custom]
exten => _.,1,NoOp(Checking Everycall.us for bad caller)
exten => _.,n,AGI(check-everycall-us.agi|${CALLERID(number)})
exten => _.,n,NoOp(Everycall.us returned score of ${badcallscore})
exten => _.,n,GotoIf($["${badcallscore}" < "10"]?notbadcaller:badcaller)
exten => _.,n(badcaller),NoOp(Bad caller score above threshold - blocking call)
exten => _.,n,Playtones(busy)
exten => _.,n,Busy(20)
exten => _.,n,Macro(hangupcall,)
exten => _.,n(notbadcaller),NoOp(Bad caller score below threshold - continuing)
exten => h,1,Hangup()

Seems to work on “good” calls (passes them through) but don’t know for sure that it will properly block bad ones, since I haven’t received any yet! However the bigger issue is it’s not returning to the proper point in the original context, resulting in incomplete calls.

Note: In the call to curl, the -m 2 option means to wait no more than two seconds for a response from Everycall.us.

The original (major) problem was apparently the lack of the #!/bin/bash, although I did catch a comma that should not have been there.

Until I or someone else figures out why it’s not returning to the proper point in the original context - otherwise you may have missed calls!

/var/lib/asterisk/agi-bin/check-everycall-us.agi still contains this:

#!/bin/bash

BADCALLSCORE="0"
RETRIEVEDSCORE=`/usr/bin/curl -s -m 2 -A Mozilla/4.0 http://www.everycall.us/query\?${1}`
BADCALLSCORE=`echo ${RETRIEVEDSCORE} | grep score | sed -e 's/.*score=//' -e 's/ <br\/>//'`

if [ -z $BADCALLSCORE ]
then
	BADCALLSCORE=0
fi

echo "SET VARIABLE badcallscore ${BADCALLSCORE}"

But instead of the context shown above in /etc/asterisk/extensions_custom.conf, use this (it may still be a bit more verbose than necessary) (EDIT: also see the next message for a possibly better alternative to this part):

[custom-check-everycall-us]
exten => _.,1,NoOp(Checking Everycall.us for bad caller)
exten => _.,n,AGI(check-everycall-us.agi|${CALLERID(number)})
exten => _.,n,NoOp(Everycall.us returned score of ${badcallscore})
exten => _.,n,GotoIf($[0${badcallscore} < 10]?notbadcaller)
exten => _.,n,NoOp(Bad caller score above threshold - blocking call)
exten => _.,n,Playtones(busy)
exten => _.,n,Busy(20)
exten => _.,n,Macro(hangupcall,)
exten => _.,n(notbadcaller),NoOp(Bad caller score below threshold - continuing)
exten => _.,n,Goto(from-trunk,${EXTEN},1)
exten => h,1,Macro(hangupcall,)

Now for what makes this more difficult - in every trunk you want to monitor, you need to go into the trunk setup and change the statement “context=from-trunk” to “context=custom-check-everycall-us” (without the quotes, of course). The good part about this method is you can choose to check incoming calls on some trunks but not others. The bad part is you have to change it in every trunk that you want to check. Also, if you are already using a custom context in a trunk for another reason, you’ll have to change the jump from that context into this one rather than from-trunk (and make sure the EXTEN variable is set correctly). For example, the fix to get incoming DID on a ViaTalk line becomes:

[custom-from-viatalk]
exten => _.,1,Noop(Fix ViaTalk DID, set DTMF mode)
exten => _.,n,SIPDtmfMode(inband)
exten => _.,n,Set(EXTEN=[i]your-viatalk-number[/i])
exten => _.,n,Goto(custom-check-everycall-us,[i]your-viatalk-number[/i],1)
exten => h,1,Macro(hangupcall,)

EDIT: This should still be considered experimental code. Use it at your own risk, and please let us know how it works for you, particularly if you actually receive any calls that score high enough to get “the treatment.” Also please watch for any “false positives” - keep in mind that I am NOT a programmer, so don’t rely on my code without testing it for yourself!

I don’t really like terminating calls outside of the usual FreePBX dialplan - I never know if I am causing unintentional bad things to happen. So if you use this context in /etc/asterisk/extensions_custom.conf rather than the ones in the previous messages…

[custom-check-everycall-us]
exten => _.,1,NoOp(Checking Everycall.us for bad caller)
exten => _.,n,AGI(check-everycall-us.agi|${CALLERID(number)})
exten => _.,n,NoOp(Everycall.us returned score of ${badcallscore})
exten => _.,n,GotoIf($[0${badcallscore} < 10]?notbadcaller)
exten => _.,n,NoOp(Bad caller score above threshold - changing DID to 9999999999)
exten => _.,n,Goto(from-trunk,9999999999,1)
exten => _.,n(notbadcaller),NoOp(Bad caller score below threshold - continuing)
exten => _.,n,Goto(from-trunk,${EXTEN},1)
exten => h,1,Macro(hangupcall,)

… then you can create an inbound route with a DID of 9999999999 and then select whatever destination you like. I personally like the option “Terminate call: Play Ringtones to the caller until they hangup” but you can do whatever you like, including creating a special IVR just for those folks, if that’s your bent. And you are terminating the call inside the FreePBX dialplan, which may have some implications for call detail recording, etc.

By the way, I don’t really know what would be considered a high score at everycall.us - I picked 10 as the threshold, but that may require some tweaking once you have some actual experience with this.

You emailed me and I replied, but your email address does not work. “451 Temporary local problem”

This thread is great though. I like Wiseoldowl’s idea of changing the DID on telemarketer calls. That never occurred to me.

Hi guys I just spent some time reading through the thread, seems like there has been much progress made.

Wiseoldowl- Thanks for all of your effort. Are you satisfied with the speed and performance? The reason that my email isn’t working is that I moved to a dedicated server and haven’t moved the email accounts yet. Minor details, sorry for that.

Anyhow, I would agree that a score of 10 or more is a good starting point to filter calls out.

I don’t have a PBX system (as I think I stated earlier) so I’m curious how the interface with EveryCall.us is working…?

Well, the speed of the lookups seems to be nearly instantaneous, as far as I can tell… when I watch the CLI (a terminal window that lets me watch the calls progressing through the switch) I don’t even notice any hesitation. However the only problem I have is that our system is basically a home system that is primarily used for “intercom” type service and although we do get occasional incoming calls, they primarily come from the same few numbers (people we know) and are always scored -1 (so far), which I assume means they are not in your list at all. Yes, occasionally someone else stumbles in, but that hasn’t happened in the last couple of days (should have had this up before the election!).

So I am not really the best person to test this, but since I started this thread I at least wanted to put in the effort to try to get something working. But in the testing I have done, I have yet to see the lookup slow down the call flow. So I can say that I am very satisfied with the speed of the lookups, and as far as I can tell the code I wrote is working fine, but I do wish that others would test it and report back (sometimes the person who writes the code is not in any position to judge it objectively!).

Thanks for your effort in getting the interface up and running!

For those that don’t want to have to wander through this thread every time you want to set this up, try this page:

http://www.freepbx.org/support/documentation/howtos/how-to-automatically-reject-calls-from-telemarketers-and-other-junk-cal

Please let me know (in this thread) if you spot any errors or if anything isn’t clear.

Thanks for your feedback wiseoldowl, sounds like the performance is good which is nice to hear. It would be great if we could get a few more folks testing the interface to get some more feedback.

Also your how to thread is awesome, great work. I might need to look into purchasing a PBX system for my house just so I can use this interface!!!

Another thought I had was to try and use a number spoofing service to try and replicate a junk call to confirm that it is handled correctly. I am not sure if there are any free services, but www.spoofcard.com is one that I see advertised… it might be worth a shot?

You don’t really need to purchase a PBX, just grab a PC that you are no longer using (as long as it is not older than dirt and has at least 512K memory it should work, though a meg is better, and it should probably have at least a 10 GB hard drive though some people have got by with a smaller one) and download the ISO file for any of the distributions you might like to try, burn that to a CD, and stick it in the CD-rom drive and boot it up, then follow the installation screens. You can have a working PBX in about an hour, though obviously there’s a bit of a learning curve after that. Note this will destroy anything previously on the hard drive.

Distributions that you could try include AsteriskNow, Elastix, and PBX in a Flash. If you are a raw beginner you might prefer the last one because of the amount of online documentation, but I don’t care for it because it makes some assumptions about how you will use your system that I don’t necessarily agree with (and also the last time we tried it, it was really slow in completing calls for some unknown reason). We use Elastix right now, which has a great online e-book you can download called Elastix Without Tears (link is to PDF file), but AsteriskNow was not yet available (in its current incarnation) when we did that install, otherwise we might have tried it (the only thing I don’t care for about Elastix is that they don’t automatically let you get to the “unembedded” FreePBX, by default you are taken to their branded version which isn’t always kept current with the current FreePBX version. But the “unembedded” FreePBX is only one click away, so it’s usually not that big of a deal, and they do have some nice additions that come with their package). Steer clear of Trixbox, they have “forked” the FreePBX distribution and are doing their own thing.

You don’t even need to buy any other equipment just to try it out - you can use softphones (I like the free edition of Zoiper, it is a cross-platform softphone) for testing. But eventually you may want to buy some SIP-based phones or, if money is tight, an unlocked Linksys PAP2-NA (note the NA suffix) adapter or two, into which you can plug regular analog phones (for business applications you can buy an adapter that supports 8 phones at once, but most businesses prefer the SIP-based phones - but you said you were going to try this in your home).

I will also say to beware of bad advice from certain people who lurk in the #elastix or #freepbx IRC channels, should you choose to go there. The people that come off sounding as though they know everything are often telling you how THEY would do things, which isn’t necessarily the BEST way. Don’t trust advice from anyone who isn’t a FreePBX developer (and note that they don’t spend much time on those IRC channels), at least not until you have lurked a while and figure out who knows what they are talking about and who is full of B.S., unless you like doing things the hard/expensive/wrong way! :slight_smile:

Thanks for the intro to the different PBX systems that are out there, I think I may have found a holiday project to work on ;). I assumed that I needed some specific hardware, not just a PC… I’ll look into what you suggested over Christmas and see if I can get something running at my home.

This will actually be quite easy for me, I will pickup one of the Linksys adapters you suggested and plug my Panasonic wireless phone system into that, which is the only phone I have in the house… everything else is wireless stations.

Thanks again for your help, much appreciated.

I want to gauge your interest in the value of being able to report spam calls that make it through to EveryCall.us directly from your FreePBX system. If it is possible to setup a code such as #555 which prompts the user to enter a small amount of data to report the call to EveryCall.us.

Here are a couple of my thoughts:

  • #555 (or similar) would prompt the user with a couple of options:
    • Report last call.
    • Pick a category of the call (1 - Unknown, 2 - Telemarketing, etc.)
  • Calls reported via #555 would automatically add the received call to the black list.
  • Need function for inbound whitelist.
  • With an EveryCall.us account, users could manage their FreePBX blacklist from the EveryCall.us user administration area. This would eliminate having to manage the blacklist through the PBX and would potentially allow administration of it from any computer.
  • With an account, users could manage their whitelist also.
  • With an account users to could log in to add more information like who called, add a call comment, etc.

What do you guys think? I imagine this would require significant effort on the FreePBX programming side of things, as it undoubtedly would on my side. However I could rather easily extend the EveryCall.us API to include this functionality.

Thoughts?

While I think this would be a good idea, it’s probably a bit beyond my coding abilities, but feel free to have at it. :slight_smile: And there is another issue - in FreePBX the “last calling number” feature just doesn’t seem to work right. If I pick up the phone and dial *69, it gives me “Information about your last call” - which is just wrong (it always reads an extension number that in fact is incapable of making an outgoing call to my extension). I don’t know if anyone else has that problem, and maybe I should have filed a bug report on it but I was never sure if it was something I might have done (though I can’t imagine what).

Another problem is that even if the “last call” thing did work right, there’s always a chance another call may come in just as you pick up to dial the code, and that might be considered the “last call” by the time you finish dialing, resulting in the wrong number being reported.

I’m NOT throwing cold water on your idea, in fact I think it’s a good one, but I am just saying that it appears for now you may not be able to trust the “last call” information stored by the system. Of course, I’m sure no one would mind if you actually discovered the bug and patched it. :slight_smile:

(Is anyone else having problems with the last call not being properly reported when you dial *69, particularly with incoming calls from outside the system, that is, NOT from another extension? I know that this USED to work but for some reason it does not appear to be working properly in the FreePBX 2.4.0 branch included in the current Elastix distribution).

Did they turn off this service?

http://www.everycall.us/query?502-447-4497

Hi,

I was just browsing on how to configure our FreePBX office phone systems in blocking some calls. I guess I found the answer here:

http://www.freepbx.org/support/documentation/howtos/how-to-automatically-reject-calls-from-telemarketers-and-other-junk-cal