Anybody else experiencing issues:
They are having another major service outage. They are calling it âDegraded performanceâ but all of our inbound through them right now is hard down.
Vitelity is not experiencing DDoS. They have had major internal infrastructure issues recently. I have a previously scheduled call with a regional VP later today and will provide updates here.
1 Like
Is Vitelity part of the DDoS attack (or part of sip.us - since thereâs a response here from sip.us)? Itâs been fine all week and havenât had issues until just today. Unless they just got targeted today?
Really looking forward to hearing any updates you have in the near future after your meeting with the regional VP. We use Vitelity for about 90% of our end users - the other 10% use VoIP.ms. The last month or so has been extremely detrimental to our business with inbound/outbound call issues week to week with Vitelity and then VoIP.ms being down for close to a week after the DDoS attacks.
My operations manager wants me to look into other options for SIP trunk/voip services with all the issues weâve had recently with Vitelity/VoIP.ms
Well your outbound I at least suggest routing elsewhere for now. For inbound, youâll need to port those numbers out if you want them spread out over different carriers.
Call with RVP and Inteliquent COO is postponed until next week since they cannot pull a technical resource for the call due to todayâs issues⌠
This is going to be a tough one in the US with STIR/SHAKEN. You end up giving your users a B attestion score which can cause some carriers (mobile for sure) to mark it unverified or even block it.
Thereâs gotta be a better way to handle it besides providing any backup outbound providers a list of your DIDs and settling for a B score, right? It seems like that hasnât really been thought through.
Actually it was thought through very well. STIR/SHAKEN is an Operating Carrier level exchange. You, as a wholesaler, are not an Operating Carrier therefore you must go through one that will sign calls on your behalf. So that means you have to follow their rules and the rules are if the callerid you present to them isnât theirs, they mark it as a B. You can get around this by becoming an Operating Carrier and then sign the calls as valid off your network.
Things are changing and VoIP now has all sorts of eyes on it due to IP becoming the primary transportation of things. During the Kariâs Law discovery it was found that VoIP providers were woefully negligent in having 911 locations even registered for their users. I know this is true, Iâve dealt with a few over the years and they all cite the same reason âWhy pay for it when it is barely used? Itâs a waste of moneyâ. Wrong.
More along the lines of STIR/SHAKEN there was a point where VoIP providers, even registered as a CLEC, still could not be their own number authority and had to get DIDs from an existing tier 1 carrier. They loosened that up a few years back and there is now correlating data that shows allowing VoIP providers to be a number authority is in line with the uptick in the amount of SPAM/robo calls that are being generated now. To the point, the FCC may reconsider allowing VoIP providers to be number authorities. Which will suck and force us to user Tier 1âs for STIR/SHAKEN compliance.
I am convinced this is how they are going to get control of things. They are going to hammer down on the Tier 1 players to make it trickle down to their downstreamâs who will also have to trickle it down to theirs if they want to keep calls flowing. There are too many low tier VoIP providers who just connect FreePBX (or other FOSS systems) to places like VoIP Innovations or Vitelity and âlet carrier deal with thingsâ in regards to telecom compliance. However, they present themselves to customers as a provider that can give them voice services and all sorts of features with the service.
I fully believe the time of people playing at being a provider while passing the buck to their upstreamâs and their platforms is having the sun set on it. You want to be a voice provider, you have to start acting like a voice provider.
(and Iâm not directing this at anyone. âYouâ is being used in a very broad sense.)
So the biggest concern here is carriers marking calls with a B attestation as unverified. Regardless of our thoughts of ITSPs and âvoiceâ providers (I agree with your points on that), the bottom line is that your outbound traffic is at risk of being delegitimized unless you are willing to stick with a single provider for class A attestation, or sign the calls yourself.
Correct which is a way to ensure that providers are playing ball with the same rules. Being a provider shouldnât be as easy as signing up for a platform in 5 minutes.
But if I have 5 terminating providers and they have rules about CPS and call limits, I can then spread my calls over them to avoid these limits. That is a valid thing to do but it also means that bad actors can do the same thing (which is what they do now). Most of the time the callerid being used is spoofed so no way it belongs to any of the terminating carriers or it belongs to another carrier not being used for termination.
Is this a perfect solution as is? No and Iâm sure it will be improved upon. However, the more and more IP (VoIP) becomes prominent the more and more robo/spam calls are increasing. Part of that is due to being able to throw calls out a dozen networks at once.
The other problem is that mobile carriers have different rules to play by than landline/VoIP carriers. They are allowed to block both incoming and outgoing calls over their network without consent of the user. Landline/VoIP, however, has to terminate the call. We canât block it because the destination is $0.10 to call but mobile carriers can.
So copper/SS7 based carriers donât do STIR/SHAKEN while IP based has no choice. For the most part the B level score isnât going to matter to landline/VoIP providers, they wonât really take action on it. They will leave that to the user or provide a service the user must opt for (even if auto optin is done). The SS7 carriers wonât care. The mobile carriers are the only ones that will be the concern.
I mean havenât you also noticed that mobile carriers have been prepending a + in front of any number presented? This includes valid US numbers in 10 digit formats make a call from the 313 area code look like +3135551212 and will even display it as âNetherlandsâ for caller name. I had users complaining about that for months, I had to start presenting 11 digit callerid for them to get call backs from mobile phones because trying to call back from call history on the cell phone attempted a international call to +3135551212 instead of calling the US.
In regard to Vitelity, I had a call with them today which included the Director of their Voice Engineering and their NOC director. Basically:
- They have been working towards upgrading their infrastructure to be in line with Inteliquent, and on the same network as theirs. This is largely the cause, or indirectly related to the cause of their recent issues in the last few months.
- They have had to roll back their outbound platform migration, but are saying that they have roughly 20% of their customer base on the new platform running without issue. However, being on this new platform would not have avoided the service issues they had on 9/22 and 9/30-10/1. The issues they had on 9/14 were not experienced by anyone on the new platform.
- They are hoping to have their infrastructure upgraded by âThanksgivingâ is what they said. Worst case is Q1 2022. They did not inspire confidence when speaking about possible issues between now and then.
- The issue from 9/30-10/1 seems to be something different, but they were not able to tell me anything other than it being an âExterior Network Eventâ. This in particular sounds different than the previous issues since they really could not provide me with any information, though I am only speculating.
I wouldnât be surprised if they have more issues in the next few months related to their infrastructure. I hope this information is useful to anyone reading.
This is interesting. Toward the end of last month there seems to have been multiple players who were impacted by either BGP routing issues or DDoS attacks. Not saying this was the case with these guys, but at least a half dozen service providers I use sent out generic explanations as to what was going on around that timeframe. Look at the Facebook issues yesterday. They supposedly botched their BGP router configs and that results in a ~4 hour outage.
Other folks who have also apparently âbotchedâ recently
voip.ms
bandwidth.com
facebook.com
Best advice , âjust donât Botch ;-)â
1 Like
We can go on all day speculating and going off-topic about facebook 
I will post back here once a post-mortem becomes available.
With respect, the call you had seems to me so far to just reflect âtheir speculationâ, no?