SIPStation Totally Down!?

General Help
1

I have 2 customers on the FreePBX SIPSTation SIP trunking product. They are both in retail sales & Saturday is their biggest day. They are both currently totally down. FreePBX module shows Asterisk registration request has been sent with no response. This is for both the primary & secondary SIP trunks in the SIPStation module in FreePBX.

I have emailed [email protected] to report the issue & request immediate help. They only provide Mon-Fri next business day email response for support. I think this type of support availability is totally unacceptable for situations when you are down. It is a different story for support issues where the customer’s system is at fault.

It seems that it was a mistake for me to put customers that do business on Saturday on your service. I assumed (big mistake) that your service had high availability & wouldn’t be down on weekends. You have the secondary SIP trunk configured with the service, which gives the impresison that both should hopefully/likely not be down at the same time.

Now I have 2 customers today each with 4-6 sales people who are losing business because their phones don’t ring in on this busy shopping Saturday. I have no way to contact a human being working today who actually has the ability to forward their DIDs to their cell phones so that they can at least field sales calls.

I am hoping that maybe someone here who works for SIPStation or who has contact details with someone who does, will see this post & notify someone who can actually get the service back up today.

2

Both trunks are now back up on both customers’ systems. I have not made any changes or restarts to FreePBX or asterisk. Internet has been up the entire time. Firewalls have not been restarted. I assume this was an issue on the side of SIPStation.

If someone actually saw my post or email & fixed it, thank you.

On a different note, I have been in touch with a sales person from Bandwidth.com & will be moving both customers to their service which provides 24x7 telephone support.

3

That’s interesting as Bandwidth.com is the provider of SipStation.

It has been said many times before that the FreePBX project does not run the infrastructure nor have any input on the product. Bandwidth.com is the core corporate sponsor of FreePBX.

On a personal note I am sorry you had difficulty with something that was recommended on this web site.

4

Doug,

I don’t know what was down in the infrastructure this morning, if anything. I did check my logs on a server that maintains connection with sipstationa and it showed down also. If I get an update I’ll let you know.

I can assure you that the infrastructure is monitored 24x7. Although the support is M-F, which is stated and part of the cost structure, the infrastructure is monitored and supported 24x7 and if something is down, it is attended to as fast as possible.

There is no question that it is nice to be able to call into support and talk to someone, which you will be able to do with the direct Bandwidth.com support. However, the reality is that if the service does go down it will be up as quickly as possible regardless of being able to call in.

So, always frustrating if something does go down and having dual servers geographically diverse is designed to minimize this, it is definitely being attended to on the very infrequent occasion that it happens though.

5

First off, I do appreciate the responses.

@SkykinOH - I do know that the Bandwidth.com infrastructure is what SIPStation uses. In fact, this was a part of why I chose to use SIPStation. Having used Bandwidth.com SIP trunking in the past & being aware of their venerable position & service, I was comfortable of the idea of using SIPStation, knowing that it was “really” kind of reselling Bandwidth.com’s SIP trunking, and simply excluding the phone-based support, etc.

More recently I was speaking with a Bandwidth.com rep & he said that although SIPStation uses Bandwidth.com’s infrastructure & the contract for service actually seems to be directly with Bandwidth.com, it is NOT actually the same SIP trunking service, and it is NOT using the same Bandwidth.com infrastructure/equipment that Bandwidth.com’s native SIP trunking service does. He made specific reference to the fact that SIPStation does SIP registration to authenticate our SIP endpoint & establish the IP calls need sent to, whereas Bandwidth.com does not use SIP registration & instead requires a static IP address on our side, which they setup as the SIP endpoint on their end. Having used Bandwidth.com’s SIP trunking in my company a few years ago, I remembered this & realized that he was probably right, at least to the extent that the SIP trunk is setup differently. On the other hand, it is clear to see that the SIP trunks setup for SIPStation are pointing the SIP endpoints trunk1.phonebooth.net & trunk2.phonebooth.net. Phonebooth being the hosted PBX service from Bandwidth.com, it would seem that these are actually the same SIP trunk endpoints that Bandwidth.com uses for their Phonebooth hosted PBX service. But the Bandwidth.com sales guy gave me the impression otherwise. He also specifically stated that the Bandwidth.com SIP trunking service has more redundancy built-in & more geographic locations from which they can send calls into us, and used the static IP address requirement & lack of SIP registration being required to support this assertion.

So I am a bit unclear as to who is correct here. Maybe everyone is & there is more to the story/picture.

@Philippe - Again, thank you very much for the response. Receiving a response usually makes a tremendous difference to me. There is nothing more frustrating than having a service down that is completely out of your (my) control & feeling that you are speaking into a black hole as your only channel of support for resolution. So THANK YOU! :slight_smile:

Aside from that,

  1. It seems that someone edited my 2nd post/reply & somehow the last sentence got added to my reply rather than yours or Scott’s.

  2. So if you don’t see any sign of problem with the infrastructure/logging, etc. maybe it was a transient issue with Level3 or other upstream ISP that Bandwidth.com uses…?

Regarding the service in general:
3. I have been trying to determine the best way to financially directly support the FreePBX project, as we use it to create solutions for our small business customers & we certainly don’t want it to go anywhere. I felt like the SIPStation SIP trunking service was valuable & something I could trust as being more or less equivalent to direct from Bandwidth.com. This ptrunkingresented to me an excellent way to start pushing some kind of (although currently pretty minor) financial support toward the FreePBX project (I’m sure you guys don’t make too much on these individual trunks, but something is better than nothing, so if I can do something, then I will). It was easy for me to justify using SIPStation for these reasons, as the SIP service itself is a necessity of course.

  1. Regarding moving from SIPStation to Bandwidth.com, one of my clients on SIPStation is currently paying CenturyLink over $18/mo to maintain a market expansion line & their toll free #. SIPStation doesn’t support toll free, so they are paying CL to forward the toll free to the market expansion line & the market expansion line to the company’s main DID which is currently in SIPStation’s system. If you guys supported having a toll-free number on their account, they could drop that $18/mo service from CL & save some money. Bandwidth.com does support having a toll-free #, and I think it is $6-8/mo. Bandwidth.com is also willing to negotiate pricing in such a way that it seems like the customer will be winning on both ends by switching to them. Add that 24x7 support & it just seems like a good move. Add to that the fact that they’ll set me up as a referral partner & I get a commission too & it again strengthens their position.

  2. Truthfully if you guys just had a feature in the SIPStation web interface that enabled us to forward their DID to someone’s cell phone, this would have even been a major issue. Still inconvenient, but definitely workable. I know the SIPStation FAQ has said this is coming & you guys are working hard on it, but I think it has said that for nearly a year. Has anyone really been working on it?

  3. Sorry that I made such a big fuss this morning in the first place. I was seeing the situation as both of these customers having absolutely NO phone service ALL DAY today & that would have been a very big problem for them & me as their I.T. & phone guy. However, it seemed that the issue only lasted until around 10:45-11am Mountain Time. Still having now phones for 2-3 hours in the morning is not great for these customers, but it’s a lot better than the worst-case scenario that I thought I was looking at.

Also, if it helps in troubleshooting or figuring out what the problem was, I found out that their phones would actually ring on inbound calls, but after answering, there was NO audio for either party. So it seems like RTP media wasn’t making it through either way. I was also told that outbound calls would be silent & then just hang up. I’m not sure whether those actually rang through to the called party.

Thanks.

6

Doug, I added the sentence to your second post. I wanted to update my post and I must have gone too fast and edited your post. I should be more careful editing knowing I have an o-line at this site.

In any event you seem to have a good handle on things. Too bad you are not in Norther Ohio, I would love to sell you show you some of my on-net SIP/IAX trunking products. Knowledgeable users are hard to come by.

My entire intent, and it seems you understand this. Is none of the team on the FreePBX side has access to the SIPStation platform or could do anything for you. Certainly everyone cares when things don’t work as they should. I know from personal experience Philippe is a strong advocate of this community.

7

dkmortensen,

remember you are talking to a ‘sales’ guy, who is definitely telling you what he understand to be correct.

The ‘phonebooth’ SIP proxy’s you point to are simply part of a scaleable architecture that Bandwidth has that their hosted phone service, some trunks and a lot more run on top of. To make a long story short, Bandwidth has one very solid backbone SIP architecture that is used with everything from wholesale to SIPSTATION to hosted phone service to their republic wireless cell phone offerings. There are different entry points, proxy systems, etc that serve different purposes (such as enabling the ability to register vs. direct static IPs) but in the end, and what really matters, is all the media ultimately flows through the same media servers and backbone and interconnect you to the PSTN.

8

Hello Philippe, Scott & others,

It is Saturday 12:57pm Mountain Time. Our client is down again. Inbound calls fail. They don’t even hit the client’s asterisk server. Outbound calls are succeeding. The Asterisk full log shows registration to trunk2.phonebooth.net is timing out continually.

Needless to say this is very disruptive & frustrating for our client. I have sent an email to [email protected] to report the issue & request the DID be forwarded to a manager’s cell phone for now. I have not received a response & have no assurance of whether anyone will even look at my email until Monday. Maybe the issue will not even be resolved until mid-day Monday (if no one looks at it until Monday morning).

I am pleading for someone to please take 10-20 minutes out of their Saturday to look at this. I am currently on vacation. But realizing that our client (who is in retail sales [Saturday is their biggest day]) is completely down & helpless, I am willing to take a few minutes out of my vacation to look at this remotely & see if I can help them. I am asking one of you to please be willing to do the same & give a few minutes to look at this.

Additionally a relevant side-note, we have since switched one of our clients from SIPStation to Bandwidth.com SIP trunking. They are having no problems today. They also have been unaffected the few other times in the past 4-6 months that SIPStation has had more minor disruptions. Both clients are using Comcast for internet & are located in the same city. So I am starting to believe that there is indeed something different with the Bandwidth.com SIP infrastructure which causes it to be more reliable that SIPStation.

In any case I would greatly appreciate any help any of you may be able to give at this time.

Sincerely,

9

Doug,

Not sure if I made it clear in the past or not that I nor Tony have any direct contact with Bandwidth that would be able to expedite service on a weekend.

I don’t know if Philippe does however what I do know is that he is traveling or in the process of traveling to meet Tony, Bryan and myself at Astricon.

If there was anything I could do for you I would not hesitate.

If you are in the continental US and require a carrier that has SLA’s and returns customer calls on the weekend please feel free to PM me as I may have some options for you.

Again, sorry for your trouble.

10

Scott,

Thanks for the response. I didn’t know whether anyone here had the ability to access the hosting infrastructure for SIPStation or have the ability to influence the people there who do. In the past I got the impression that Philippe had visibility into the technical services status on the Bandwidth.com SIP servers.

In any case, we are certainly at the mercy here of anyone willing to help, since the SIPStation service doesn’t include Saturday support.

We actually are in the process of migrating to Bandwidth.com’s SIP trunking, due to too many issues like this. SIPStation’s unreliability certainly isn’t worth saving $5/channel monthly. All it takes is 1-2 missed sales due to outages & it would have paid for a year’s worth of the additional $5/channel to Bandwidth.com to have the reliability.

The intro page at sipstation.com makes the distinction between SIPStation & Bandwidth.com based on support. But I think that is inaccurate. Surely that is a difference. But it is not the primary difference. The primary difference is reliability. Rather than the Bandwidth.com shirt reading “I love support”, it should read “I love reliability”. And reliability should be a primary distinction that is emphasized on that page in order to be accurate.

The info that on that page was one of my primary indicators for me to feel comfortable going with SIPStation. I asked myself “what do I need support for”? To help setup my trunks? To assist in troubleshooting if I have trouble? I am very comfortable with asterisk & SIP, and I think that I can hold my own in terms of accurate troubleshooting & trunk setup. So I think I’ll save the money and forgo the 24x7 telephone support. The page makes it clear that both services are “Powered by Bandwidth.com the SIP trunking leader”. That gives the impression that they are using the same infrastructure & systems behind the scenes. Indeed this was how I helped my 2 customers become comfortable with it. I used Bandwidth.com SIP trunking service as the ONLY inbound/outbound telco service for my business for about 2 years previously. I could attest to my customers from first-hand experience that Bandwidth.com is a first-rate high reliable SIP trunk provider. I presumed that as long as Bandwidth.com’s SIP trunking service is up (which is extremely likely due to the number of customers using it), then the SIPStation service would also be up. That’s where I was wrong.

So now of my two clients located about 1 mile from each other using the exact same Asterisk/FreePBX systems, both using Comcast, the one utilizing Bandwidth.com’s service has been up during at least the past 2-3 outages experienced by my customer who is still on SIPStation. And this current outage has already lasted 6 business hours (9am-3pm) on the busiest retail day of the week (Saturday). At this point it is clear that I made a major mistake and have done my customer a huge disservice by using SIPStation SIP trunking. I wish that the sipstation.com intro page would have made reference to reliability. Because that would have steered me in the right direction. But it did not.

With that said, I still would like to support the FreePBX project financially, but will not be doing so via SIPStation any longer. I have been considering replacing the Rhino appliances in our lineup with FreePBX appliances available via Schmooze. I might do further research into this later.

But for now, I have got to get our customer’s SIP trunk back up ASAP.

Thanks Scott. And if there is anyone else here who can help, I sure would appreciate it. My customer’s retail business is still open for a few more hours today.

Sincerely,

11

Doug,

I feel your pain, a service that is endorsed by Schmooze/FreePBX needs to be a LOT more responsive to keep that endorsement and S/F need to confirm that endorsement and/or caveate that they are only for hobbyists or remove it, I stopped using them when they required inbound but that inbound was flakey at best.

Perhaps Vitelity can get you up and runnung in short order for outbound, they always have for me. For inbound you are totally hosed though until banker’s hours on Monday.

FWIW and for the future, having redundant service is critical to keep your customers happy. Outbound is easy, Inbound less so but many VSP’s allow a fail over to PSTN or server to cover that situation.

Good Luck

12

The sipstation.com front/info page which spells out the key differences between the two services (SIPStation & Bandwidth.com) should state the following very clearly for SIPStation: “We won’t fix weekend outages”

That is really the primary problem here. I don’t need support right now. There is nothing wrong with my customer’s asterisk box. Support is not what I need. The problem is not on my end. It is SIPStation’s service available. They are down. Because it is a weekend, my understanding is that they are not committed to do anything about it.

Anyway, outbound calling is working fine. Inbound is where the problem is.

Thanks for your time. Enjoy the rest of your weekend. I am going to get back to my vacation now.

13

I am envious of SIPStation, neither you nor I have that laxity, Generally a telco will guarantee 5 9’s 99.999% or about 3 minutes a year of downtime, shame on any provider that doesn’t even care to offer 1 nine support.

I wonder if your problems are symptomatic of the precursor to a widening of the DOS attack against CallCentric last week. . . . .

14

Dicko,
Just to set the record straight, Schmooze has no involvement with SipStation, nor have we had involvement with SipStation.

15

I fully understand and there was no such inferential claim at least not from me, as I currently look directly at the right sidebar on this particular page , and that is omnipresent on most every page in this site SIPStation has cornered the market on this site, presumably they paid for that privilege, so to say that there is NO “involvement” might be reasonably questioned, no?

Many of those squibs state flatly out front that “not all VOIP is created equally” given the apparent current problems, I can only agree, some actually work at the weekend also ;-). I am simply holding there feet to the fire, not Schmooze, can we expect a response before they have their wheaties on monday morning. ?

16

Let me be clear, Bandwidth owns this site, the FreePBX trademark and all associated intellectual property so Schmooze has no more editorial control than any other admin.

It would be hard for Philippe to tell his employer they can’t advertise on their own site.

17

So my customer was officially down from 9am-5:30pm Saturday. Pretty horrible for a business in retail sales.

The service ended up coming back up & SIP trunk registration succeeded again at that time. I did nothing to cause this. It was on the side of SIPStation as the service provider.

Probably not much more I need to say here. Anyone interested in using SIPStation for business (or any other critical) phone service should carefully read this thread, and they can then make their own intelligent decision.

With that said, I do maintain the recommendations I made on Saturday regarding the changes that should be made to the sipstation.com intro page. The issue is not a distinction of support. Yes support is important and is a clear difference between these two options. But MUCH more important than support is reliability (particularly regarding weekend outages not being given attention that they should). If there are significant reliability issues, talking about support is a moot point.

Philippe. I would urge you to make this change to the sipstation.com intro page. Reliability should be given the greatest emphasis in terms of making the distinction between the two services. The “powered by Bandwidth.com” for both services is misleading. It gives the impression that the reliability for either service should be the same.

I know you guys are solid individuals and work hard to provide a great service (developing & managing the FreePBX product). I know that you do not want customers to have bad experiences with your product or recommended associated products & services. This is why I would urge you to make this change on your intro page. Be fair to your customers. And don’t invite more unsuspecting small business VoIP guys like myself to end up having similar experiences like what I & my clients have had to endure. FreePBX is a reputable product. Please don’t damage its credibility by having repeats like this.

With that said, I want to thank you for your work & dedication to the FreePBX project. We enjoy it & it is a nice product for our customers. We appreciate the people who support open source projects like this. As I have previously expressed, I am desirous to support the project financially where it seems logical for my clients. I thought SIPStation would be a nice way to do this. I was wrong. At some point in the future, I will look further into the appliances you guys are selling.

Until then, I’ll be working this week to get my last SIPStation client moved over to standard Bandwidth.com SIP trunking.

18

I don’t think anyone was asking anybody to “bite the hand that feeds you”, but to extend the canine metaphor, please just don’t roll over and play dead when there is a fox in the hen-house, there is nothing wrong with a dog barking back at “His Masters Voice”, he is fed and sheltered to guard and protect his master’s interests after all and he at least is awake every day of the week and certainly knows where his master snores.

Presumably, it now Monday and the problem has been fixed for now.

FWIW, in retrospect, and over the years, every significant attack against my systems have started early on a Sunday morning, To my eternal chagrin, some where successful . . .

19

Philippe does not manage the sipstation website.

20

I think we all appreciate the independent original thinking of FreePBX.

That “Bandwidth owns this site, the FreePBX trademark and all associated intellectual property” comes as a bit of a kick in the face to me a noisy and inquisitive open-sourcer. I was not aware of that in any license I accepted, mea culpa for making assumptions.

Perhaps someone can explicitly state all and any associations between FreePBX, Schmooze, Bandwidth and SIPStation, never mind the finger pointing when things don’t work, just please so state explicitly who pays what to who and who does what for who, who has what responsibility for what and who talks over the water cooler to who.

ref:-

[root@reminders freepbx-2.10.0rc1]# grep -R “bandwidth.com” .
./amp_conf/htdocs/admin/views/footer_content.php:$html .= ‘Bandwidth.com’ . br();
./amp_conf/htdocs/admin/modules/sipstation/sipstation.utility.php:phonebooth.bandwidth.com:5060 9192221234 585 Timeout Sat, 27 Jun 2009 00:33:47

and

Copyright © 2012 Bandwidth.com FreePBX is a Registered Trademark of Bandwidth.com All Rights Reserved. Bandwidth.com and Schmooze Com, Inc. are the proud sponsors of the FreePBX project.

not so much “owners” anywhere here.