FreePBX | Register | Issues | Wiki | Portal | Support

SensuSangomaCheckService CRITICAL: rsyslogd DOWN


(Tom Brock) #1

Upgraded to FreePBX 14.0.1.4.

Now I am getting a Constant RMS response every few minutes.

SensuSangomaCheckService CRITICAL: rsyslogd DOWN

Not really sure how to track this down.
Any direction?

Also, Not sure where this should be categorized at!

Thanks
PS New to linus so I am sorta lost!!


(Bryan Walters) #2

Can you please open a ticket at support.sangoma.com under the RMS department so we can gather some additional information to be able to further assist with this?


(Marbled) #3

Hi!

How? Are we talking of the FreePBX distro or something deployed on another distro?

Did you use the FreePBX distro distro upgrade “script” or the pbx upgrader module?

Can you issue

chkconfig rsyslog on
service rsyslog start

I too had a non-working Syslog server after the FreePBX 14 distro upgrade…

Linux… :wink:

Linus (Torvalds) is the name of the guy who created Linux…

Good luck and have a nice day!

Nick


(Tom Brock) #4

Hey, Actually, Linus was a Fat Finger Typo… FFT

I had installed a 13 distro and got it configured, then decided to follow the directions for distro upgrade to 14.
The one where I had to use command line to download the upgrade and install it.

I issued the two commands you stated above.
The first one… responded with a list of commands that could be used via the service rsyslog command line.
The second, responded with a redirect statement…

i then did a service rsyslog status,
it replied with active and running since (date/time) 19 seconds ago.

Will see if that clear’s the problem, and if that fix holds after a reboot before I go through the ticket approach.

Thanks, and I will update you after I see if that takes care of it.

Greatfully
Tom

I did get a condition cleared email. Will give it an hour, then reboot and see what happens.
Thanks


(Marbled) #5

Hi!

:wink:

OK, so it is the FreePBX distro…

Sorry, I got one of them wrong, I just edited my post…

I don’t have access to my test system right now so I could not check if rsyslog use SysV or Systemd… The fact you got a redirect suggests it’s SystemD…

(There’s still a mix of both on the distro, just like what it is derived from, Centos… I thought rsyslog was one of the services started the old way…)

I was so unsure which type it was that it got reflected in the command I gave you… A Systemd service you normally always use systemctl with, a SysV one you have to use two different commands…

That said, the commands should be transformed in their equivalent which should be (take this with a grain of salt, I have not taken enough :coffee: today, I keep on getting things slightly wrong…):

systemctl enable rsyslog 
systemctl start rsyslog

Actually it won’t since I got the command which should have done that wrong… Can you issue a

systemctl enable rsyslog

(if rsyslog is indeed Systemd you won’t get a redirect message when you issue that command…)

at the command line…

Actually if this is the result of a distro upgrade you might want to follow the troubleshooting section of the wiki page you got the procedure from and create a ticket (a “Distro upgrader” one though, not what Bryan had asked you to do and post the logs it asks for in it…

Since this is the result of a distro upgrade, I am curious about something…

Since your upgrade, has anything been added to your CDR (report)? This is something which frequently has problems…

Good luck and have a nice day!

Nick


(Tom Brock) #6

Hey, Nick,

Thanks for all the info…

But the whole thing just flat lined!!!

that seemed to have resolved the issue, so I went to resolve an error on the dashboard about an email for fax for a user,
When I added the email, saved it, and went to "Apply Config"
it seems to have locked the whole thing up. Cant get it to do anything…

Oh, just as i typed that, the browser popped up an error.

Error!
Error: Did not receive valid response from server

XHR response code: 500 XHR responseText: undefined jQuery status: error

So now, I am back to not knowing the direction, maybe I should have just stayed at 13… I sure dread all that set up again.

thanks
Tom


(Marbled) #7

Only that issue and it will only stick if you issue the corrected command…

Only the web interface or has it stopped processing calls?

Most likely an Ajax call failing or something similar that is failing because the system no longer responds…

You do know that the upgrade you ran is a beta, right?

That said I don’t remember anyone reporting that her/his system was working and stopped working…

Are you still able to access the system remotely by SSH ir directly from the console?

That’s the only way we can hope to debug it…

Good luck and have a nice day!

Nick


(Tom Brock) #8

the console seems to be responding, its at command prompt,

i am logged in…

I just dont know a lot about using it.

I didnt realize it was beta, but this is just for me to use, no big production, not a major deal.
just something for me to tinker with along

i also have access to gui just cant seem to reload the configuration.

but it is allowing me to navigate around

I just dreading thinking about having to remember and type it all in again :wink:


(Tom Brock) #9

Nick,

I have been tinkering around with it this evening, not sure what I have done, but seem to have everything resolved. Afraid to reboot , with my luck, lol

I cant tell you what I have done, but, it has all resolved and RMS has cleared everything, first time since I stepped up to 14…

Only thing I wonder about now is, Seems like memory use is a little high… running about 70% on average…

Hope the Reboot does not cause it to blow up!!!

Thanks for all your time and info.

Regards
Tom


(Tom Brock) #10

and of course, after reboot, i am back to problems, red button, “Can not Connect to asterisk”

RMS ’ reporting failures now too “CRITICAL: asterisk, fail2ban, rsyslogd DOWN”

So, i am back to trying to pick it out.

Tom


(Tom Brock) #11

went to console, issued command "fwconsole restart"
it seems to have helped start the services, but, now i get a big banner at top when i do anything that says "undefined"
stays a couple of seconds and goes away.

so something not right


(Marbled) #12

Hi!

FreePBX 14 itself is stable…

The FreePBX 14 / Sangoma 7 distro is stable…

The pbx upgrade module for people who have FreePBX 13 on a non-distro system is stable…

However, the distro upgrade “script”, the “script” which converts a FreePBX 13 distro install into a FreePBX 14/Sangoma 7 install is very much beta…

Chances are you will get a working install but there’s a risk of having one or more things that don’t quite work right and need fixing…

I think that you went further but I wish you could have sent screenshots of why you could not…

Good luck and have a nice day!

Nick


(Marbled) #13

Hi!

When the system has not fully completed its startup you can see that…

Personally I don’t use RMS so I can’t comment but for a little period of time asterisk down could be ok…

Fail2ban people currently have problem with having it perform as they want (ie no jails), I didn’t think it could show up that way… Does the situation resolves by itself after a minute or so?

For rsyslogd, did you issue the systemctl enable rsyslog command I asked you to do (when I realized my typo earlier)?

Good luck and have a nice day!

Nick


(Marbled) #14

Hi!

I have the same problem, it’s a bug… The devs know about it…

It will eventually be fixed…

If your only problem end up being this, you are Ok until the bug is resolved…

Good luck and have a nice day!

Nick


(Tom Brock) #15

Thanks for all the info.

Where I am leaving it for tnight is it seems to be ok, even after the last reboot.

Still memory seems a little high, and, not sure about the CDR showing the calls, will have to test that in the AM and see what happens

thanks
Tom


(Marbled) #16

Hi Tom!

Did you change anything or was the problem caused by the boot not having fully completed when you tried to access it (and, of course, that undefined bug…)?

http://www.linuxatemyram.com/ maybe?

If you end up having problem with it take a look at this:

That problem seems relatively common and, for now, has only two known causes…

Good luck and have a nice day!

Nick


(Tom Brock) #17

Good Morning Nick,

Been trying to process some of the info you listed on the CDR.

I had restarted some services, and rebooted again, and everything had came up fine after that,

I think the memory is a little high because it keeps triggering the RMS alarm. running about 72%, and jumping up to about 81% even when sitting idle… that was why I thought there was something up with memory.

now, back to CDR… its not logging calls.

I am going through some of the data in the link you provided.
Came across one post that said to do a
yum install asterisk14-odbc

I issued that command from the console itself.

I end up with an conflict error.

Asterisk13-core conflicts with Asterisk14-core…

so, does that mean that something didnt get upgraded when the upgrade script ran?

If so, what do i do? is there a way to re run the upgrade script and see if it will correct the issue?


(Marbled) #18

Hi Tom!

You didn’t pick up the right line for your version (I am pretty sure I provided both)…

You need to type

yum install asterisk13-odbc

since you are running Asterisk 13 on your FreePBX 14… Tthat’s perfectly legit, don’t worry… I do the same…

For more info on Asterisk version numbering you can take a look at this: https://wiki.asterisk.org/wiki/display/AST/Asterisk+Versions

I assume you already checked if your /etc/odbc.ini file was present?

Good luck and have a nice day!

Nick


(Tom Brock) #19

I was just thinking 14… wasnt thinking ASTERISK being 13… what I get for thinking.

Yea, the /etc/odbc.ini is present…

Tom


(Tom Brock) #20

If I do a systemctl restart odbc.service
it replies with failed, unit not found