RestApps Parking "Loading Application" Timeout's

juesor · February 7, 2017, 5:33pm

I have a user that is using rest apps parking but occasionally get an error on the screen that says “loading application” then it times out.

My first place to look is in httpd error logs and i did the following. These times line up with when the user was having this issue.

httpd]# cat error_log | grep 1X.XXX.X.14
[Mon Feb 06 08:22:09 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Mon Feb 06 16:09:47 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Mon Feb 06 16:09:50 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Mon Feb 06 16:09:52 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Mon Feb 06 16:09:55 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Mon Feb 06 16:10:01 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking

The oddity is that this is not 100% causing failure messages. Parking works and then it doesn’t then strings of times it doesn’t like above.

lgaetz · February 7, 2017, 5:44pm

Try upgrading Phone Apps to edge:

fwconsole ma --edge upgrade restapps
fwconsole r

juesor · February 7, 2017, 7:58pm

Phone Apps 13.0.86.9 Stable Sangoma Technologies Corporation Commercial Enabled

I was on.

Phone Apps 13.0.86.8 Stable Sangoma Technologies Corporation Commercial Enabled

I’ll watch it do you have release notes on 13.0.86.9

juesor · February 10, 2017, 8:26pm

Still this morning and i upgraded when you requested on the 7th.

Phone Apps 13.0.86.10 Stable Sangoma Technologies Corporation Commercial Enabled

I upgraded just now to 13.0.86.10

cat /var/log/httpd/error_log | grep client

[Mon Feb 06 08:22:09 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Mon Feb 06 10:00:56 2017] [error] [client 1X.XXX.X.11] client denied by server configuration: /var/www/html/restapps/voicemail
[Mon Feb 06 10:01:11 2017] [error] [client 1X.XXX.X.11] client denied by server configuration: /var/www/html/restapps/voicemail
[Mon Feb 06 10:01:26 2017] [error] [client 1X.XXX.X.11] client denied by server configuration: /var/www/html/restapps/voicemail
[Mon Feb 06 10:01:27 2017] [error] [client 1X.XXX.X.11] client denied by server configuration: /var/www/html/restapps/voicemail
[Mon Feb 06 10:01:29 2017] [error] [client 1X.XXX.X.11] client denied by server configuration: /var/www/html/restapps/voicemail
[Mon Feb 06 10:01:41 2017] [error] [client 1X.XXX.X.11] client denied by server configuration: /var/www/html/restapps/voicemail
[Mon Feb 06 16:09:47 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Mon Feb 06 16:09:50 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Mon Feb 06 16:09:52 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Mon Feb 06 16:09:55 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Mon Feb 06 16:10:01 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Tue Feb 07 13:19:38 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Tue Feb 07 13:38:03 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Tue Feb 07 14:56:06 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Tue Feb 07 15:11:58 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Wed Feb 08 15:03:44 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Wed Feb 08 16:19:15 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Fri Feb 10 06:59:30 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/voicemail
[Fri Feb 10 06:59:43 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/voicemail
[Fri Feb 10 06:59:45 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/voicemail
[Fri Feb 10 06:59:51 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/voicemail
[Fri Feb 10 06:59:57 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/voicemail
[Fri Feb 10 06:59:58 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/voicemail
[Fri Feb 10 10:06:56 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Fri Feb 10 10:46:20 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Fri Feb 10 10:46:23 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Fri Feb 10 10:46:25 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Fri Feb 10 10:47:10 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking

Do you have any other suggestions? Should i submit a ticket online outside of forums?

cynjut · February 10, 2017, 9:01pm

This might be a permissions issue. Have you issued the following commands:

fwconsole chown
fwconsole reload

juesor · February 16, 2017, 10:27pm

I did both commands this morning At this time the last time i got the error was at 10:24:37 today.

Logs are below happened every day multiple times a day i will watch for it tomorrow.

[Mon Feb 13 07:50:47 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Mon Feb 13 07:59:23 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Mon Feb 13 09:01:37 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Mon Feb 13 09:16:15 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Mon Feb 13 10:10:32 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Mon Feb 13 11:01:51 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Mon Feb 13 11:26:50 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Tue Feb 14 10:12:30 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Tue Feb 14 10:44:37 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Tue Feb 14 11:19:46 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Tue Feb 14 11:20:43 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Tue Feb 14 12:22:26 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Tue Feb 14 12:45:02 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Tue Feb 14 15:33:42 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Wed Feb 15 07:32:01 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Wed Feb 15 09:07:10 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Wed Feb 15 09:18:42 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Wed Feb 15 09:19:37 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Wed Feb 15 09:19:42 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Wed Feb 15 09:21:11 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Wed Feb 15 09:24:21 2017] [error] [client 1X.XXX.X.17] client denied by server configuration: /var/www/html/restapps/parking
[Wed Feb 15 09:25:49 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Wed Feb 15 09:25:52 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Wed Feb 15 10:23:42 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Wed Feb 15 10:24:17 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Wed Feb 15 11:16:34 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Wed Feb 15 12:12:55 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Wed Feb 15 14:11:28 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Wed Feb 15 15:19:53 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Wed Feb 15 16:04:09 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking
[Thu Feb 16 10:24:37 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking

juesor · February 17, 2017, 4:13pm

[Fri Feb 17 06:58:04 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/voicemail
[Fri Feb 17 06:59:59 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/voicemail
[Fri Feb 17 07:00:07 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/voicemail
[Fri Feb 17 07:00:17 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/voicemail
[Fri Feb 17 07:02:54 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/voicemail
[Fri Feb 17 07:02:56 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/voicemail
[Fri Feb 17 09:28:10 2017] [error] [client 1X.XXX.X.14] client denied by server configuration: /var/www/html/restapps/parking

Same issue today.

Seems to still be down. Suggestions ?

cynjut · February 17, 2017, 5:43pm

Did you notice that it’s not the exact same issue?

There is something hinky in your webserver permissions below the restapps directory. Have you looked in the directory (from an SSH connection or from the console) to see what kinds of things are in there (like maybe a .htaccess file that shouldn’t be there)? Also, look at the permissions on the /var/www/html/restapps file and make sure they make sense…

I don’t actually have a solution for you - still trying to figure out where the issues are.

juesor · February 21, 2017, 8:35pm

I think you are thinking i am not in an ssh session every day. Or that i don’t know that asterisk:asterisk should be the owner to /var/www/html/*

But don’t worry this isn’t my first server.

If you look client denied by server configuration is always voicemail and parking. So i know its both apps that are being denied.

restapps]# ll
total 120
-rw-rw-r-- 1 asterisk asterisk 5133 Feb 10 14:25 application_callforward_freepbx.zip
-rw-rw-r-- 1 asterisk asterisk 6408 Feb 10 14:25 application_conferences_freepbx.zip
-rw-rw-r-- 1 asterisk asterisk 5131 Feb 10 14:25 application_contactmanager_freepbx.zip
-rw-rw-r-- 1 asterisk asterisk 5134 Feb 10 14:25 application_daynight_freepbx.zip
-rw-rw-r-- 1 asterisk asterisk 5950 Feb 10 14:25 application_donotdisturb_freepbx.zip
-rw-rw-r-- 1 asterisk asterisk 6406 Feb 10 14:25 application_endpoint_freepbx.zip
-rw-rw-r-- 1 asterisk asterisk 6232 Feb 10 14:25 application_findmefollow_freepbx.zip
-rw-rw-r-- 1 asterisk asterisk 6109 Feb 10 14:25 application_parking_freepbx.zip
-rw-rw-r-- 1 asterisk asterisk 6731 Feb 10 14:25 application_presencestate_freepbx.zip
-rw-rw-r-- 1 asterisk asterisk 5135 Feb 10 14:25 application_queues_agent_freepbx.zip
-rw-rw-r-- 1 asterisk asterisk 5128 Feb 10 14:25 application_queues_freepbx.zip
-rw-rw-r-- 1 asterisk asterisk 6181 Feb 10 14:25 application_restapps_freepbx.zip
lrwxrwxrwx. 1 asterisk asterisk 53 Jan 10 09:14 applications.php -> /var/www/html/admin/modules/restapps/applications.php
-rw-rw-r-- 1 asterisk asterisk 5135 Feb 10 14:25 application_timeconditions_freepbx.zip
-rw-rw-r-- 1 asterisk asterisk 5129 Feb 10 14:25 application_voicemail_freepbx.zip
-rw-rw-r-- 1 asterisk asterisk 5138 Feb 10 14:25 application_voicemail_transfer_freepbx.zip
lrwxrwxrwx 1 asterisk asterisk 46 Jan 10 09:22 image.php -> /var/www/html/admin/modules/restapps/image.php
lrwxrwxrwx. 1 asterisk asterisk 49 Jan 10 09:14 restapps.php -> /var/www/html/admin/modules/restapps/restapps.php
lrwxrwxrwx. 1 asterisk asterisk 45 Jan 10 09:14 sync.php -> /var/www/html/admin/modules/restapps/sync.php

As you can see asterisk:asterisk owns everything i’ve checked this against another freepbx vm i have and permissions are the same.

I also have no idea why this is happening. Do you see this in /var/log/httpd/error_log on your servers?

cynjut · February 21, 2017, 9:26pm

OK. Things we know you know:

File permissions and directory permissions are established at the creation of the file/directory. The process that owns the file/directory is the owner and the group that owns the higher-level directory is the group of the directory.
You’ve double checked to make sure that the owner of the webserver (which is where the problem appears to be coming from) is the same as the owner of Asterisk. Note this is different than the owners of the files that are executed by the webserver (sync.php).
The problem is clearly in the web interface (or one of the web interfaces) because you are getting an Apache error when you try to access the files.

There are only a couple of possible points for the problem. The first is that you have an “.htaccess” file somewhere in your configuration that is restricting access to the web assets. Another possibility is that you have something in your Apache Configuration that is keeping the server from accessing the files.

Things to look for are whether the “follow symlinks” options are set up correctly and who the owner of the files are when they get created.

Note that this problem is a disconnect between Asterisk and Apache. Something in one of the configurations is causing you to either not have access to the directory, or not have permissions to the files once you get there.

I do not use UCP or restapps, but I’ve been using Asterisk and Apache for many, many years. The problem you are having is clearly a problem is Apache. Whether it’s being caused by FreePBX or an artifact from some other source is the only question in my mind. You also need to check the IP address ranges for your Allow/Deny configurations in Apache. If you have “tightened up” the security by only allowing access to the webserver from 127.0.0.1, you will get an error like this. This can be done in a configuration file or in a ‘.htaccess’ file.

Simply reposting that your stuff is broke is not helping me help you get to the bottom of this. To be clear - the problem is in your Apache Configuration. I don’t know what or where, but the problem is definitely a problem in Apache.

juesor · February 22, 2017, 10:30pm

I agree.

Lets continue from here.

This is not happening 100% of the time. The user can use a restapp to park fine and then 10 minutes later it fails. If for any reason is works that shows that it is not an issue with security.

Now to answer your points.

The ownership of the files are correct as other servers are working with no issues.
Yes I’ve double checked that the user that is starting httpd is correct.
Agree.

Now I can navigate to the URL

http://XXX.XXX.XXX.XXX:84/applications.php/parking/main?linestate=$$LINESTATE$$&user=1101

{“application_name”:“parking”,“application_display”:“Parking”,“page_name”:“main”,“type”:“action”,“exitPath”:null,“layout”:[{“type”:“transfer”,“value”:“70”,“dtmf”:"##"}],“action”:[],“error”:[]}

So that tells me that i am getting some response from the parking app.

I agree that its something in restapps and apache and not asterisk. I’m going to resort to entering an official ticket.

juesor · May 11, 2017, 1:23pm

Ok so this inheritable looks like an issue with phoneapps

/var/www/html/restapps/voicemail

Do you have that directory on your servers?

cd /var/www/html/restapps/voicemail

-bash: cd: /var/www/html/restapps/voicemail: No such file or directory

I do not have that directory but it seems like every time they hit the voicemail button it tries to go there and fails.