Finally got this working, reporting my issues here in case someone with similar issues stumbles upon this thread.
The T54W phones would not download the xml files by https (may be because I do not have a valid certificate setup). I could via the xml files in browser on https but phones would wait awhile and fail.
I am doing auto provisioning via http, but my default admin port was set to 8080. I could not download php/xml files via http with admin port on 8080.
Once I changed the admin port back to default 80, I was able to download php/xml files via http.
Phones can now download the php/xml files via http.
Auto Provisioning was another headache. Eventually I found out that I had the config settings wrong, was using features.remote_phonebook.data.1.url instead of the correct remote_phonebook.data.1.url
My working basefile edits (using EPM):
directory_setting.remote_phone_book.enable = 1
remote_phonebook.data.1.name = Internal
remote_phonebook.data.1.url = http://<pbx-ip>/ylab.php
features.remote_phonebook.enable = 1
remote_phonebook.data.2.name = External
remote_phonebook.data.2.url = http://<pbx-ip>/cm_to_yl_ab.php
With directory_setting.remote_phone_book.enable = 1
You do not need the favorite_settings.xml to enable the remote directory. But it will put local directory as the first option and remote phonebook as the second option after hitting the directory key.
I have added this code to change the order:
directory_setting.local_directory.priority = 2
directory_setting.remote_phone_book.priority = 1
Or to get rid of the local directory
directory_setting.local_directory.enable = 0
Yealink has a Config Parameter Excel file that has all the config parameters and reading it is what lead me to find my mistake.
Big thanks to JaredBusch for maintaining the php scripts. Was going to buy PhoneApps just for phonebook feature, which may not even be supported on all yealink phones.