Hints disappearing

I’m having a weird issue with a system I just migrated from v13 to v15 using Backup and Restore. The hints will all of a sudden disappear, and asterisk starts to throw warnings about the extension doesn’t exist in from-internal.

So, hints after a fwconsole reload show as such:

    -= Registered Asterisk Dial Plan Hints =-
[email protected]       : PJSIP/104&Custom:DND  State:Unavailable     Presence:not_set         Watchers  2
[email protected]       : PJSIP/105&Custom:DND  State:Unavailable     Presence:not_set         Watchers  2
[email protected]       : PJSIP/106&Custom:DND  State:Unavailable     Presence:not_set         Watchers  2
[email protected]       : PJSIP/107&Custom:DND  State:Unavailable     Presence:not_set         Watchers  1
[email protected]       : PJSIP/100&Custom:DND  State:Idle            Presence:not_set         Watchers  4
[email protected]       : PJSIP/101&Custom:DND  State:Idle            Presence:not_set         Watchers  2
[email protected]       : PJSIP/102&Custom:DND  State:Idle            Presence:not_set         Watchers  4
[email protected]       : PJSIP/103&Custom:DND  State:Idle            Presence:not_set         Watchers  4
[email protected]       : PJSIP/108&Custom:DND  State:Idle            Presence:not_set         Watchers  1
[email protected]       : PJSIP/109&Custom:DND  State:Idle            Presence:not_set         Watchers  1
*[email protected]    : ccss:PJSIP/199        State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : park:[email protected]   State:Idle            Presence:not_set         Watchers  0
*[email protected]    : park:[email protected]   State:Idle            Presence:not_set         Watchers  0
*[email protected]    : park:[email protected]   State:Idle            Presence:not_set         Watchers  0
*[email protected]    : park:[email protected]   State:Idle            Presence:not_set         Watchers  0
*[email protected]    : park:[email protected]   State:Idle            Presence:not_set         Watchers  0
*[email protected]    : park:[email protected]   State:Idle            Presence:not_set         Watchers  0
*[email protected]    : park:[email protected]   State:Idle            Presence:not_set         Watchers  0
*[email protected]    : park:[email protected]   State:Idle            Presence:not_set         Watchers  0
[email protected]       : PJSIP/111&Custom:DND  State:Idle            Presence:not_set         Watchers  1
[email protected]       : PJSIP/110&Custom:DND  State:Idle            Presence:not_set         Watchers  1
[email protected]       : PJSIP/113&Custom:DND  State:Unavailable     Presence:not_set         Watchers  1
[email protected]       : PJSIP/112&Custom:DND  State:Unavailable     Presence:not_set         Watchers  1
[email protected]       : park:[email protected]   State:Idle            Presence:not_set         Watchers  0
[email protected]       : park:[email protected]   State:Idle            Presence:not_set         Watchers  0
[email protected]       : park:[email protected]&  State:Idle            Presence:not_set         Watchers  0
[email protected]       : park:[email protected]   State:Idle            Presence:not_set         Watchers  0
[email protected]       : park:[email protected]   State:Idle            Presence:not_set         Watchers  0
[email protected]       : park:[email protected]   State:Idle            Presence:not_set         Watchers  0
[email protected]       : park:[email protected]   State:Idle            Presence:not_set         Watchers  0
[email protected]       : park:[email protected]   State:Idle            Presence:not_set         Watchers  0
[email protected]       : park:[email protected]   State:Idle            Presence:not_set         Watchers  0
*[email protected]    : PJSIP/199&Custom:DND  State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : ccss:PJSIP/102        State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : ccss:PJSIP/103        State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : ccss:PJSIP/100        State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : ccss:PJSIP/101        State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : ccss:PJSIP/106        State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : ccss:PJSIP/107        State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : ccss:PJSIP/104        State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : ccss:PJSIP/105        State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : ccss:PJSIP/108        State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : ccss:PJSIP/109        State:Unavailable     Presence:not_set         Watchers  0
_*[email protected] : Custom:DEVCF${EXTEN:  State:Unavailable     Presence:                Watchers  0
*[email protected]    : ccss:PJSIP/113        State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : ccss:PJSIP/112        State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : ccss:PJSIP/111        State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : ccss:PJSIP/110        State:Unavailable     Presence:not_set         Watchers  0
_*[email protected]: Custom:DEVDND${EXTEN  State:Unavailable     Presence:                Watchers  0
[email protected]       : PJSIP/199&Custom:DND  State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : PJSIP/106&Custom:DND  State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : PJSIP/107&Custom:DND  State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : PJSIP/104&Custom:DND  State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : PJSIP/105&Custom:DND  State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : PJSIP/102&Custom:DND  State:Idle            Presence:not_set         Watchers  0
*[email protected]    : PJSIP/103&Custom:DND  State:Idle            Presence:not_set         Watchers  0
_*[email protected]    : ${DB(AMPUSER/${EXTEN  State:Unavailable     Presence:                Watchers  0
*[email protected]    : PJSIP/100&Custom:DND  State:Idle            Presence:not_set         Watchers  0
*[email protected]    : PJSIP/101&Custom:DND  State:Idle            Presence:not_set         Watchers  0
*[email protected]: Custom:DAYNIGHT0      State:Idle            Presence:not_set         Watchers  1
*[email protected]    : PJSIP/108&Custom:DND  State:Idle            Presence:not_set         Watchers  0
*[email protected]    : PJSIP/109&Custom:DND  State:Idle            Presence:not_set         Watchers  0
_*[email protected]: Custom:FOLLOWME${EXT  State:Unavailable     Presence:                Watchers  0
*[email protected]    : PJSIP/113&Custom:DND  State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : PJSIP/112&Custom:DND  State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : PJSIP/111&Custom:DND  State:Idle            Presence:not_set         Watchers  0
_*[email protected]   : MWI:${EXTEN:3}@${DB(  State:Unavailable     Presence:                Watchers  0
*[email protected]    : PJSIP/110&Custom:DND  State:Idle            Presence:not_set         Watchers  0

Then, after a few minutes I start to receive the below error:

NOTICE[2036]: res_pjsip_exten_state.c:421 new_subscribe: Endpoint '100' state subscription failed: Extension '100' does not exist in context 'from-internal' or has no associated hint

but for each subscription on each phone. During this time, running another core show hints reveals that many of the hints have been removed:

    -= Registered Asterisk Dial Plan Hints =-
*[email protected]    : park:[email protected]   State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : park:[email protected]   State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : park:[email protected]   State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : park:[email protected]   State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : park:[email protected]   State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : park:[email protected]calls   State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : park:[email protected]   State:Unavailable     Presence:not_set         Watchers  0
*[email protected]    : park:[email protected]   State:Unavailable     Presence:not_set         Watchers  0
[email protected]       : park:[email protected]   State:Unavailable     Presence:not_set         Watchers  0
[email protected]       : park:[email protected]   State:Unavailable     Presence:not_set         Watchers  0
[email protected]       : park:[email protected]&  State:Unavailable     Presence:not_set         Watchers  0
[email protected]       : park:[email protected]   State:Unavailable     Presence:not_set         Watchers  0
[email protected]       : park:[email protected]   State:Unavailable     Presence:not_set         Watchers  0
[email protected]       : park:[email protected]   State:Unavailable     Presence:not_set         Watchers  0
[email protected]       : park:[email protected]   State:Unavailable     Presence:not_set         Watchers  0
[email protected]       : park:[email protected]   State:Unavailable     Presence:not_set         Watchers  0
[email protected]       : park:[email protected]   State:Unavailable     Presence:not_set         Watchers  0
_*[email protected] : Custom:DEVCF${EXTEN:  State:Unavailable     Presence:                Watchers  0
_*[email protected]: Custom:DEVDND${EXTEN  State:Unavailable     Presence:                Watchers  0
_*[email protected]    : ${DB(AMPUSER/${EXTEN  State:Unavailable     Presence:                Watchers  0
*[email protected]: Custom:DAYNIGHT0      State:Idle            Presence:not_set         Watchers  1
_*[email protected]: Custom:FOLLOWME${EXT  State:Unavailable     Presence:                Watchers  0
_*[email protected]   : MWI:${EXTEN:3}@${DB(  State:Unavailable     Presence:                Watchers  0

in particular all of the @ext-local hints that the phones use for their BLF subscriptions. Running a fwconsole reload will temporarily resolve this.