I was able to get it to work to my liking using one of the recipes on the web page I referred to…
The accents/special characters were already mangled and I was willing to correct the entries which had them so as long as I was not losing anything else I was ok with correcting any mangled entry…
I also had a very recent backup (yesterday)…
I did many tries (restoring from my backup after each).
Essentially what I ran was this:
DB="asterisk"
(
echo 'ALTER DATABASE `'"$DB"'` CHARACTER SET utf8 COLLATE utf8_unicode_ci;'
mysql "$DB" -e "SHOW TABLES" --batch --skip-column-names \
| xargs -I{} echo 'ALTER TABLE `'{}'` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;'
) \
| mysql "$DB"
and
DB="asteriskcdrdb"
(
echo 'ALTER DATABASE `'"$DB"'` CHARACTER SET utf8 COLLATE utf8_unicode_ci;'
mysql "$DB" -e "SHOW TABLES" --batch --skip-column-names \
| xargs -I{} echo 'ALTER TABLE `'{}'` CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;'
) \
| mysql "$DB"
I modified the script I had to use the UTF-8 collate sequence documented here: [FREEPBX-8057] Wrong charset in CDR records/connection - Sangoma Issue Tracker
The first time I tried this it didn’t work, I got the following error message
ERROR 1071 (42000): Specified key was too long; max key length is 1000 bytes
This was caused by the table asterisk.faxpro_hook_core which is used by the “Fax Configuration Professional” module. Wanting to play it safe I restored from my backup and I then deinstalled it the module (so that it would remove the table cleanly).
I reran both scripts.
This time it completed successfully…
I then reinstalled the “Fax Configuration Professional” module and disabled it (this is what I do with modules I don’t currently use but are part of the distro, I don’t normally uninstall them.
It recreated the table but now the page and id fields (which are part of the key) are no longer 255 characters long but 150… I guess with the conversion to UTF-8 the key was now too long (the max is 1000 for the type of index used on this table apparently) and has been shortened… As this table was created before I switched to FreePBX 13 it had the old length…
I then tried to enter characters which were not supported by Latin1 (ISO-8859-1)…
I tried characters which needed ISO-8559-16 (or a variant of Unicode) and even Cyrillic and both were correctly handled…
(I have a very basic understanding of Romanian and Russian…)
I then fixed all the entries which had their characters mangled because of the initial upgrade to FreePBX 13, there weren’t that many…
The only problem I noticed is that some screens seem not to handle the apostrophe (ie ’ ) correctly but I don’t think this is related to runing this script, they probably need to escape them. The IVR screen (IIRC) did but you see the backslash it precedes the apostrophe with in some places while the time group screen seems to discard an entry that has it altogether (probably because the SQL fails to execute because of that (most likely unescaped) apostrophe…)…
Obviously this is not for everyone but as for me I am happy with the results… The accents/special characters were already mangled to begin with so as long as I wasn’t losing anything else I was ok with correcting them after…
Thank you and have a nice day!
Nick