Superfecta - SFDatabase - still no correct conversion of special characters (umlauts)

FreePBX Applications / Modules
1

Hi all,

since years there are problems with correct conversion of special characters in the Superfecta module. At least this is the case for the Superfecta SFDatabase module when the database already uses UTF-8 as standard characterset. Nowadays almost all databases should use UTF-8 as default, I think.

There has already been a similar thread about this issue. However, it didn’t work for me:

I was able to solve this issue by making some changes in Superfecta.class.php. The problem was to find out the input encoding. See the patch below. I hope this will help others, too.

--- a/Superfecta.class.php        2021-09-07 13:10:04.548720875 +0000
+++ b/Superfecta.class.php        2021-09-07 14:50:58.138476254 +0000
@@ -207,10 +207,15 @@
                        if (!empty($callerid)) {
                                $found = true;
                                //$first_caller_id = _utf8_decode($first_caller_id);
-                               $callerid = trim(strip_tags($callerid));
-                               if ($superfecta->isCharSetIA5()) {
-                                       $callerid = $superfecta->stripAccents($callerid);
+                               if (function_exists('mb_detect_encoding')) {
+                                       if (mb_detect_encoding($callerid, ['UTF-8'], true) == 'UTF-8') {
+                                               $callerid = utf8_decode($callerid);
+                                       }
                                }
+                               $callerid = trim(strip_tags($callerid));
+//                             if ($superfecta->isCharSetIA5()) {
+//                                     $callerid = $superfecta->stripAccents($callerid);    
+//                             }
                                //Why?
                                $callerid = preg_replace("/[\";']/", "", $callerid);
                                //limit caller id to the first 60 char
@@ -218,9 +223,11 @@

                                // Display issues on phones and CDR with special characters
                                // convert CNAM to UTF-8 to fix
-                               if (function_exists('mb_convert_encoding')) {
+                               if (function_exists('mb_detect_encoding')) {
                                        $this->out("Converting result to UTF-8");
-                                       $callerid = mb_convert_encoding($callerid, "UTF-8");
+                                       $encoding = mb_detect_encoding($callerid, ['ASCII', 'CP1252', 'ISO-8859-1', 'UTF-8'], true);
+//                                     $callerid = mb_convert_encoding($callerid, "UTF-8");
+                                       $callerid = iconv($encoding . '//TRANSLIT', 'UTF-8', $callerid);
                                }

                                //send off
1 Like
2

I also have to edit superfecta-class.php…for an Austrian template (herold.at)…my text looks like this:

			$callerid = trim($callerid);

			$found = false;
			if (!empty($callerid)) {
				$found = true;
				$callerid = strtr($callerid, array("ü" => "ue", "ö" => "oe", "ä" => "ae", "Ü" => "Ue", "Ö" => "Oe", "Ä" => "Ae", "ß" => "ss"));
				$callerid = trim(strip_tags($callerid));
				if ($superfecta->isCharSetIA5()) {
					$callerid = $superfecta->stripAccents($callerid);
				}
				//Why?
				$callerid = preg_replace("/[\";']/", "", $callerid);
				//limit caller id to the first 60 char
				$callerid = substr($callerid, 0, 60);

				// Display issues on phones and CDR with special characters
				// convert CNAM to UTF-8 to fix
				if (function_exists('mb_convert_encoding')) {
					$this->out("Converting result to UTF-8");
					$callerid = iconv('ISO-8859-15//TRANSLIT', 'UTF-8', $callerid);
				}

				//send off

The problem is that I always have to add my workaround, when I update Superfecta :frowning:

Do you think that your patch works for me too?

3

Hi,

the changes that I made are similar to yours. The biggest difference is that I don’t explicitly convert some characters but try to find out what the original encoding is. I was able to make some successful tests with ISO-8859-1 and UTF-8.

Yes, I think it should work four you, too. But at the end you should try it yourself. Please let me know if there are any problems.

If everything is fine I will make a pull request in Github.

Thank you in advance

4

I suspect that the substitution code is a legacy of times when phones wouldn’t display the special characters properly, that still may be the case for some phones.

A PR at github will be ignored. Please have a signed CLA on file and raise a PR at git.freepbx.org

5

No…I needed (and still need) the substitutions for all my phones…Cisco, Sangoma and Digium. My knowledge is very limited, and I only had a problem with the herold.at-template/site. It could well be that Herold.at does not properly advertise its characterset.
I will try the patch…thanks.

6

Ok…I can confirm…the new patch (@VoipChicken) works great with Herold.at. All Umlaute are correctly displayed. Wonderful…thanks a lot!!!

This is the text I used (just copied the text into the class file, starting at line 208…hopefully it is correct):

                        $found = true;
                       if (function_exists('mb_detect_encoding')) {
                               if (mb_detect_encoding($callerid, ['UTF-8'], true) == 'UTF-8') {
                                       $callerid = utf8_decode($callerid);
                               }
                        }
                       $callerid = trim(strip_tags($callerid));

                        //Why?
                        $callerid = preg_replace("/[\";']/", "", $callerid);
                        //limit caller id to the first 60 char
							$callerid = substr($callerid, 0, 60);
                        // Display issues on phones and CDR with special characters
                        // convert CNAM to UTF-8 to fix
                       if (function_exists('mb_detect_encoding')) {
                                $this->out("Converting result to UTF-8");
                               $encoding = mb_detect_encoding($callerid, ['ASCII', 'CP1252', 'ISO-8859-1', 'UTF-8'], true);
                               $callerid = iconv($encoding . '//TRANSLIT', 'UTF-8', $callerid);
                        }

                        //send off
7

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.