Converting Special Chars from HTML to UTF-8 ascii standard?

[Kaelon]

Originally Posted by Grim77

Kaelon -- Just curious if we ever found a solution to this? I'm working on the 3.7 mod now, and would like to find a solution that doesn't require a non-standard php library.

Hi Grim77,

Unfortunately, no. Any of my users that have special characters in their usernames (such as accents, which are very common in Romance languages such as Spanish and French) have never been able to log in to our chat room properly. My recommendation would be to definitely allow special characters in the future.

Let me know how your progress goes with regards to this.

Thanks,
Juan

[Grim77]

Ok, Jaun -- We're working on the next update for v3.7 now. I'll look into this and see what we can do.

[Kaelon]

Originally Posted by Grim77

Ok, Jaun -- We're working on the next update for v3.7 now. I'll look into this and see what we can do.

Great, thanks, Chris!

[Grim77]

The release candidate is now online for v3.7 integration. You can get it from http://forums.addoninteractive.com/s...ead.php?t=3915

If you prefer to stick with 3.5/3.6, this is the code I've found to work but admittedly only tested on v3.7, though I don't think the way usernames are stored in the database has changed.

mb_convert_encoding almost does the trick, but not quite. I found the following code posted at php.net, and modified it so that HTML character codes aren't used for anything other than UTF-8 characters in the > 8 bit range, and it also allows for special characters (like '<') -- though some usernames with these special characters aren't permitted by the AddonChat chat software.

I've tested it using various English, Spanish and Arabic characters, and it seems to be working. Again though, if you're running v3.7 -- just download the release candidate and let me know if you run into any problems

PHP Code:

   /*
      UTF-8 to Numeric HTML Entity Conversion      
         Credit to: http://us3.php.net/manual/en/function.utf8-decode.php#75941
         ** Modified to only return HTML entities for characters out of 8 bit ASCII range.
         ** Modified to use htmlspecialchars_uni() function.
   */
   function utf8_to_html ($data)
   {
      return htmlspecialchars_uni(preg_replace("/([\\xC0-\\xF7]{1,1}[\\x80-\\xBF]+)/e", '_utf8_to_html("\\1")', $data));
   }

   function _utf8_to_html ($data)
   {
      $ret = 0;
      foreach((str_split(strrev(chr((ord($data{0}) % 252 % 248 % 240 % 224 % 192) + 128) . substr($data, 1)))) as $k => $v)
         $ret += (ord($v) % 128) * pow(64, $k);
         
      if($ret < 256)
         return chr($ret);
         
      return "&#$ret;";
   } 


To use, insert the above code at the end of your authentication script, then find the following code:

PHP Code:

if ($userinfo = $db->query_first('SELECT userid, usergroupid, membergroupids, password, salt FROM ' . TABLE_PREFIX . 'user WHERE username = "' . addslashes(htmlspecialchars_uni($username)) . '"')) 


and replace it with:

PHP Code:

if ($userinfo = $db->query_first('SELECT userid, usergroupid, membergroupids, password, salt FROM ' . TABLE_PREFIX . 'user WHERE username = "' . $db->escape_string(utf8_to_html($username)) . '"'))