Converting Special Chars from HTML to UTF-8 ascii standard?

[Grim77]

The release candidate is now online for v3.7 integration. You can get it from http://forums.addoninteractive.com/s...ead.php?t=3915

If you prefer to stick with 3.5/3.6, this is the code I've found to work but admittedly only tested on v3.7, though I don't think the way usernames are stored in the database has changed.

mb_convert_encoding almost does the trick, but not quite. I found the following code posted at php.net, and modified it so that HTML character codes aren't used for anything other than UTF-8 characters in the > 8 bit range, and it also allows for special characters (like '<') -- though some usernames with these special characters aren't permitted by the AddonChat chat software.

I've tested it using various English, Spanish and Arabic characters, and it seems to be working. Again though, if you're running v3.7 -- just download the release candidate and let me know if you run into any problems

PHP Code:

   /*
      UTF-8 to Numeric HTML Entity Conversion      
         Credit to: http://us3.php.net/manual/en/function.utf8-decode.php#75941
         ** Modified to only return HTML entities for characters out of 8 bit ASCII range.
         ** Modified to use htmlspecialchars_uni() function.
   */
   function utf8_to_html ($data)
   {
      return htmlspecialchars_uni(preg_replace("/([\\xC0-\\xF7]{1,1}[\\x80-\\xBF]+)/e", '_utf8_to_html("\\1")', $data));
   }

   function _utf8_to_html ($data)
   {
      $ret = 0;
      foreach((str_split(strrev(chr((ord($data{0}) % 252 % 248 % 240 % 224 % 192) + 128) . substr($data, 1)))) as $k => $v)
         $ret += (ord($v) % 128) * pow(64, $k);
         
      if($ret < 256)
         return chr($ret);
         
      return "&#$ret;";
   } 


To use, insert the above code at the end of your authentication script, then find the following code:

PHP Code:

if ($userinfo = $db->query_first('SELECT userid, usergroupid, membergroupids, password, salt FROM ' . TABLE_PREFIX . 'user WHERE username = "' . addslashes(htmlspecialchars_uni($username)) . '"')) 


and replace it with:

PHP Code:

if ($userinfo = $db->query_first('SELECT userid, usergroupid, membergroupids, password, salt FROM ' . TABLE_PREFIX . 'user WHERE username = "' . $db->escape_string(utf8_to_html($username)) . '"'))