katie hunter
11-19-2013, 10:20 PM
Hi,
I have this long ticket with vbulletin regarding this issue and up until now, we couldn't tackle what is the problem.
A short summary, i have installed a new vb 4.x as a dev demo and have made the database utf8 from the start, and changed the vb language charset to utf8 but certain characters do show up as diamond shape.
I have done the following and still have the same issue on a fresh vb installation, so what am i missing? This topic will be helpful for others as well.
First, there are your table and field encodings, these all have to match, and for your case preferably be in UTF-8.
Next, you need to make sure the blank character stripper is not filled in with any entries.
next, the vBulletin language, and character set for it should be in UTF-8.
Finally, the server needs to have mbstring, and iconv installed and configured into PHP, and then, there shouldn't be an issue on a fresh install.
All the best,
Zachery Woods
Support Team, vBulletin & Forum Runner
I've also read this topic http://www.vbulletin.com/forum/forum/vbulletin-legacy-versions-products/legacy-vbulletin-versions/vbulletin-3-6-questions-problems-and-troubleshooting/265581-language-charset-problems-things-to-check?t=259250
Last but not least anyone is experience with Milad (https://vborg.vbsupport.ru/member.php?u=82891) charset converter as well? https://vborg.vbsupport.ru/showthread.php?t=118308 I pmed him, we exchanged one pm, where he asked to see my phpmyadmin but he never replied back afterward. I wanted to show him my life forum phpmyadmin through teamview, i guess he didn't like this idea. Anyone has worked with Milad before?
In anyways, i do have the same issue on my live forum which the situation is more complicated, but lets focus on the dev demo first. I mean this is a fresh vb installation with utf8 database and charset language is set from the start to utf8 yet i see some dimaond shape.
When i set debug mode on, this was what i found (this is from my dev demo):
[root@web388 ~]# mysql --default-character-set=latin1 nihonomaru_vb4 -e 'SELECT title FROM post WHERE postid IN (06)'
+---------------------------------------------------------------------------------------------------------------------------+
| title |
+---------------------------------------------------------------------------------------------------------------------------+
| [阿久多のえ,森山大輔,西安,のら猫長� �] 百華繚爛 ～西安画集～ [Akuta Noe,Moriyama Daisuke,Nishi Iori] Hyakka Kenr |
+---------------------------------------------------------------------------------------------------------------------------+
[root@web388 ~]#
I was told that the output shows that my data was corrupted before it was stored in the database. So at this point, it is absolutely impossible to recover the original data. To solve this problem for future posts, you need to figure out why your data is being corrupted "on the way in" (before it reaches the database).
Anyone tried using vb 4.x using utf8 and had no issue? If so, what is wrong with my settings or what am i missing? This is a fresh vb installation.
Also anyone cares to make a utf8 converter, Xenforo has one that convert db from latin to utf8 when migrating from vb to xenforo. I never found a good converter for vbulletin community, hence it would be really great if we have one to support utf8 for vbulletin 4.x
I have this long ticket with vbulletin regarding this issue and up until now, we couldn't tackle what is the problem.
A short summary, i have installed a new vb 4.x as a dev demo and have made the database utf8 from the start, and changed the vb language charset to utf8 but certain characters do show up as diamond shape.
I have done the following and still have the same issue on a fresh vb installation, so what am i missing? This topic will be helpful for others as well.
First, there are your table and field encodings, these all have to match, and for your case preferably be in UTF-8.
Next, you need to make sure the blank character stripper is not filled in with any entries.
next, the vBulletin language, and character set for it should be in UTF-8.
Finally, the server needs to have mbstring, and iconv installed and configured into PHP, and then, there shouldn't be an issue on a fresh install.
All the best,
Zachery Woods
Support Team, vBulletin & Forum Runner
I've also read this topic http://www.vbulletin.com/forum/forum/vbulletin-legacy-versions-products/legacy-vbulletin-versions/vbulletin-3-6-questions-problems-and-troubleshooting/265581-language-charset-problems-things-to-check?t=259250
Last but not least anyone is experience with Milad (https://vborg.vbsupport.ru/member.php?u=82891) charset converter as well? https://vborg.vbsupport.ru/showthread.php?t=118308 I pmed him, we exchanged one pm, where he asked to see my phpmyadmin but he never replied back afterward. I wanted to show him my life forum phpmyadmin through teamview, i guess he didn't like this idea. Anyone has worked with Milad before?
In anyways, i do have the same issue on my live forum which the situation is more complicated, but lets focus on the dev demo first. I mean this is a fresh vb installation with utf8 database and charset language is set from the start to utf8 yet i see some dimaond shape.
When i set debug mode on, this was what i found (this is from my dev demo):
[root@web388 ~]# mysql --default-character-set=latin1 nihonomaru_vb4 -e 'SELECT title FROM post WHERE postid IN (06)'
+---------------------------------------------------------------------------------------------------------------------------+
| title |
+---------------------------------------------------------------------------------------------------------------------------+
| [阿久多のえ,森山大輔,西安,のら猫長� �] 百華繚爛 ～西安画集～ [Akuta Noe,Moriyama Daisuke,Nishi Iori] Hyakka Kenr |
+---------------------------------------------------------------------------------------------------------------------------+
[root@web388 ~]#
I was told that the output shows that my data was corrupted before it was stored in the database. So at this point, it is absolutely impossible to recover the original data. To solve this problem for future posts, you need to figure out why your data is being corrupted "on the way in" (before it reaches the database).
Anyone tried using vb 4.x using utf8 and had no issue? If so, what is wrong with my settings or what am i missing? This is a fresh vb installation.
Also anyone cares to make a utf8 converter, Xenforo has one that convert db from latin to utf8 when migrating from vb to xenforo. I never found a good converter for vbulletin community, hence it would be really great if we have one to support utf8 for vbulletin 4.x