Board Optimization - vCharset Converter 1.0.0 BETA 2

THE 3.5 BRANCHE IS CLOSED, A NEW VERSION FOR 3.6 HAD BEEN RELEASED.

To use this modification, please update to 3.6.x and use this.

By syrianmeds.net

Description: This script converts the database of vBulletin forums (from and to) any of the charsets that are supported by iconv. also it has the ability to convert the tables of the hacks you had installed them.

Caution: This is BETA PRODUCT, provided "As Is" without warranties, Test it using a copy of your vBulletin database not using your live vBulletin, Since this is a BETA PRODUCT use it for testing only.

Why this product?
My forums encoding is windows-1256, so I can't get full advantages of AJAX functions like the quick reply and the quick edit without some encoding problems.
I decided to convert my encoding and build this script. and I'll convert to utf-8 (Done).
This product was tested on a copy of real live vBulletin and it seems to be good.

Requirements:

• vBulletin 3.5.4+
• iconv php extension is installed on your system, or your PHP version is PHP5

Supported Encodings:
See http://www.gnu.org/software/libiconv/#introduction

Installation:

1. Upload files to their respective locations.
2. Import the file product product-charset_converter.xml (Choose Yes when Upgrade)
3. Refresh your admincp, go to vCharset Converter->Settings and set (the Source Charset Encoding & the Destination Charset Encoding).
4. Go to vBulletin Options -> Censorship Options -> Blank Character Stripper, empty its value and click save.

How to use?

1. Close your forum.
2. Backup your database.
3. Add a new language and call it English for example, its charset is your Destination Charset Encoding.
4. Set your options (profile.php?do=editoptions) to use it as a default language.
5. Change the charset of your old langauge into Destination Charset Encoding.
6. Go to vCharset Converter -> Manage tables. I'll leave this to you to explore it. But edit the userfield table and the custom fields to it.
7. Go to vCharset Converter -> vCharset Converter. if you see a message like:
   this means you need to install iconv.
8. You will see a drop-down menu. select the table you want to convert, set the number of items to convert per cycle (default 1000) and press submit.
9. You have to do this with each table in the drop-down menu.
10. Test your forum search for Non-English letters, if you don't get the expected results, read the "Search Problems".
11. Open your forum.
12. After finishing tell me what are your impression, idea and faced problems.

Search Problems
What is the problem? After converting you may get no (well-expected) results when doing search, I had faced this problem after converting to UTF-8.
What is my circumstances? Mysql 4.1.x, Full-Text search and I converted to utf-8.
What is the reason? Full-Text search has Restrictions that say: "all columns in a FULLTEXT index must use the same character set and collation.".
So if you use fulltext search with MySQL 4.1.x and convert to utf-8 perform these queries: (after converting post and thread tables)

ALTER TABLE post MODIFY pagetext mediumtext COLLATE utf8_unicode_ci
ALTER TABLE post MODIFY title VARCHAR(250) COLLATE utf8_unicode_ci
ALTER TABLE thread MODIFY title VARCHAR(250) COLLATE utf8_unicode_ci

Dont forget to add your table prefix if you have

- If you use vBulletin built-in search engine (with any MySQL version), then consider to Rebuild Search Index (Maintenance -> Update Counters -> Rebuild Search Index)
You may read this thread http://www.vbulletin.com/forum/showthread.php?t=191315

Email Problems
After the conversion, my users had reported that they couldn't read the notification emails. the emails are sent with utf-8 encoding.
Not all the users do know how to change the encoding of a page. you may learn them, this is the only solution.
Problem had been faced with Hotmail (confirmed by me) and Yahoo (unconfirmed).
No problem with Gmail (confirmed by me).

Known issues:

1. Users titles had gone away after converting. (Still exists in beta 2)

Versions

• 1.0.0 BETA 1 (initial release). (11. Jun 2006)
• 1.0.0 BETA 2 new features: (4. Aug 2006)
  1. Manage Tables: Now you can add or edit your tables, this will make converting of additional tables very easy.
  2. New built-in images to mark the tables:
     • Attachment 50952 Table is ready to be converted.
     • Attachment 50955 Table is uknown.
     • Attachment 50954 No need to convert this table.
     • Attachment 50953 Table Converted
  3. Now you can see all the tables in your database, with their statues and you have the ability to change this status to "no need" or you can add the fields and the primaryid field, or you can edit the information of any table before you go on converting it.

Forums had been Converted using this tool: (PM me to list your forum)

• http://www.basharialep.com/forum/ from windows-1256 into UTF-8

Support is given for who clicked "install" only.

Click to recieve updates.

Nominate it for Mod o.t. Month if you love it.

[Milad]

Post reserved.

[Barakat]

thanks Milad i will test it soon

[Panagioths]

Your script is extremely good. During the last 4 days I am trying to convert the database of my forum from greek ISO to utf8, I tried hundrend ways without the desired output and when I installed the vCharset all conversions made automagically and without any problems as fas as I can see now.

I have a question: I removed a part of data of my database by accident and now in product list (in admincp) vCharset is not shown anymore (in order to uninstall it since the convert is done). Also menu "vCharset" in admincp is not shown anymore. But I've got a table "charset_convert" in my database so I assume there are some parts of vCharset installtion still in forum.

Is there a sql command that I can run in order to remove any part of vCharset in my forum?

[Milad]

Thanks for your feedback

I assume you ran the script on a test board not on live one.

vCharset converter must remain in your database after finishing, it doesn't uninstal itself, you can uninstall it manually. yes you can uninstall the product, drop the table charset_convert and delete the product files.

Because you delete something in your database, you don't see the product, I think you deleted or epmtied product table.

I advise you to return to your backup, and test my script many times before you make decision to go ahead with it.

Remember I am still considering this product as a BETA product.

Thanks for your feedback again.

[Milad]

Also if you have some hacks, extensions or plugins that collect some text information in the database, you have to convert them too, this will be explained after the gold is released.

[Omranic]

thanks alot Milad
thats extremely good

Go on developing this
we need to see a strong stable version as soon as posible

[Milad]

Thanks

Please to keep yourselves updated press install.

[blacklude]

Originally Posted by Milad

Thanks for your feedback

I assume you ran the script on a test board not on live one.

vCharset converter must remain in your database after finishing, it doesn't uninstal itself, you can uninstall it manually. yes you can uninstall the product, drop the table charset_convert and delete the product files.

Because you delete something in your database, you don't see the product, I think you deleted or epmtied product table.

I advise you to return to your backup, and test my script many times before you make decision to go ahead with it.

Remember I am still considering this product as a BETA product.

Thanks for your feedback again.

is there any reason that the convertor must stay after finishing the convertion?
thanks!

[Milad]

Originally Posted by blacklude

is there any reason that the convertor must stay after finishing the convertion?

Never

But i want you to know that the product doesn't uninstall itself after the converting, you should uninstall it yourself.

Regards