vB Global Translator - Multiply your indexed pages & put search traffic on autopilot

[Dave Hybrid]

**Text Removed**

[Dave Hybrid]

v1.x Upgrade instructions:

Delete your old translate database
Delete translate.php, translateflags.php and overloadcache.txt
Delete both translate plugins
Delete code added to header, footer or navbar template
Delete /flags/ folder

Follow the install instructions above

This version requires a fresh install.

Features coming in next version:

* Meta description and meta keywords will be translated in the output source code, better for SEO and search engine indexing

[dothanhtrung]

Can you add language vietnamese ?? :D

[Sweeks]

Looks great! Sending a donation over to NLP-er in a moment :) Thanks for this Dave! Blimey it is much more faster once cached than the old script, great work!

[deadlySniper]

This is awesome! Thanks!

[Dave Hybrid]

Looks great! Sending a donation over to NLP-er in a moment :) Thanks for this Dave! Blimey it is much more faster once cached than the old script, great work!

Thanks, DB speed credits to NLP-er, I'm sure he will be happy with the donation.

[Sweeks]

It honestly is a vast improvement, it may take a while on the first load but once its cached the page loads in a matter of seconds :up:

There is a problem or I may be getting this wrong, the Chinese simplified shows it all like this:

电脑，游戏和技术 （ 1查看）
青少年交谈，并获得咨询有关的游戏，电脑，技术和更多的对我们的青少年论坛。

Is that right?

[Dave Hybrid]

It honestly is a vast improvement, it may take a while on the first load but once its cached the page loads in a matter of seconds :up:

There is a problem or I may be getting this wrong, the Chinese simplified shows it all like this:

电脑，游戏和技术 （ 1查看）
青少年交谈，并获得咨询有关的游戏，电脑，技术和更多的对我们的青少年论坛。

Is that right?

Caching will always take some time, there is work to be done, the page is being translated on the fly but this version makes caching faster to, as much as possible.

What is wrong with Chinese? Looks like Chinese to me he he.

[Sweeks]

I wasnt sure if it was chinese lol just looks like a load of odd symbols with numbers on them to me :D

It is much faster once the pages have been cached. Just tested it and it only takes a few seconds to translate not 30 seconds once cached etc :)

[valdet]

Having so many new pages (translated pages) in your website might raise red flags for Google.

Do you think this is legit with Google TOS ?

Nevertheless, this is a brilliant idea.
Thanks for your time.

[Dave Hybrid]

I wasnt sure if it was chinese lol just looks like a load of odd symbols with numbers on them to me :D

It is much faster once the pages have been cached. Just tested it and it only takes a few seconds to translate not 30 seconds once cached etc :)

Yep, it's pretty much perfect speed wise on my forums.

I just check your forum, Chinese and all others for that matter are working fine.

Thanks.

[GoTTi]

shouldnt you have a english flag on there? there is no way to goto english if ur in another language. if someone clicks the language by accident they wouldnt know how to go back to english.

[Sweeks]

shouldnt you have a english flag on there? there is no way to goto english if ur in another language. if someone clicks the language by accident they wouldnt know how to go back to english.

It is bad for SEO so it is best removed, I am sure they could easily click your logo or something to get back :)

[Dave Hybrid]

Having so many new pages (translated pages) in your website might raise red flags for Google.

Do you think this is legit with Google TOS ?

Nevertheless, this is a brilliant idea.
Thanks for your time.

It is probably a grey area, it's original content and was once created by members so is not technically automated or junk content.

There have been similar scripts before I did this for vB, the other scripts has issues but did work. A few friends and I have been using these other scripts that are out for years and never had anything bad happen.

Although this idea may be new to you and others, it's been around a long time, other coders tried using the actual Google Translate website, this was slow and buggy but did work. Goggles new Translate API service is why I thought I'd try my luck.

[Dave Hybrid]

It is bad for SEO so it is best removed, I am sure they could easily click your logo or something to get back :)

If you click any url is goes back to your base language anyway. Most people only view one page, so it really doesn't matter.

[Sweeks]

Dave I have a question, is this purely for the SE's or do you think foreign signed in members will find it useful too? I am considering allowing only guests to view the flags, what do you think? :)

[Dave Hybrid]

I only show the flags to guests, this is machine translation, so, the bots will love it and send you visits, but, those visits will read and understand what they can (it's pretty good but not perfect) but are unlikely to stick around for long, unless they can speak you base language. As I have said before, if you want to add different language forums to your site do just that, make the forums, get a foreign moderator and install some language packs. This is machine translation for traffic and ad revenue pretty much. I make tons extra $$ with it and it didn't take any extra work to write the 1,000's of pages of content, that was it's goal when I designed it. Nothing else. ;P

I hope this helps.

[GoTTi]

i cant get this to work. ive installed it, done the DB stuff. when i click a flag nothing happens, its just loading and does nothing. no data is being written to the mysql DB.

[Dave Hybrid]

i cant get this to work. ive installed it, done the DB stuff. when i click a flag nothing happens, its just loading and does nothing. no data is being written to the mysql DB.

Do you have CURL enabled on your site?

Could also ask you host to turn on php errors in the browser so you get an error message.

[GoTTi]

Do you have CURL enabled on your site?

Could also ask you host to turn on php errors in the browser so you get an error message.

cURL support enabled
cURL Information libcurl/7.19.3 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5

[GoTTi]

how long does it take to start re-writting all the data? i see now see something in the wt_cache_short table but nothing else is going on.

[Dave Hybrid]

As soon as you click a flag it should start adding data. If it's adding data it's working. 1st time translations are not instant, the words have to be sent to google and then translated, then returned and put in the DB, 2nd page load is normal speed. It will take a few weeks to cache your entire site.

What is your URL?

[GoTTi]

What is your URL?


please check ur PM inbox.

[GoTTi]

As soon as you click a flag it should start adding data. If it's adding data it's working. 1st time translations are not instant, the words have to be sent to google and then translated, then returned and put in the DB, 2nd page load is normal speed. It will take a few weeks to cache your entire site.



so this doesnt automatically start indexing the new language? the flags have to be clicked on by someone when they are in a thread in order to get the new language indexing to go?

[Dave Hybrid]

so this doesnt automatically start indexing the new language? the flags have to be clicked on by someone when they are in a thread in order to get the new language indexing to go?

The search bots follow the flags and activate the translation, you do not need to do a thing aside from test it.

[dacho]

Some questions :

If you have a English board and someone write a Post/Reply in Chinese or Hebrew - what you do with that ?
for example i have a Hebrew forums so what the members can do with other language, if someone from China write to them?
what going do happen do the data if i :
Data Usage - 1.81 GB
Index Usage - 963.42 MB
Members - 117,205
Post - 266,431
Reply - 1,479,829

[Dave Hybrid]

Some question :

If you have a English board and someone write a Post/Reply in Chinese or Hebrew - what you do with that ?
for example i have a Hebrew forums so what the members can do with other language ?
what going do happen do the data if i :
Data Usage - 1.81 GB
Index Usage - 963.42 MB
Post - 266,431
Reply - 1,479,829

This is a translator, your members can still post in the same languages that you currently allow. No different. It doesn't change how vB works.

[GoTTi]

here is a error i am getting in the phmyadmin. when i goto reload the db to see if anything is being added to the 3 tables i see this error right now:

Error
SQL query:
SHOW CHARACTER SET;
MySQL said:
#5 - Out of memory (Needed 1677360 bytes)

[Dave Hybrid]

here is a error i am getting in the phmyadmin. when i goto reload the db to see if anything is being added to the 3 tables i see this error right now:

Error
SQL query:
SHOW CHARACTER SET;
MySQL said:
#5 - Out of memory (Needed 1677360 bytes)

It's particular to your host so send their support a message explaining that you are getting that error when running a cron. Possibly they need to up your php memory limit or something.

[Legendery.]

thank you so much

[GoTTi]

testing this on a different site on a different server i am getting this error when i click the flags:

Fatal error: Call to undefined function mysql_set_charset() in /home/SITE/public_html/forum/translate.php on line 35

[Dave Hybrid]

thank you so much

You're welcome! :D

[jaryx]

Hi, Dave. Are you remember me?

Great job! Now it's works on my site without change hosting.
1st time works very slow, 2nd, 3rd... works ok. Server is cheap ;)

I have only one problem but my admin can fix it.
Warning: set_time_limit() has been disabled for security reasons in [path]/translate.php on line 28

THX again.

[GoTTi]

testing this on a different site on a different server i am getting this error when i click the flags:

Fatal error: Call to undefined function mysql_set_charset() in /home/SITE/public_html/forum/translate.php on line 35

what would be causing this error on a different site and server?

[Dave Hybrid]

Hi, Dave. Are you remember me?

Great job! Now it's works on my site without change hosting.
1st time works very slow, 2nd, 3rd... works ok. Server is cheap ;)

I have only one problem but my admin can fix it.
Warning: set_time_limit() has been disabled for security reasons in [path]/translate.php on line 28

THX again.

Your host is being over-protective, imo, get a new host. ;P

[Fynnon]

interesting...but people will be tempted to post in their native language
i wonder how much the database will grow...

its not very easy to install this and, i think ill wait untill you make an easier install but i`ll keep my eye on this :-)

great work and great idea so far !

[Dave Hybrid]

interesting...but people will be tempted to post in their native language
i wonder how much the database will grow...

its not very easy to install this and, i think ill wait untill you make an easier install but i`ll keep my eye on this :-)

great work and great idea so far !

As soon as they hit reply the script throws them back to the vbulletin database and your base language, so i doubt it. They are also smart enough to know it's a machine translation.

I don't see why people worry so much about such trivial things, users don't do it on my sites and if they did I'd just delete the post.

Your vB databases will not grow anymore than normal as this does not interfere with vB, the translate database only holds sentences so will grow no larger than a normal vbulletin DB, if not much less!

Actually the install looks more complex than it is, give it a try, you can always revert your changes.

I hope this answers your concerns. :)

[Gemma]

Nice idea but I couldn't get it to work. When I click on a flag I get a white screen :(

[Dave Hybrid]

Nice idea but I couldn't get it to work. When I click on a flag I get a white screen :(

Is CURL enabled on your server?

[Gemma]

Yes

[Dave Hybrid]

Ask your host to turn on php error message in the browser, it should output an error.

99% of install problems are because of your server config or the install wasn't followed correct, check everything thoroughly.

[Bounce]

WHOOOO!!! I thought this was a goner :)

THANK YOU DAVE!!!! :up::up::up:

[Dave Hybrid]

WHOOOO!!! I thought this was a goner :)

THANK YOU DAVE!!!! :up::up::up:

You're welcome!

Make sure you install and test it because your current setup isnt translating a thing, not sure if you are aware.

[Bounce]

You're welcome!

Make sure you install and test it because your current setup isnt translating a thing, not sure if you are aware.

Just installed just now, yes when I click on a flag its all still in english? How is that?

[Dave Hybrid]

Just installed just now, yes when I click on a flag its all still in english? How is that?

No idea tbh, i can tell you've skipped parts of the install, so i would advise following the upgrade and new install instructions to the T.

[Bounce]

Ah never deleted the old plugins and overload text, sorry was in a hurry as was all excited LOL :)

[Dave Hybrid]

Doesn't feel right mate, see my previous post, you need to fresh install, new flags files, complete new database, everything, delete everything, start fresh and do not skip parts and it will work much better.

[Dave Hybrid]

LOL, ok. :)

[jaryx]

A small note to the flags.
English language (EN) is... english from England, not the U.S.
When Sir Thomas Wyatt wrote his poems, then the Americans running the prairie for the bizons. ;)

ps.
My english is bad english, but english. ;)

[Dave Hybrid]

Ok, well, en code is the only one Google Translate offers. I used US flag as there are more US Internet users than UK. Trying to cater for the larger audience lol. :)

[Bounce]

Doesn't feel right mate, see my previous post, you need to fresh install, new flags files, complete new database, everything, delete everything, start fresh and do not skip parts and it will work much better.

I still does not no, right off to delete it all and start again closely following the instructions, I'm a bit confused (not hard LOL) by this...

Now change $originalEncoding variable to match your forum base encoding. The default is english (iso-8859-2).

Dumb that I am I presume I don't change that as my forums english?

BE right back LOL :)

[puertoblack2003]

thank you :up: installed so far no issue to report.

[Dave Hybrid]

thank you :up: installed so far no issue to report.

Looks great on your site and very fast, nice one!

[Bounce]

Can't I drop the tables in the DB or it needs to be brand new ?

[Dave Hybrid]

Brand new dude. Everything is new. All files, all code, database format and tables. Fresh install it.

[GoTTi]

testing this on a different site on a different server i am getting this error when i click the flags:

Fatal error: Call to undefined function mysql_set_charset() in /home/SITE/public_html/forum/translate.php on line 35

[puertoblack2003]

Looks great on your site and very fast, nice one!

thanks


on this instruction

You need to comment out the flag for your base language otherwise you can cause duplicate URLs for those pages and this can be bad for SEO.

All the flags are on separate lines in alpha order, find your language and add <!-- to the start and --> to the end of the line.

eg; <!-- language flag code -->

Upload both files to your forum root, if your forums are in a /forums/ folder, subdomain or similar upload the files there. Upload the flags folder to your domain root, thats the domain root.




I want to make sure that i follow you so it should be like this


<a href="<?php $xbit = (count($_GET)>1) ? "&" : "?"; echo (strstr($_SERVER["REQUEST_URI"],'hl='.@$_GET['hl'])) ? str_replace($xbit.'hl='.@$_GET['hl'], '', $_SERVER["REQUEST_URI"]) . $xbit . 'hl=en' : str_replace($xbit.'hl='.@$_GET['hl'], '', $_SERVER["REQUEST_URI"]) . $xbit . "hl=en"; ?>"><img src="/flags/United States.gif" alt="English" border="0" /></a>
<!-- to the start and -->

[Dave Hybrid]

<!-- <a href="<?php $xbit = (count($_GET)>1) ? "&" : "?"; echo (strstr($_SERVER["REQUEST_URI"],'hl='.@$_GET['hl'])) ? str_replace($xbit.'hl='.@$_GET['hl'], '', $_SERVER["REQUEST_URI"]) . $xbit . 'hl=en' : str_replace($xbit.'hl='.@$_GET['hl'], '', $_SERVER["REQUEST_URI"]) . $xbit . "hl=en"; ?>"><img src="/flags/United States.gif" alt="English" border="0" /></a> -->

There.

It will stop that flag showing, so it should disappear.

[GoTTi]

wt_cache_medium
wt_cache_short

these 2 tables on the db have lines in it now. the short has 20 and the medium has 3. but the text still isnt converting over to other languages. this is on the site i sent u on the PM.

the other error i am having is on another site different server

[puertoblack2003]

<!-- <a href="<?php $xbit = (count($_GET)>1) ? "&" : "?"; echo (strstr($_SERVER["REQUEST_URI"],'hl='.@$_GET['hl'])) ? str_replace($xbit.'hl='.@$_GET['hl'], '', $_SERVER["REQUEST_URI"]) . $xbit . 'hl=en' : str_replace($xbit.'hl='.@$_GET['hl'], '', $_SERVER["REQUEST_URI"]) . $xbit . "hl=en"; ?>"><img src="/flags/United States.gif" alt="English" border="0" /></a> -->

There.

It will stop that flag showing, so it should disappear.


gotcha thanks :up:

[NLP-er]

testing this on a different site on a different server i am getting this error when i click the flags:

Fatal error: Call to undefined function mysql_set_charset() in /home/SITE/public_html/forum/translate.php on line 35

Wrong version of PHP. From manual about mysql_set_charset() This function requires MySQL 5.0.7 or later

[Bounce]

arghhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh lol :D

I've installed this (I THINK) as per the instructions, even created new database and sql user for it, on clicking a flag I get

Warning: mysql_fetch_array(): supplied argument is not a valid MySQL result resource in [path]/translate.php on line 148

then the database error,

Invalid SQL:

UPDATE session
SET lastactivity = 1246230400, location = '/forum/?hl=ja', inforum = 0, inthread = 0, incalendar = 0, badlocation = 0, useragent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11 (.NET CLR 3.5.30729)'
WHERE sessionhash = '726fe064a7efc501fc5f743000592a85';

MySQL Error : MySQL server has gone away

Must have fooked up the install somewhere but fooked if I can figure out where LOL

No data is being "written" into the database either, LOL the joys

[NLP-er]

wt_cache_medium
wt_cache_short

these 2 tables on the db have lines in it now. the short has 20 and the medium has 3. but the text still isnt converting over to other languages. this is on the site i sent u on the PM.

the other error i am having is on another site different server

For the first translation of some page to some language you have to be very patient. A lot of things to translate by google... And It is not possible to skip this time - it have to be translated. But after that having this page translated again to this language if very fast, because google isn’t asked for that anymore - just taken from DB.

Give it a chance and just wait till page is translated.

[NLP-er]

arghhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh lol :D

I've installed this (I THINK) as per the instructions, even created new database and sql user for it, on clicking a flag I get

Warning: mysql_fetch_array(): supplied argument is not a valid MySQL result resource in [path]/translate.php on line 148

then the database error,



Must have fooked up the install somewhere but fooked if I can figure out where LOL

No data is being "written" into the database either, LOL the joys

On line 148 there is no mysql_fetch_array() call - probably you have old version. Please update it :)

[puertoblack2003]

arghhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh lol :D

I've installed this (I THINK) as per the instructions, even created new database and sql user for it, on clicking a flag I get

Warning: mysql_fetch_array(): supplied argument is not a valid MySQL result resource in [path]/translate.php on line 148

then the database error,



Must have fooked up the install somewhere but fooked if I can figure out where LOL

No data is being "written" into the database either, LOL the joys



your issue is right here brotha

MySQL Error : MySQL server has gone away

timing out issue ...i clocked my to 60 second before it translate then after that it's quick

[Dave Hybrid]

Some large pages can take a few minutes, test it on pages will low text. It's not a big deal as the search bots will hit large pages a few times to translate them in the background. Next time the load from cache and are fast.

[puertoblack2003]

One more question i have a friend that is from Germany.The only thing he noticed that for each page you have to click on the flag.Is that how it works?

[Dave Hybrid]

One more question i have a friend that is from Germany.The only thing he noticed that for each page you have to click on the flag.Is that how it works?

Yep, that is correct.

[Bounce]

On line 148 there is no mysql_fetch_array() call - probably you have old version. Please update it :)

thanks, think I did doh lol

weird now i'm getting the error..

Warning: mysql_connect() [function.mysql-connect]: Unknown MySQL server host 'dbname_trans' (1) in [path]/translate.php on line 33

Warning: Cannot modify header information - headers already sent by (output started at [path]/includes/class_core.php:3277) in [path]/includes/functions.php(5721) : eval()'d code on line 5

but its translating the pages ....

http://www.hibeesbounce.com/forum/showthread.php?t=86933&hl=ja

Is it just a case of waiting or have I got something else wrong lol

yea I did LOL

$dbusername = "bdname";
$dbpassword = "me";
$mysqlserver = "localhost"; > had this as the db name pmsl :)
$dbname = "dbname";

THAT link now translates :):)

Still get myqsl server has gone away on forumhome thou, patience needed?

[GoTTi]

Wrong version of PHP. From manual about mysql_set_charset() This function requires MySQL 5.0.7 or later

PHP Version 5.1.4


??

[Dave Hybrid]

thanks, think I did doh lol

weird now i'm getting the error..



but its translating the pages ....

http://www.hibeesbounce.com/forum/showthread.php?t=86933&hl=ja

Is it just a case of waiting or have I got something else wrong lol

Double check your MySQL data you entered is correct ;)

[GoTTi]

For the first translation of some page to some language you have to be very patient. A lot of things to translate by google... And It is not possible to skip this time - it have to be translated. But after that having this page translated again to this language if very fast, because google isn?t asked for that anymore - just taken from DB.

Give it a chance and just wait till page is translated.

how long are we to wait for the site to be translated? a few hours went by and nothing happend so far.

[NLP-er]

PHP Version 5.1.4


??

http://us3.php.net/manual/en/function.mysql-set-charset.php

I know that first question can look silly, but have to be sure - no other ideas :p
- Are you sure you check PHP version on server where script is intalled?
- Are you sure php was properly installed? In PHP 5+ MySQL is not enabled by default (In order to have these functions available, you must compile PHP with MySQL support - http://us3.php.net/manual/en/mysql.requirements.php) If not here you have PHP installation instructions: http://us3.php.net/manual/en/mysql.installation.php

[NLP-er]

how long are we to wait for the site to be translated? a few hours went by and nothing happend so far.

Definitively not that long ;) 30 seconds, maybe 1minute if there is lot of text or lot of short words (i.e. tag cloud). On my forum I sometimes have such behavior that it translates, and translates, and... And after reload it is already translated. I assume that those are some browser sever issues because of long download time, and after reload it is already cached and goes fast.

So if reload gives you also nothing and you see that number of rows in cache tables is same - definitively it's something wrong. Need more info to be able to say what... some error logs?

[GoTTi]

nlp - only 2 tables are doing "something" on 1 of the sites. the medium and short tables are writing something in there. the cache isnt writing anything....what could be causing the cache not to write?

[NLP-er]

nlp - only 2 tables are doing "something" on 1 of the sites. the medium and short tables are writing something in there. the cache isnt writing anything....what could be causing the cache not to write?

those tables are to separate short, medium and long translations. Most forums have lot of short translations, then some medium and sometimes long. So it is ok.

I just wrote my own spider which goes to every page generated by vbSEO sitemap generator (must done little change in their code to automatically add all translations to sitemap) and also add rest languages available in google (over 40).

Now I have in sitemap 56 000 addresses. Spider will work for long time, and I will give you all some image about DB cache size.

Right now it just started only 100 pages done (not counting those which was clicked by users or other spiders), but it will already give you some proportions (keep in mind that I have over 40 languages not 28).

wt_cache_short 70 000 entries
wt_cache_medium 11 000 entries
wt_cache 5 000 entries

So it is ok that your wt_cache don't have to mutch data.

Also you can heck does wt_cache was appropriately created during installation. You need MySQL 5.0.3 or higher to be able to have it works, because I used varchar(65000) and in previous mysql versions varchar could have max size 255. If you have lower version - you can still use it. Just change VARCHAR(65000) to TEXT for column oryginalText drop table and create again from changed code.

When I finish my spidering ;) I will made some automatic performance tests. Also change some other solutions (different indexes sizes, other order of filtering in SQL WHERE clause...). So it is possible that it will be even faster :)

Also good news is that when my spider makes translations all the time it doesn't affects speed of normal site generation (all lags are because of google translation server). So you can be calm - even if you wait long time for translation other users don’t feel it at all being on forum :)

[ShawneyJ]

You need to comment out the flag for your base language otherwise you can cause duplicate URLs for those pages and this can be bad for SEO.

All the flags are on separate lines in alpha order, find your language and add <!-- to the start and --> to the end of the line.

eg; <!-- language flag code -->


i dont need to worrie about this part if im using english correct?
thx.

if i do have to worry about this step, can i ask the code i have to put around this line plz???

<a rel="novbseo"href="<?php echo (strstr($_SERVER["VBSEO_URI"],'?hl='.@$_GET['hl'])) ? str_replace('?hl='.@$_GET['hl'], '', $_SERVER["VBSEO_URI"]) . '?hl=en' : str_replace('?hl='.@$_GET['hl'], '', $_SERVER["VBSEO_URI"]) . "?hl=en"; ?>"><img src="/flags/United States.gif" alt="English" border="0" /></a>

if im completely not making sense please explain thanks m8.

[Sweeks]

<!-- <a rel="novbseo"href="<?php echo (strstr($_SERVER["VBSEO_URI"],'?hl='.@$_GET['hl'])) ? str_replace('?hl='.@$_GET['hl'], '', $_SERVER["VBSEO_URI"]) . '?hl=en' : str_replace('?hl='.@$_GET['hl'], '', $_SERVER["VBSEO_URI"]) . "?hl=en"; ?>"><img src="/flags/United States.gif" alt="English" border="0" /></a> -->

[Dave Hybrid]

those tables are to separate short, medium and long translations. Most forums have lot of short translations, then some medium and sometimes long. So it is ok.

I just wrote my own spider which goes to every page generated by vbSEO sitemap generator (must done little change in their code to automatically add all translations to sitemap) and also add rest languages available in google (over 40).

Now I have in sitemap 56 000 addresses. Spider will work for long time, and I will give you all some image about DB cache size.

Right now it just started only 100 pages done (not counting those which was clicked by users or other spiders), but it will already give you some proportions (keep in mind that I have over 40 languages not 28).

wt_cache_short 70 000 entries
wt_cache_medium 11 000 entries
wt_cache 5 000 entries

So it is ok that your wt_cache don't have to mutch data.

Also you can heck does wt_cache was appropriately created during installation. You need MySQL 5.0.3 or higher to be able to have it works, because I used varchar(65000) and in previous mysql versions varchar could have max size 255. If you have lower version - you can still use it. Just change VARCHAR(65000) to TEXT for column oryginalText drop table and create again from changed code.

When I finish my spidering ;) I will made some automatic performance tests. Also change some other solutions (different indexes sizes, other order of filtering in SQL WHERE clause...). So it is possible that it will be even faster :)

Also good news is that when my spider makes translations all the time it doesn't affects speed of normal site generation (all lags are because of google translation server). So you can be calm - even if you wait long time for translation other users don’t feel it at all being on forum :)

I just use Googlebot lol, I have 500,000 total entrys after 24hrs. :eek:

Also no slow down at all, pages still fast as usual. DB size is 100MB so far.

[Calash]

Well I am still getting the database errors I quoted from the previous version, however the database is being populated and I actually finally got a page to show up in another language.

I will let this run for a bit now and see if I get more translated pages. Great job with this mod :)

[Dave Hybrid]

Well I am still getting the database errors I quoted from the previous version, however the database is being populated and I actually finally got a page to show up in another language.

I will let this run for a bit now and see if I get more translated pages. Great job with this mod :)

I checked your site, seems to be working fine.

The DB errors will happen, you can reduce them but upping the time out limit of MySQL.

Some pages take some time to translate due to the amount of text on the page and MySQL get's impatient, they will stop eventually as translations load from cache and do not impact the working of your site.

[jaryx]

Is it possible to do that?:

When the page will be translated for the first time, user will see the message "Please wait ... Loading ..."

[Dave Hybrid]

Is it possible to do that?:

When the page will be translated for the first time, user will see the message "Please wait ... Loading ..."

I think you're missing the point, any current users on your site are not interested in other languages so they aint gonna click anything often.

The users that come from google to your translated pages will see the page load instant, because for the page to be indexed in the first place the search bot would of already translated it and cached it for you.

So it really isn't necessary imo.

[Calash]

I checked your site, seems to be working fine.

The DB errors will happen, you can reduce them but upping the time out limit of MySQL.

Some pages take some time to translate due to the amount of text on the page and MySQL get's impatient, they will stop eventually as translations load from cache and do not impact the working of your site.

I think it is a limit of the hosting unfortunately. I can check to see if the MySQL limits can be altered but I may not have much of a choice.

What would be the best way to limit the script to only vB pages? Since I have so many different mods that it will not work with I think it would be best to wrap the flag code in conditionals. Has anybody done this yet?

[ShawneyJ]

<!-- <a rel="novbseo"href="<?php echo (strstr($_SERVER["VBSEO_URI"],'?hl='.@$_GET['hl'])) ? str_replace('?hl='.@$_GET['hl'], '', $_SERVER["VBSEO_URI"]) . '?hl=en' : str_replace('?hl='.@$_GET['hl'], '', $_SERVER["VBSEO_URI"]) . "?hl=en"; ?>"><img src="/flags/United States.gif" alt="English" border="0" /></a> -->

hey mate you talking to me or what??? speak! if so thx.

[Dave Hybrid]

I think it is a limit of the hosting unfortunately. I can check to see if the MySQL limits can be altered but I may not have much of a choice.

What would be the best way to limit the script to only vB pages? Since I have so many different mods that it will not work with I think it would be best to wrap the flag code in conditionals. Has anybody done this yet?

Yes, wrap them in conditionals so the flags only show on certain pages, i only put them on my showthread pages and to guests because those are the one that will pull the traffic and use the flags.

----------------------------------------

<if condition="is_member_of($bbuserinfo,1)">
<if condition="(THIS_SCRIPT == 'showthread')">
-code-
</if>
</if>

[Dave Hybrid]

hey mate you talking to me or what??? speak! if so thx.

I believe they were, basically you need to remove the flag your base language is in.

So find the line for english and put

<!--

at the start. and

-->

at the end.

Or just delete the line altogether, same thing happens, the flag will not display.

[Calash]

Yes, wrap them in conditionals so the flags only show on certain pages, i only put them on my showthread pages and to guests because those are the one that will pull the traffic and use the flags.

----------------------------------------

<if condition="is_member_of($bbuserinfo,1)">
<if condition="(THIS_SCRIPT == 'showthread')">
-code-
</if>
</if>

That works perfectly. The translation also seems to work better on the showthread in general, probably because there are less mods than on Forumhome.

Still getting some database errors but once on the first access. After that everything comes up.

Thanks again :)

[T2x]

Hey is there a way to block certain parts of the page from being translated?

This sort of functionality would be VERY nice.

For example

Text that gets translated here
<!--notranslatestart-->
This text doesnt get translated
<!--notranslateend-->
text that gets translated here

Do you think it can be done?

[Dave Hybrid]

Not yet, we may do it in the future. Working on some other features right now but it's been noted, thanks.

[Bounce]

Yes, wrap them in conditionals so the flags only show on certain pages, i only put them on my showthread pages and to guests because those are the one that will pull the traffic and use the flags.

----------------------------------------

<if condition="is_member_of($bbuserinfo,1)">
<if condition="(THIS_SCRIPT == 'showthread')">
-code-
</if>
</if>

:up: that works better for me as well :D

[merkaz]

In the database i see the " originaltext "
But the " translated " is empty ...

https://vborg.vbsupport.ru/external/2009/06/2.jpg

[Dave Hybrid]

In the database i see the " originaltext "
But the " translated " is empty ...

http://www.alhnuf.com/up/pics-gif/upload/uploads/images/a-fb5177e961.jpg

I cant check anything without a url?

[T2x]

Hey dave, very nice plugin.

I figured it out for you :D

<!--notranslatestart-->
Do not translate this text
<!--notranslateend-->

That is the syntax.

Change the plugin code to the following:

// Enable UTF-8 characters
if(isset($_GET['hl'])) {
require_once("translate.php");
$output = str_replace('lang="en"', 'lang="'.$_GET['hl'].'"', $output);
header ('Content-type: text/html; charset=utf-8');
// Keep remainders of <style and <script tags!
$time = time(); // Unique Tag Identifier!
preg_match_all('|<script[^>]*>(.*?)</script>|si', $output,$scripttags, PREG_SET_ORDER);
for($i=0;$i<count($scripttags);$i++)
{
$output = str_replace($scripttags[$i][0], "<a name=\"$i\"></a>",$output);
}
//preg_match_all("<style[^>]*>.*</style>",$buffer,$styletags);
preg_match_all('|<style[^>]*>(.*?)</style>|si',$output,$styletags);
for($i=0;$i<count($styletags);$i++)
{
$output = str_replace($styletags[1][$i], "<a name=\"s$i\"></a>",$output);
}
preg_match_all('|<!--notranslatestart-->(.*?)<!--notranslateend-->|si', $output,$notranslatetags, PREG_SET_ORDER);
for($i=0;$i<count($notranslatetags);$i++)
{
$output = str_replace($notranslatetags[$i][0], "<a name=\"t$i\"></a>",$output);
}
$output=callback($output);
// Place back remainders of <style and <script tags!
for($i=0;$i<count($styletags);$i++)
{
$output = str_replace("<a name=\"t$i\"></a>", $notranslatetags[$i][0], $output);
}
for($i=0;$i<count($styletags);$i++)
{
$output = str_replace("<a name=\"s$i\"></a>", $styletags[1][$i], $output);
}
for($i=0;$i<count($scripttags);$i++)
{
$output = str_replace("<a name=\"$i\"></a>",$scripttags[$i][0] ,$output);
}
//print_r($scripttags);
//die("c:".count($scripttags).":".count($styletags));
}

Also you are going to make google angry.

You need to add a @curl_setopt($ch, CURLOPT_REFERER,"http://$_SERVER[SERVER_NAME]$_SERVER[VBSEO_URI]"); (this is vbseo only code)
according to google.

and adding an api key to the request doesn't hurt.

http://code.google.com/apis/ajaxsearch/signup.html

at http://code.google.com/apis/ajaxlanguage/documentation

it says:

Flash and other Non-Javascript Environments

For Flash developers, and those developers that have a need to access the AJAX Language API from other Non-JavaScript environments, the API exposes a simple RESTful interface. In all cases, the method supported is GET, and the response format is a JSON encoded result with embedded status codes. For google.language.translate, the POST method is available. Applications that use this interface must abide by all existing terms of use. An area to pay special attention to relates to correctly identifying yourself in your requests. Applications MUST always include a valid and accurate http referer header in their requests. In addition, we ask, but not require, that each request contains a valid API Key. By providing a key, your application provides us with a secondary identification mechanism that is useful should we need to contact you in order to correct any problems.

so basically you just change the request to look like this

@curl_setopt($ch, CURLOPT_URL, "http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&key=KEYHERE&langpair=".urlencode($fl.'|'.$tl)."&q=".urlencode($text));

[tpearl5]

I'm trying to figure out a way I could use this with vbSEO. Ideally the url's should be in the format site.com/xx/thread_name/ where xx is the language abbreviation. Anyone have any ideas?

Also, is there any reason that links on the translated page shouldn't be to the translated version of corresponding page? Otherwise the user would have to keep clicking their flag for each new page.

[T2x]

I'm trying to figure out a way I could use this with vbSEO. Ideally the url's should be in the format site.com/xx/thread_name/ where xx is the language abbreviation. Anyone have any ideas?

Also, is there any reason that links on the translated page shouldn't be to the translated version of corresponding page? Otherwise the user would have to keep clicking their flag for each new page.

As far as vbseo urls, im not sure you would be able to make it work unless you changed your entire url structure and made existing urls be site.com/en/thread_name/

As for HREF rewriting, I checked the code and I believe its a feature dave is working on but currently its not in place.

The idea is to boost search traffic not to provide a comprehensive translation experience.

The google translation gadget offers a much faster translation experience for users if this is what you are looking for:

http://translate.google.com/translate_tools

[Dave Hybrid]

I'm trying to figure out a way I could use this with vbSEO. Ideally the url's should be in the format site.com/xx/thread_name/ where xx is the language abbreviation. Anyone have any ideas?

Also, is there any reason that links on the translated page shouldn't be to the translated version of corresponding page? Otherwise the user would have to keep clicking their flag for each new page.

Rewrite the URLs from ?hl= to directorys using htaccess ;)

[T2x]

Rewrite the URLs from ?hl= to directorys using htaccess ;)

good point. I just realized that that would actually be pretty easy

[tpearl5]

Rewrite the URLs from ?hl= to directorys using htaccess ;)
Right, but what about words in the URL being translated?

[Dave Hybrid]

Right, but what about words in the URL being translated?

You're asking if we can do that?

As far as I'm aware domain names and URLs are only allowed in English, that just the way the net is...

[GoTTi]

trying this again....

[ShawneyJ]

I believe they were, basically you need to remove the flag your base language is in.

So find the line for english and put

<!--

at the start. and

-->

at the end.

Or just delete the line altogether, same thing happens, the flag will not display.


ah yes thanks ;)EN flag seems to be gone and so far working great.
cheers ;)

[music legend]

i have curl enabled
https://vborg.vbsupport.ru/ (http://img188.imageshack.us/i/70960634.jpg/)

and i get a white screen whenever i press a flag button

[Sweeks]

Hey dave, very nice plugin.

I figured it out for you :D

<!--notranslatestart-->
Do not translate this text
<!--notranslateend-->

That is the syntax.

Change the plugin code to the following:

// Enable UTF-8 characters
if(isset($_GET['hl'])) {
require_once("translate.php");
$output = str_replace('lang="en"', 'lang="'.$_GET['hl'].'"', $output);
header ('Content-type: text/html; charset=utf-8');
// Keep remainders of <style and <script tags!
$time = time(); // Unique Tag Identifier!
preg_match_all('|<script[^>]*>(.*?)</script>|si', $output,$scripttags, PREG_SET_ORDER);
for($i=0;$i<count($scripttags);$i++)
{
$output = str_replace($scripttags[$i][0], "<a name=\"$i\"></a>",$output);
}
//preg_match_all("<style[^>]*>.*</style>",$buffer,$styletags);
preg_match_all('|<style[^>]*>(.*?)</style>|si',$output,$styletags);
for($i=0;$i<count($styletags);$i++)
{
$output = str_replace($styletags[1][$i], "<a name=\"s$i\"></a>",$output);
}
preg_match_all('|<!--notranslatestart-->(.*?)<!--notranslateend-->|si', $output,$notranslatetags, PREG_SET_ORDER);
for($i=0;$i<count($notranslatetags);$i++)
{
$output = str_replace($notranslatetags[$i][0], "<a name=\"t$i\"></a>",$output);
}
$output=callback($output);
// Place back remainders of <style and <script tags!
for($i=0;$i<count($styletags);$i++)
{
$output = str_replace("<a name=\"t$i\"></a>", $notranslatetags[$i][0], $output);
}
for($i=0;$i<count($styletags);$i++)
{
$output = str_replace("<a name=\"s$i\"></a>", $styletags[1][$i], $output);
}
for($i=0;$i<count($scripttags);$i++)
{
$output = str_replace("<a name=\"$i\"></a>",$scripttags[$i][0] ,$output);
}
//print_r($scripttags);
//die("c:".count($scripttags).":".count($styletags));
}

Also you are going to make google angry.

You need to add a @curl_setopt($ch, CURLOPT_REFERER,"http://$_SERVER[SERVER_NAME]$_SERVER[VBSEO_URI]"); (this is vbseo only code)
according to google.

and adding an api key to the request doesn't hurt.

http://code.google.com/apis/ajaxsearch/signup.html

at http://code.google.com/apis/ajaxlanguage/documentation

it says:



so basically you just change the request to look like this

Can you tell us all how to implement this now please :)

[T2x]

Alright I have updated the translation exceptions a little bit.

Basically go to your plugins in VB, and if you remember you created one that hooked "global_complete".

Find that plugin, and edit it.

Replace that plugin with this:

// Enable UTF-8 characters
if(isset($_GET['hl'])) {
require_once("translate.php");
$output = str_replace('lang="en"', 'lang="'.$_GET['hl'].'"', $output);
header ('Content-type: text/html; charset=utf-8');
// Keep remainders of <style and <script tags!
$time = time(); // Unique Tag Identifier!
preg_match_all('|<script[^>]*>(.*?)</script>|si', $output,$scripttags, PREG_SET_ORDER);
for($i=0;$i<count($scripttags);$i++)
{
$output = str_replace($scripttags[$i][0], "<a name=\"$i\"></a>",$output);
}
//preg_match_all("<style[^>]*>.*</style>",$buffer,$styletags);
preg_match_all('|<style[^>]*>(.*?)</style>|si',$output,$styletags);
for($i=0;$i<count($styletags);$i++)
{
$output = str_replace($styletags[1][$i], "<a name=\"s$i\"></a>",$output);
}
preg_match_all('/<!--ntstart-->.*?<!--ntend-->/', $output,$notranslatetags, PREG_SET_ORDER);
for($i=0;$i<count($notranslatetags);$i++)
{
$output = str_replace($notranslatetags[$i][0], "<a name=\"t$i\"></a>",$output);
}
$output=callback($output);
// Place back remainders of <style and <script tags!
for($i=0;$i<count($notranslatetags);$i++)
{
$output = str_replace("<a name=\"t$i\"></a>", $notranslatetags[$i][0], $output);
}
for($i=0;$i<count($styletags);$i++)
{
$output = str_replace("<a name=\"s$i\"></a>", $styletags[1][$i], $output);
}
for($i=0;$i<count($scripttags);$i++)
{
$output = str_replace("<a name=\"$i\"></a>",$scripttags[$i][0] ,$output);
}
//print_r($scripttags);
//die("c:".count($scripttags).":".count($styletags));
}
$output = str_replace("<!--ntstart-->", "", $output);
$output = str_replace("<!--ntend-->", "", $output);



I have changed the tags to <!--ntstart--> and <!--ntend-->

What you can do with this is if you have some piece of information that you dont want to be translated, wrap it in those tags. You can do so in your templates or php files or whatever.

You should never see those tags in your actual html page as they will be removed.

I have also made some improvements in regards to translating numbers and times. I've found that by disabling translation of time likes 01:30 PM and by disabling translation of numbers like 32,423, you sacrifice a bit of readibility for foreign users but the pages load MUCH faster. I haven't included this in any of my posts yet but I will soon.

[Sweeks]

I want to mainly know how to avoid Google being unhappy, how do I implement the API key? Thanks

[T2x]

Ok sorry, first of all here is a small bug fix of what i posted earlier, as it was slightly bugged.


// Enable UTF-8 characters
if(isset($_GET['hl'])) {
require_once("translate.php");
$output = str_replace('lang="en"', 'lang="'.$_GET['hl'].'"', $output);
header ('Content-type: text/html; charset=utf-8');
// Keep remainders of <style and <script tags!
$time = time(); // Unique Tag Identifier!
preg_match_all('|<script[^>]*>(.*?)</script>|si', $output,$scripttags, PREG_SET_ORDER);
for($i=0;$i<count($scripttags);$i++)
{
$output = str_replace($scripttags[$i][0], "<a name=\"$i\"></a>",$output);
}
//preg_match_all("<style[^>]*>.*</style>",$buffer,$styletags);
preg_match_all('|<style[^>]*>(.*?)</style>|si',$output,$styletags);
for($i=0;$i<count($styletags);$i++)
{
$output = str_replace($styletags[1][$i], "<a name=\"s$i\"></a>",$output);
}
preg_match_all('/<!--ntstart-->.*?<!--ntend-->/s', $output,$notranslatetags, PREG_SET_ORDER);
for($i=0;$i<count($notranslatetags);$i++)
{
$output = str_replace($notranslatetags[$i][0], "<a name=\"t$i\"></a>",$output);
}
$output=callback($output);
// Place back remainders of <style and <script tags!
for($i=0;$i<count($notranslatetags);$i++)
{
$output = str_replace("<a name=\"t$i\"></a>", $notranslatetags[$i][0], $output);
}
$output = str_replace("<!--ntstart-->", "", $output);
$output = str_replace("<!--ntend-->", "", $output);
for($i=0;$i<count($styletags);$i++)
{
$output = str_replace("<a name=\"s$i\"></a>", $styletags[1][$i], $output);
}
for($i=0;$i<count($scripttags);$i++)
{
$output = str_replace("<a name=\"$i\"></a>",$scripttags[$i][0] ,$output);
}
//print_r($scripttags);
//die("c:".count($scripttags).":".count($styletags));
}


As far as making google happy, ( This code will ONLY work if you have VBSEO)

Do this:

1. Open up your translate.php.

find these lines:
@curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
@curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);

change it to:

@curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
@curl_setopt($ch, CURLOPT_REFERER,"http://$_SERVER[SERVER_NAME]$_SERVER[VBSEO_URI]");
@curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);

2. Also in translate.php

Go to http://code.google.com/apis/ajaxsearch/signup.html

Get an api key for your site, copy it

find:

@curl_setopt($ch, CURLOPT_URL, "http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&langpair=".urlencode($fl.'|'.$tl)."&q=".urlencode($text));


change it to


@curl_setopt($ch, CURLOPT_URL, "http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&key=COPYYOURKEYHERE&langpair=".urlencode($fl.'|'.$tl)."&q=".urlencode($text));

Of course replace "COPYYOURKEYHERE" with the key you got from google.


That's all.

Easy as pie.

This will ensure google knows where the requests are coming from and that they can contact you in case of problems instead of just banning you from all of google by ip.

[Dave Hybrid]

FYI we have never had a site banned, after years of running these and similar scripts.

At most google will ban your IP from the API service for 24hrs, no big deal.

[T2x]

FYI we have never had a site banned, after years of running these and similar scripts. At most google will ban your IP from the API service, then you just get a new one for your server, no big deal.

Unfortunately an IP change for many sites can be a big problem.

I would not be able to easily change the ip of my site.

I always say better safe than sorry.

[Dave Hybrid]

Unfortunately an IP change for many sites can be a big problem.

I would not be able to easily change the ip of my site.

I always say better safe than sorry.

They have only ever banned my IP for 24hrs and only on the API service. You're worrying about nothing.

[nascimbeni]

I tried to Install the 2.0 version and I was able to show the flags on the site without crashing. Also the db tables are being filled correctly ( both source and translation ) but after I click on 1 flag I still get the message below

--------------------------------------------------------------------------------------
Content Encoding Error
The page you are trying to view cannot be shown because it uses an invalid or unsupported form of compression.
The page you are trying to view cannot be shown because it uses an invalid or unsupported form of compression.

* Please contact the website owners to inform them of this problem.
--------------------------------------------------------

Paolo

[Sweeks]

Im not sure whether to bother, is there any harm in adding it Dave? Thanks :up:

[Dave Hybrid]

Im not sure whether to bother, is there any harm in adding it Dave? Thanks :up:

I've been using scripts like this one years. Google will not ban your site, de-index you, penalize you or anything else.

At worst they may block your IP from the translate APi for 24hrs, they used to do this on their translate website but I have NEVER had it happen with the API, that may change.

Seriously, Google bans scraper sites, sites using only rss feeds and other autogenerated shit. It does not ban quality sites with good content be it in english or auto translated.

[Sweeks]

I think I will just keep it as is, thanks for the advice Dave :up:

[tpearl5]

You certainly can't be hurting anything by including an API key.

Here's another question... How about displaying flags based on where the IP is from? I understand this would involve a small index of IP's, but perhaps it could be an exclusion. For example, 'if not from US, display flags'

edit - nevermind - that would be a very large index either way that would have to be referenced on every visit, which sounds very expensive

[GoTTi]

ok i left it installed overnight and i checked the mysql tables and this is what it looks like

https://vborg.vbsupport.ru/external/2009/06/1.jpg

still not loading anything in the cache table. and when i click any flags, nothing is loading for me. any ideas?

[Dave Hybrid]

ok i left it installed overnight and i checked the mysql tables and this is what it looks like

https://vborg.vbsupport.ru/external/2009/06/1.jpg

still not loading anything in the cache table. and when i click any flags, nothing is loading for me. any ideas?

The wt_cache hold very long phrases, so if your pages are all full of short words it wont fill up. Try translating a thread with an article or something in it.

Either way if some of them are filling up then caching is working as far as i can see, is there an actual problem you are having?

[Geraldm]

Well I installed the new version 2.0 script yesterday.

My forum currently has 4,745 threads and 17,052 posts and today my DB looks like this:

https://vborg.vbsupport.ru/

34.7MB in 24 hours ?!!?!? I'll let you know how big this DB gets after a week.....

Cheers, and thanks for the great script!

Regards,
Gerald.

[TheLastSuperman]

Well I installed the new version 2.0 script yesterday.

My forum currently has 4,745 threads and 17,052 posts and today my DB looks like this:

https://vborg.vbsupport.ru/

34.7MB in 24 hours ?!!?!? I'll let you know how big this DB gets after a week.....

Cheers, and thanks for the great script!

Regards,
Gerald.

Skeptical and no reply needed as it is strictly IMO however a friendly warning... large databases equal more hassle to restore (tell me your backing up people :p) and some host have limits on DB sizes so be sure to check and see if your host does (including the fine print) otherwise good concept but tagged for now until I see more :D.

S-MAN

[music legend]

i have curl enabled
http://img188.imageshack.us/img188/9239/70960634.jpg (http://img188.imageshack.us/i/70960634.jpg/)

and i get a white screen whenever i press a flag button

im still having this problem

[NLP-er]

ok i left it installed overnight and i checked the mysql tables and this is what it looks like

https://vborg.vbsupport.ru/attachment.php?attachmentid=101278&stc=1&d=1246373739

still not loading anything in the cache table. and when i click any flags, nothing is loading for me. any ideas?

You can always set $enablecache = false; in translate.php - then you will know does your page translates at all. I'm quite sure that cache is not an issue here and we can be sure by temporary disabling it.

[NLP-er]

Well I installed the new version 2.0 script yesterday.

My forum currently has 4,745 threads and 17,052 posts and today my DB looks like this:

http://www.delphi-php.net/images/records24.jpg

34.7MB in 24 hours ?!!?!? I'll let you know how big this DB gets after a week.....

Cheers, and thanks for the great script!

Regards,
Gerald.

Well - in vb you have 1 content. Here it is multiplied for each translation. As long as you have space on server - who cares?... ;)

[Dave Hybrid]

Exactly, would you delete normal posts because your DB got a big big?

No, you'd keep them and upgrade the server if needed, so what's the difference... lol.

People worried about gaining too much content, that's a new one he he.

[T2x]

You certainly can't be hurting anything by including an API key.

Here's another question... How about displaying flags based on where the IP is from? I understand this would involve a small index of IP's, but perhaps it could be an exclusion. For example, 'if not from US, display flags'

edit - nevermind - that would be a very large index either way that would have to be referenced on every visit, which sounds very expensive


Nah its actually very easy to do. I have an ip database on my site which performs a similar function.

I would recommend using javascript instead to read the browser language.

var nav_lng="nolang"; //default language string
if(navigator.language)nav_lng=navigator.language;
else if(navigator.browserLanguage)nav_lng=navigator.bro wserLanguage;
else if(navigator.userLanguage)nav_lng=navigator.userLa nguage;
else if(navigator.systemLanguage)nav_lng=navigator.syst emLanguage;
else if(window.navigator.language)nav_lng=window.naviga tor.language;


for example, for english visitors the nav_lng variable will then contain either en-US or en-GB(for users from the UK or AU)


I actually have dynamic javascript translation going on as well right now using this and it works fine.

[burlesque]

I was also getting the blank page when I clicked on a flag, however when I set the caching to 'false' it translated fine.

However, when I add set the cache to 'true' I get the white page again.

But if I set the cache to false once again, everything translates fine, until I set the cache to 'true'...

.. and then I get a blank page again.

The db has been installed correctly but I'm at a loss as to why this happens.

[music legend]

I was also getting the blank page when I clicked on a flag, however when I set the caching to 'false' it translated fine.

However, when I add set the cache to 'true' I get the white page again.

But if I set the cache to false once again, everything translates fine, until I set the cache to 'true'...

.. and then I get a blank page again.

The db has been installed correctly but I'm at a loss as to why this happens.

where is the caching option you turned off?

[NLP-er]

Exactly, would you delete normal posts because your DB got a big big?

No, you'd keep them and upgrade the server if needed, so what's the difference... lol.

People worried about gaining too much content, that's a new one he he.

:D exactly I'm very happy when new posts arrives on my forum - more is better, or people will go somewhere where they will find what they are looking for...

[burlesque]

in translate.php

$enablesession = false; //ignore
$enablecache = false; //true - enable cache setting, false - disable cache setting

[NLP-er]

I was also getting the blank page when I clicked on a flag, however when I set the caching to 'false' it translated fine.

However, when I add set the cache to 'true' I get the white page again.

But if I set the cache to false once again, everything translates fine, until I set the cache to 'true'...

.. and then I get a blank page again.

The db has been installed correctly but I'm at a loss as to why this happens.

In such case cache is an issue here. Can you PM to me some more details? I will gladly see what is gooing on :)

[music legend]

yeah that fixed my white screen as well when i turned it to false it worked

if its set to true i just get the white screen

[burlesque]

In such case cache is an issue here. Can you PM to me some more details? I will gladly see what is gooing on :)

PM Sent :)

[music legend]

PM Sent :)

let me know if yall figure out the problem because im sure its the same one i am having

[Dave Hybrid]

v2.1

* Meta description and meta keywords now translated in the output source code, better for SEO and search engine indexing
* Note no change to files

To update just replace your vB Global Translator plugin code with what is below.

Also change the Execution Order: to 50

// Enable UTF-8 characters
if(isset($_GET['hl'])) {
$output = str_replace('lang="en"', 'lang="'.$_GET['hl'].'"', $output);
header ('Content-type: text/html; charset=utf-8');
// Keep remainders of <style and <script tags!
$time = time(); // Unique Tag Identifier!
preg_match_all('|<script[^>]*>(.*?)</script>|si', $output,$scripttags, PREG_SET_ORDER);
for($i=0;$i<count($scripttags);$i++)
{
$output = str_replace($scripttags[$i][0], "<a name=\"$i\"></a>",$output);
}
//preg_match_all("<style[^>]*>.*</style>",$buffer,$styletags);
preg_match_all('|<style[^>]*>(.*?)</style>|si',$output,$styletags);
for($i=0;$i<count($styletags);$i++)
{
$output = str_replace($styletags[1][$i], "<a name=\"s$i\"></a>",$output);
}
require_once("translate.php");
$output=callback($output);
// Place back remainders of <style and <script tags!
for($i=0;$i<count($styletags);$i++)
{
$output = str_replace("<a name=\"s$i\"></a>", $styletags[1][$i], $output);
}
for($i=0;$i<count($scripttags);$i++)
{
$output = str_replace("<a name=\"$i\"></a>",$scripttags[$i][0] ,$output);
}

// Translate META tags.
preg_match_all("|<meta[^>]+name=\"([^\"]*)\"[^>]" . "+content=\"([^\"]*)\"[^>]+>|i",$output, $out,PREG_PATTERN_ORDER);
$meta = array();
for ($i=0;$i < count($out[1]);$i++) {
if (strtolower($out[1][$i]) == "keywords") $meta['keywords'] = $out[2][$i];
if (strtolower($out[1][$i]) == "description") $meta['description'] = $out[2][$i];
}

// Start replacing the original META tags with translated tags.
$output=str_replace("<meta name=\"description\" content=\"".$meta['description']."\" />","<meta name=\"description\" content=\"".html_entity_decode(callback($meta['description']))."\" />",$output);
$output=str_replace("<meta name=\"keywords\" content=\"".$meta['keywords']."\" />","<meta name=\"keywords\" content=\"".html_entity_decode(callback($meta['keywords']))."\" />",$output);
}

[Sweeks]

Nice one this mod just keeps getting better :up:

[Dave Hybrid]

Thank you, more to come. ;P

[Dave Hybrid]

In such case cache is an issue here. Can you PM to me some more details? I will gladly see what is gooing on :)

Thanks for helping out. :up:

[Sweeks]

Maybe throw in that API key option just to put everyone at peace if it doesnt have a performance hit :)

[Geraldm]

Hi just tried v2.1 and it's not translating the meta keywords or meta Description as described??

Dave, you know my forum as I've PMed you it in the past ... can you please take a look? :)

Thanks,
Gerald.

[Dave Hybrid]

Hi just tried v2.1 and it's not translating the meta keywords or meta Description as described??

Dave, you know my forum as I've PMed you it in the past ... can you please take a look? :)

Thanks,
Gerald.

You pasted the new plugin code and changed the Execution order?

PM me the url again please.

[Dave Hybrid]

Hi just tried v2.1 and it's not translating the meta keywords or meta Description as described??

Dave, you know my forum as I've PMed you it in the past ... can you please take a look? :)

Thanks,
Gerald.

I managed to find your old PM, it looks fine to me mate.

[Geraldm]

I managed to find your old PM, it looks fine to me mate.

Ahh it was only translating the keywords and not the description ...... I re-read your instructions and noticed I missed the bit about changing the execution order :) I've now changed it to 50 and yes it's now translating the keywords and desc.... Great stuff! :)

Thank you!

[Dave Hybrid]

Great, you're welcome. :up:

[GoTTi]

You can always set $enablecache = false; in translate.php - then you will know does your page translates at all. I'm quite sure that cache is not an issue here and we can be sure by temporary disabling it.

its doing the same thing. i put false on the .php and reuploaded, reloaded the pages, and clicked a flag and all its doing is loading, then it stops.

[GoTTi]

i just installed a testboard, clean, no mods on it and only installed this mod here, made a post, clicked a flag and nothing is happening. what could possibly be causing this to not work?

[NLP-er]

Since some installations still have problems using cache and we are working on that, for those who has running vb global translator I have some optimizations :)

1. Query optimization - change in template.php code between /* Check cache for translation */ and /* -- if not found, proceed with Google Translate */ to:

/* Check cache for translation */
if ($enablecache) {
$sql=null;
$length = strlen($text);
if ($length<=50) {
$sql = mysql_query("SELECT translated FROM wt_cache_short WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1");
} else if ($length > 255) {
$sql = mysql_query("SELECT translated FROM wt_cache WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1");
} else {
$sql = mysql_query("SELECT translated FROM wt_cache_medium WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1");
}

while($t = mysql_fetch_array($sql)) {
return $lsto.$t['translated'].$rsto;
}
}
/* -- if not found, proceed with Google Translate */


Unnecessary data was taken from DB. Hope Dave will make new release quickly and include this change :)

2. Index changes. Since for someone DB was growing fast I made some tests. It is enough to change indexes.

For installed products run those queries:

alter table wt_cache_short drop index tl;
create index originaltext on wt_cache_short (originaltext, tl);

alter table wt_cache_medium drop index tl;
create index originaltext on wt_cache_medium (originaltext, tl);

alter table wt_cache drop index tl;
create index originaltext on wt_cache (originaltext(50), tl);


For new installations it will be:

CREATE TABLE wt_cache (
id INT AUTO_INCREMENT NOT NULL PRIMARY KEY,
tl VARCHAR(10),
originaltext VARCHAR(65000),
translated TEXT,
INDEX (originaltext (50), tl)
) ENGINE = MYISAM, CHARACTER SET utf8 COLLATE utf8_general_ci;

CREATE TABLE wt_cache_medium (
id INT AUTO_INCREMENT NOT NULL PRIMARY KEY,
tl VARCHAR(10),
originaltext VARCHAR(255),
translated VARCHAR(1000),
INDEX (originaltext, tl)
) ENGINE = MYISAM, CHARACTER SET utf8 COLLATE utf8_general_ci;

CREATE TABLE wt_cache_short (
id INT AUTO_INCREMENT NOT NULL PRIMARY KEY,
tl VARCHAR(10),
originaltext VARCHAR(50),
translated VARCHAR(255),
INDEX (originaltext, tl)
) ENGINE = MYISAM, CHARACTER SET utf8 COLLATE utf8_general_ci;


Hope Dave will change installation manual to this one.

I realized that column tl has only 28 possible values (or less if you don’t use all flags) and that is not too good for indexes. Only by changing order from (tl, originaltext) to (originaltext, tl) my database size changed from 51MB to 44,3MB without any performance lost and even with improvement. Changing it only to (originaltext) have worst results that (originaltext, tl).

Here are some results of my tests (I have weak internet connection so don't look on times but on differences between performance in different configurations):
WITH index (tl,originaltext) DB size=51MB TESTS RESULTS:
MIN: 4484 MAX: 13266 AVG: 7823
MIN: 3188 MAX: 9703 AVG: 5848
MIN: 3797 MAX: 9594 AVG: 5683
TOTAL AVG: 6451

WITH index (originaltext,tl) DB size=44,3MB TESTS RESULTS:
MIN: 2859 MAX: 10172 AVG: 6232
MIN: 2890 MAX: 9094 AVG: 6109
MIN: 3468 MAX: 9500 AVG: 5813
TOTAL AVG: 6051

WITH index (originaltext) DB size=43,2MB TESTS RESULTS:
MIN: 4454 MAX: 10406 AVG: 5998
MIN: 3703 MAX: 13922 AVG: 6341
MIN: 2922 MAX: 14578 AVG: 6888
TOTAL AVG: 6409

Times are in ms. In each test 20 different cached translated pages was generated. For each setting 3 tests series at row was executed.

[NLP-er]

For those who have problems with using cache at all - right now I suspect that there is problem with making connection to DB. Even when all settings are ok - It seems that script is waiting for connection and DB holds it. Is someone with this problem has a clue what to change (maybe some number of connections limit) It would be good if share solution with others. As long as I'm not able to reproduce this on my side, or someone will give me access to his forum, I'm not even sure does this is a problem - and it is only a suspicion.

[NLP-er]

i just installed a testboard, clean, no mods on it and only installed this mod here, made a post, clicked a flag and nothing is happening. what could possibly be causing this to not work?

If it is clean, I think you could PM me access data to your adminCP and FTP. Then I can check what is going on :) Right now I can only tell that somesthing wrong ;)

[alqloob alsahya]

nice

Excellent subject of very beautiful

Thank you

[GoTTi]

my site is blocked from google adsense, would that matter in this situation?

[GoTTi]

If it is clean, I think you could PM me access data to your adminCP and FTP. Then I can check what is going on :) Right now I can only tell that somesthing wrong ;)

ive sent u a PM with the login.

[Dave Hybrid]

v2.2

* More improvements to database code, better speed and lower data size
* Flag files combined size reduced from 40kb to 3kb for faster page load speed

To upgrade download new package, upload new translate.php and /flags/ folder.

Run the following MySQL query on your DB via phpmyadmin.

alter table wt_cache_short drop index tl;
create index originaltext on wt_cache_short (originaltext, tl);

alter table wt_cache_medium drop index tl;
create index originaltext on wt_cache_medium (originaltext, tl);

alter table wt_cache drop index tl;
create index originaltext on wt_cache (originaltext(50), tl);

Thanks!

[GamerPerfection]

I been on holiday since 21st Jun.
You mean I have to do this all over again?

[Sweeks]

v2.2

* More improvements to database code, better speed and lower data size
* Flag files combined size reduced from 40kb to 3kb for faster page load speed

To upgrade download new package, upload new translate.php and /flags/ folder.

Run the following MySQL query on your DB via phpmyadmin.

alter table wt_cache_short drop index tl;
create index originaltext on wt_cache_short (originaltext, tl);

alter table wt_cache_medium drop index tl;
create index originaltext on wt_cache_medium (originaltext, tl);

alter table wt_cache drop index tl;
create index originaltext on wt_cache (originaltext(50), tl);

Thanks!

Yet another excellent update, much smoother now Dave :up:

[Dave Hybrid]

I been on holiday since 21st Jun.
You mean I have to do this all over again?

If you want to run the better version yes, it's called progression. ;P

[GamerPerfection]

Installed.

I noticed the overloadcache file is no longer in the zip file, can I just delete the old file that is still on my server?

[NLP-er]

Since some users complains that when cache enabled white screan apperas I made solution for that. Testes on one server with problem, if doesn't solve yours it will give at least some more information now.

Note that some IP can suffer penalties for google and translation is terribly slow then (1 known case with domain banned in AdSense). During that our connection was passing out. So this update will reestablish connection even if goole gives you really long time to wait. It allows you to fululfill cache, so after that You will not need google anymore (only for new content).

NOTE: I know also one case when google give other penalty that time to wait. The penalty was - empty response to all queries. In such case It is between you and google ;)

Change translate.php to this one and set up DB and other params.

Dave - It would be good to have it in official release :)


<?php
////////////////////////////////////////////////////
// Global Translator API
////////////////////////////////////////////////////

global $enablesession, $enablecache, $originalEncoding, $fl;
global $dbusername, $dbpassword, $mysqlserver, $dbname;

////////////////////////////////////////////////////
// SETTINGS
////////////////////////////////////////////////////
$enablesession = false; //ignore
$enablecache = true; //true - enable cache setting, false - disable cache setting
$originalEncoding = 'iso-8859-1'; //refer to your forum source code for your base encoding
$fl = 'en'; //current language (original) - refer table for language varibles

$dbusername = "SET IT";
$dbpassword = "SET IT";
$mysqlserver = "localhost";
$dbname = "SET IT";

////////////////////////////////////////////////////

if ($enablesession) {
@session_name("iPWTLang");
@session_start();
}


set_time_limit(0);


////////////////////////////////////////////////////
if ($enablecache) {
establishConnection();
}


function establishConnection() {
global $dbusername, $dbpassword, $mysqlserver, $dbname;
mysql_connect ($mysqlserver, $dbusername, $dbpassword) or die('Could not connect: ' . mysql_error());
mysql_select_db ($dbname) or die('Could not select database');
mysql_set_charset('utf8');
}
////////////////////////////////////////////////////

function api_strip ($dt) { $dt = str_replace("\\\\", '\\', $dt); return ($dt); }

function emodfix ($dt) { $dt = str_replace('\"', '"', $dt); return ($dt); }

function remove_tags ($html) {
/* Remove reserved triple tags in html */
return preg_replace ("|<<<([^><]*)>>>|e", "emodfix('\\1')", $html);
}

function add_tags ($buffer) {
/* Inject triple tags to html to preserve html content from translation using ob_start */
return preg_replace("/(^|>\/?)([^><]*)($|\/?<)/e",
"emodfix('\\1').'<<<'.emodfix('\\2').'>>>'.emodfix('\\3')",
$buffer);
}

function getServerLoad($windows=false)
{
//MY CODE TO DISABLE SERVER LOAD CHEcKING
return 1;
//END

$os = strtolower(PHP_OS);
if (strpos($os, "win") === false) {
if (file_exists("/proc/loadavg")) {
$data = file_get_contents("/proc/loadavg");
$load = explode(' ', $data);
return $load[0];

} elseif (function_exists("shell_exec")) {

$load = explode(' ', `uptime`);
return $load[count($load)-1];

} else {

return false;
}

} elseif($windows) {

if(class_exists("COM")) {
$wmi = new COM("WinMgmts:\\\\.");
$cpus = $wmi->InstancesOf("Win32_Processor");

$cpuload = 0;
$i = 0;

if(version_compare('4.50.0', PHP_VERSION) == 1) {
// PHP 4
while ($cpu = $cpus->Next()) {
$cpuload += $cpu->LoadPercentage;
$i++;
}

} else {

// PHP 5
foreach ( $cpus as $cpu ) {
$cpuload += $cpu->LoadPercentage;
$i++;
}
}

$cpuload = round($cpuload / $i, 2);
return "$cpuload%";

} else {

return false;
}
}
}

function translate($text, $fl, $tl){
if (trim($text) == null) return $text; //skip translation if string empty

/* Retain left and right spaces after translation */
$lsto = substr($text, 0, strlen($text) - strlen(ltrim($text)));
$rsto = substr($text, strlen(rtrim($text)), strlen($text) - strlen(rtrim($text)));
/* -- */

/* Declare global */
global $enablecache,$overloadval,$disinterval;
if(floatval(getServerLoad()) >= 1.5) // Numeral = server load on which to crash
{

// Write current time to file.
$fp = fopen("overloadcache.txt","w");
fwrite($fp, time() . "");
fclose($fp);


if($nfh= @fopen("overloadcache.txt","r"))
{
$overloadtime = fgets($nfh);
if($overloadtime > time()-30) // Numeral = # seconds after overload not to cache!
{
$enablecache = false;
}
fclose($nfh);
}
}

/* Check for server overloads (modification by DeViAnThans3)*/
global $originalEncoding;
if ($originalEncoding != 'utf-8') {
$text = iconv($originalEncoding, 'utf-8', trim($text));
} else {
$text = trim($text);
}

/* Check cache for translation */
if ($enablecache) {
$sql=null;
$length = strlen($text);
/*
if (!mysql_ping()) {
mysql_close();
establishConnection();
}
*/
if ($length<=50) {
$sql = mysql_query("SELECT translated FROM wt_cache_short WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1") or die('Query to short cache failed: ' . mysql_error());
} else if ($length > 255) {
$sql = mysql_query("SELECT translated FROM wt_cache WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1") or die('Query to normal cache failed: ' . mysql_error());
} else {
$sql = mysql_query("SELECT translated FROM wt_cache_medium WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1") or die('Query to medium cache failed: ' . mysql_error());
}

while($t = mysql_fetch_array($sql)) {
return $lsto.$t['translated'].$rsto;
}
}
/* -- if not found, proceed with Google Translate */

/* -Establish cURL connection to Google Translate API server- */
$ch = @curl_init();
//@curl_setopt($ch, CURLOPT_URL, "http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&langpair=".urlencode($fl.'|'.$tl)."&q=".urlencode($text));

@curl_setopt($ch, CURLOPT_URL, "http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&langpair=".urlencode($fl.'|'.$tl)."&q=".urlencode($text));

//$postParams = "v=1.0&langpair=".$fl.'|'.$tl."&q=".iconv('iso-8859-2', 'utf-8', $text);
//@curl_setopt($ch, CURLOPT_URL, "http://ajax.googleapis.com/ajax/services/language/translate");
//@curl_setopt ($Curl_Session, CURLOPT_POST, 1);
//@curl_setopt ($Curl_Session, CURLOPT_POSTFIELDS, $postParams);

@curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax)");
@curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
@curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
$in = @curl_exec ($ch);
/* -End connection- */

preg_match('/{"translatedText":"(.*?)"}/', $in, $out);
//$ttext = conv(api_strip($out[1]));
$ttext = str_replace ('\u0026', '&' , api_strip($out[1]));
//$ttext = api_strip($out[1]);
//$ttext = unicode_encode(api_strip($out[1]), 'utf-8');

/* Save to cache */
if ($enablecache) {
$length = strlen($text);

if (!mysql_ping()) {
mysql_close();
establishConnection();
}

if ($length<=50) {
mysql_query("INSERT INTO wt_cache_short SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'") or die('Insert to short cache failed: ' . mysql_error());
} else if ($length > 255) {
mysql_query("INSERT INTO wt_cache SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'") or die('Insert to normal cache failed: ' . mysql_error());
} else {
mysql_query("INSERT INTO wt_cache_medium SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'") or die('Insert to medium cache failed: ' . mysql_error());
}
}
/* -- */

return $lsto.$ttext.$rsto;
}


function callback($buffer){
global $fl;
$modifyhref = false;
//-------------------------------------------------------------------------------
global $enablesession;

// Get language setting from queries:
$lang = stripslashes(@$_GET['hl']);

if ($enablesession) {
// Save setting to server session [1]
// If language setting not found in queries, check session data: [2]
if (checklang($lang)) $_SESSION['lang'] = $lang; else $lang = @$_SESSION['lang'];
}

if (!checklang($lang) || $lang == $fl) { // If invalid language setting or language = original language, skip translation.

$tp = $buffer;
$tp = preg_replace("/<disp_lang>/is", checklang($fl), $tp);
return remove_tags($tp);

} else {
// Extract BUFFER
$tp = preg_replace("/(^|>\/?)([^><]*)($|\/?<)/e",
"emodfix('\\1').translate(emodfix('\\2'),'$fl','$la ng').emodfix('\\3')",
$buffer);

if ($modifyhref) {
$tp = preg_replace('/(<a [^><]*href\=")([^"]*)("[^><]*>)/e',
"emodfix('\\1').langconv('$lang',emodfix('\\2')).em odfix('\\3')",
$tp);

$tp = preg_replace('/(<form [^><]*action\=")([^"]*)("[^><]*>)/e',
"emodfix('\\1').langconv('$lang',emodfix('\\2')).em odfix('\\3'))",
$tp);
}

$tp = preg_replace("/<disp_lang>/is", checklang($lang), $tp);
$tp = remove_tags($tp);
return $tp;

}
//---end function
}

function langconv($hl, $href){
if ($href && substr($href, 0, 7) != 'http://' && substr($href, 0, 11) != 'javascript:' && strpos($href, "#") === false) {
if(strpos($href, "?") === false) return $href.'?hl='.urlencode($hl); else return $href.'&hl='.urlencode($hl);
} else { return $href; }
}

function checklang($hl){
$langvar = array ("sq"=>"Albanian","ar"=>"Arabic","bg"=>"Bulgarian","ca"=>"Catalan","zh-CN"=>"Chinese","hr"=>"Croatian","cs"=>"Czech","da"=>"Danish","nl"=>"Dutch","en"=>"English","et"=>"Estonian","tl"=>"Filipino","fi"=>"Finnish","fr"=>"French","gl"=>"Galician","de"=>"German","el"=>"Greek","iw"=>"Hebrew","hi"=>"Hindi","hu"=>"Hungarian","id"=>"Indonesian","it"=>"Italian","ja"=>"Japanese","ko"=>"Korean","lv"=>"Latvian","lt"=>"Lithuanian","mt"=>"Maltese","no"=>"Norwegian","pl"=>"Polish","pt"=>"Portuguese","ro"=>"Romanian","ru"=>"Russian","sr"=>"Serbian","sk"=>"Slovak","sl"=>"Slovenian","es"=>"Spanish","sv"=>"Swedish","zh-TW"=>"Taiwanese","th"=>"Thai","tr"=>"Turkish","uk"=>"Ukrainian","vi"=>"Vietnamese");
foreach ($langvar as $i => $val){ if ($i == $hl) return $val; }
return false;
}

function clearlangsetting(){
global $enablesession;
if ($enablesession) $_SESSION['lang'] = null; /* -Clear lang session data on server- */
}

/* Unicode UTF-8 Support Libraries */
/*
function conv($str) {
$str = preg_replace("/\\\u([a-zA-Z0-9]{4})/e",
"unichr('\\1')",
$str);
return html_entity_decode_utf8($str);
//return $str;
}


function unichr($hex) {
$dec = hexdec($hex);

if ($dec < 128) {
$utf = chr($dec);
} else if ($dec < 2048) {
$utf = chr(192 + (($dec - ($dec % 64)) / 64));
$utf .= chr(128 + ($dec % 64));
} else {
$utf = chr(224 + (($dec - ($dec % 4096)) / 4096));
$utf .= chr(128 + ((($dec % 4096) - ($dec % 64)) / 64));
$utf .= chr(128 + ($dec % 64));
}
return $utf;
}


function html_entity_decode_utf8($string)
{
static $trans_tbl;

// replace numeric entities
$string = preg_replace('~&#x([0-9a-f]+);~ei', 'code2utf(hexdec("\\1"))', $string);
$string = preg_replace('~&#([0-9]+);~e', 'code2utf(\\1)', $string);

// replace literal entities
if (!isset($trans_tbl))
{
$trans_tbl = array();

foreach (get_html_translation_table(HTML_ENTITIES) as $val=>$key)
$trans_tbl[$key] = utf8_encode($val);
}

return strtr($string, $trans_tbl);
}

// Returns the utf string corresponding to the unicode value
function code2utf($num)
{
if ($num < 128) return chr($num);
if ($num < 2048) return chr(($num >> 6) + 192) . chr(($num & 63) + 128);
if ($num < 65536) return chr(($num >> 12) + 224) . chr((($num >> 6) & 63) + 128) . chr(($num & 63) + 128);
if ($num < 2097152) return chr(($num >> 18) + 240) . chr((($num >> 12) & 63) + 128) . chr((($num >> 6) & 63) + 128) . chr(($num & 63) + 128);
return '';
}
*/
/* -- */

?>

[Dave Hybrid]

Installed.

I noticed the overloadcache file is no longer in the zip file, can I just delete the old file that is still on my server?

You need to delete everything and fresh install, that included all files, flags and a brand new database and plugins. Everything.

[Dave Hybrid]

Since some users complains that when cache enabled white screan apperas I made solution for that. Testes on one server with problem, if doesn't solve yours it will give at least some more information now.

Note that some IP can suffer penalties for google and translation is terribly slow then (1 known case with domain banned in AdSense). During that our connection was passing out. So this update will reestablish connection even if goole gives you really long time to wait. It allows you to fululfill cache, so after that You will not need google anymore (only for new content).

NOTE: I know also one case when google give other penalty that time to wait. The penalty was - empty response to all queries. In such case It is between you and google ;)

Change translate.php to this one and set up DB and other params.

Dave - It would be good to have it in official release :)


<?php
////////////////////////////////////////////////////
// Global Translator API
////////////////////////////////////////////////////

global $enablesession, $enablecache, $originalEncoding, $fl;
global $dbusername, $dbpassword, $mysqlserver, $dbname;

////////////////////////////////////////////////////
// SETTINGS
////////////////////////////////////////////////////
$enablesession = false; //ignore
$enablecache = true; //true - enable cache setting, false - disable cache setting
$originalEncoding = 'iso-8859-1'; - //refer to your forum source code for your base encoding
$fl = 'en'; //current language (original) - refer table for language varibles

$dbusername = "SET IT";
$dbpassword = "SET IT";
$mysqlserver = "localhost";
$dbname = "SET IT";

////////////////////////////////////////////////////

if ($enablesession) {
@session_name("iPWTLang");
@session_start();
}


set_time_limit(0);


////////////////////////////////////////////////////
if ($enablecache) {
establishConnection();
}


function establishConnection() {
global $dbusername, $dbpassword, $mysqlserver, $dbname;
mysql_connect ($mysqlserver, $dbusername, $dbpassword) or die('Could not connect: ' . mysql_error());
mysql_select_db ($dbname) or die('Could not select database');
mysql_set_charset('utf8');
}
////////////////////////////////////////////////////

function api_strip ($dt) { $dt = str_replace("\\\\", '\\', $dt); return ($dt); }

function emodfix ($dt) { $dt = str_replace('\"', '"', $dt); return ($dt); }

function remove_tags ($html) {
/* Remove reserved triple tags in html */
return preg_replace ("|<<<([^><]*)>>>|e", "emodfix('\\1')", $html);
}

function add_tags ($buffer) {
/* Inject triple tags to html to preserve html content from translation using ob_start */
return preg_replace("/(^|>\/?)([^><]*)($|\/?<)/e",
"emodfix('\\1').'<<<'.emodfix('\\2').'>>>'.emodfix('\\3')",
$buffer);
}

function getServerLoad($windows=false)
{
//MY CODE TO DISABLE SERVER LOAD CHEcKING
return 1;
//END

$os = strtolower(PHP_OS);
if (strpos($os, "win") === false) {
if (file_exists("/proc/loadavg")) {
$data = file_get_contents("/proc/loadavg");
$load = explode(' ', $data);
return $load[0];

} elseif (function_exists("shell_exec")) {

$load = explode(' ', `uptime`);
return $load[count($load)-1];

} else {

return false;
}

} elseif($windows) {

if(class_exists("COM")) {
$wmi = new COM("WinMgmts:\\\\.");
$cpus = $wmi->InstancesOf("Win32_Processor");

$cpuload = 0;
$i = 0;

if(version_compare('4.50.0', PHP_VERSION) == 1) {
// PHP 4
while ($cpu = $cpus->Next()) {
$cpuload += $cpu->LoadPercentage;
$i++;
}

} else {

// PHP 5
foreach ( $cpus as $cpu ) {
$cpuload += $cpu->LoadPercentage;
$i++;
}
}

$cpuload = round($cpuload / $i, 2);
return "$cpuload%";

} else {

return false;
}
}
}

function translate($text, $fl, $tl){
if (trim($text) == null) return $text; //skip translation if string empty

/* Retain left and right spaces after translation */
$lsto = substr($text, 0, strlen($text) - strlen(ltrim($text)));
$rsto = substr($text, strlen(rtrim($text)), strlen($text) - strlen(rtrim($text)));
/* -- */

/* Declare global */
global $enablecache,$overloadval,$disinterval;
if(floatval(getServerLoad()) >= 1.5) // Numeral = server load on which to crash
{

// Write current time to file.
$fp = fopen("overloadcache.txt","w");
fwrite($fp, time() . "");
fclose($fp);


if($nfh= @fopen("overloadcache.txt","r"))
{
$overloadtime = fgets($nfh);
if($overloadtime > time()-30) // Numeral = # seconds after overload not to cache!
{
$enablecache = false;
}
fclose($nfh);
}
}

/* Check for server overloads (modification by DeViAnThans3)*/
global $originalEncoding;
if ($originalEncoding != 'utf-8') {
$text = iconv($originalEncoding, 'utf-8', trim($text));
} else {
$text = trim($text);
}

/* Check cache for translation */
if ($enablecache) {
$sql=null;
$length = strlen($text);
/*
if (!mysql_ping()) {
mysql_close();
establishConnection();
}
*/
if ($length<=50) {
$sql = mysql_query("SELECT translated FROM wt_cache_short WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1") or die('Query to short cache failed: ' . mysql_error());
} else if ($length > 255) {
$sql = mysql_query("SELECT translated FROM wt_cache WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1") or die('Query to normal cache failed: ' . mysql_error());
} else {
$sql = mysql_query("SELECT translated FROM wt_cache_medium WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1") or die('Query to medium cache failed: ' . mysql_error());
}

while($t = mysql_fetch_array($sql)) {
return $lsto.$t['translated'].$rsto;
}
}
/* -- if not found, proceed with Google Translate */

/* -Establish cURL connection to Google Translate API server- */
$ch = @curl_init();
//@curl_setopt($ch, CURLOPT_URL, "http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&langpair=".urlencode($fl.'|'.$tl)."&q=".urlencode($text));

@curl_setopt($ch, CURLOPT_URL, "http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&langpair=".urlencode($fl.'|'.$tl)."&q=".urlencode($text));

//$postParams = "v=1.0&langpair=".$fl.'|'.$tl."&q=".iconv('iso-8859-2', 'utf-8', $text);
//@curl_setopt($ch, CURLOPT_URL, "http://ajax.googleapis.com/ajax/services/language/translate");
//@curl_setopt ($Curl_Session, CURLOPT_POST, 1);
//@curl_setopt ($Curl_Session, CURLOPT_POSTFIELDS, $postParams);

@curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax)");
@curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
@curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
$in = @curl_exec ($ch);
/* -End connection- */

preg_match('/{"translatedText":"(.*?)"}/', $in, $out);
//$ttext = conv(api_strip($out[1]));
$ttext = str_replace ('\u0026', '&' , api_strip($out[1]));
//$ttext = api_strip($out[1]);
//$ttext = unicode_encode(api_strip($out[1]), 'utf-8');

/* Save to cache */
if ($enablecache) {
$length = strlen($text);

if (!mysql_ping()) {
mysql_close();
establishConnection();
}

if ($length<=50) {
mysql_query("INSERT INTO wt_cache_short SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'") or die('Insert to short cache failed: ' . mysql_error());
} else if ($length > 255) {
mysql_query("INSERT INTO wt_cache SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'") or die('Insert to normal cache failed: ' . mysql_error());
} else {
mysql_query("INSERT INTO wt_cache_medium SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'") or die('Insert to medium cache failed: ' . mysql_error());
}
}
/* -- */

return $lsto.$ttext.$rsto;
}


function callback($buffer){
global $fl;
$modifyhref = false;
//-------------------------------------------------------------------------------
global $enablesession;

// Get language setting from queries:
$lang = stripslashes(@$_GET['hl']);

if ($enablesession) {
// Save setting to server session [1]
// If language setting not found in queries, check session data: [2]
if (checklang($lang)) $_SESSION['lang'] = $lang; else $lang = @$_SESSION['lang'];
}

if (!checklang($lang) || $lang == $fl) { // If invalid language setting or language = original language, skip translation.

$tp = $buffer;
$tp = preg_replace("/<disp_lang>/is", checklang($fl), $tp);
return remove_tags($tp);

} else {
// Extract BUFFER
$tp = preg_replace("/(^|>\/?)([^><]*)($|\/?<)/e",
"emodfix('\\1').translate(emodfix('\\2'),'$fl','$la ng').emodfix('\\3')",
$buffer);

if ($modifyhref) {
$tp = preg_replace('/(<a [^><]*href\=")([^"]*)("[^><]*>)/e',
"emodfix('\\1').langconv('$lang',emodfix('\\2')).em odfix('\\3')",
$tp);

$tp = preg_replace('/(<form [^><]*action\=")([^"]*)("[^><]*>)/e',
"emodfix('\\1').langconv('$lang',emodfix('\\2')).em odfix('\\3'))",
$tp);
}

$tp = preg_replace("/<disp_lang>/is", checklang($lang), $tp);
$tp = remove_tags($tp);
return $tp;

}
//---end function
}

function langconv($hl, $href){
if ($href && substr($href, 0, 7) != 'http://' && substr($href, 0, 11) != 'javascript:' && strpos($href, "#") === false) {
if(strpos($href, "?") === false) return $href.'?hl='.urlencode($hl); else return $href.'&hl='.urlencode($hl);
} else { return $href; }
}

function checklang($hl){
$langvar = array ("sq"=>"Albanian","ar"=>"Arabic","bg"=>"Bulgarian","ca"=>"Catalan","zh-CN"=>"Chinese","hr"=>"Croatian","cs"=>"Czech","da"=>"Danish","nl"=>"Dutch","en"=>"English","et"=>"Estonian","tl"=>"Filipino","fi"=>"Finnish","fr"=>"French","gl"=>"Galician","de"=>"German","el"=>"Greek","iw"=>"Hebrew","hi"=>"Hindi","hu"=>"Hungarian","id"=>"Indonesian","it"=>"Italian","ja"=>"Japanese","ko"=>"Korean","lv"=>"Latvian","lt"=>"Lithuanian","mt"=>"Maltese","no"=>"Norwegian","pl"=>"Polish","pt"=>"Portuguese","ro"=>"Romanian","ru"=>"Russian","sr"=>"Serbian","sk"=>"Slovak","sl"=>"Slovenian","es"=>"Spanish","sv"=>"Swedish","zh-TW"=>"Taiwanese","th"=>"Thai","tr"=>"Turkish","uk"=>"Ukrainian","vi"=>"Vietnamese");
foreach ($langvar as $i => $val){ if ($i == $hl) return $val; }
return false;
}

function clearlangsetting(){
global $enablesession;
if ($enablesession) $_SESSION['lang'] = null; /* -Clear lang session data on server- */
}

/* Unicode UTF-8 Support Libraries */
/*
function conv($str) {
$str = preg_replace("/\\\u([a-zA-Z0-9]{4})/e",
"unichr('\\1')",
$str);
return html_entity_decode_utf8($str);
//return $str;
}


function unichr($hex) {
$dec = hexdec($hex);

if ($dec < 128) {
$utf = chr($dec);
} else if ($dec < 2048) {
$utf = chr(192 + (($dec - ($dec % 64)) / 64));
$utf .= chr(128 + ($dec % 64));
} else {
$utf = chr(224 + (($dec - ($dec % 4096)) / 4096));
$utf .= chr(128 + ((($dec % 4096) - ($dec % 64)) / 64));
$utf .= chr(128 + ($dec % 64));
}
return $utf;
}


function html_entity_decode_utf8($string)
{
static $trans_tbl;

// replace numeric entities
$string = preg_replace('~&#x([0-9a-f]+);~ei', 'code2utf(hexdec("\\1"))', $string);
$string = preg_replace('~&#([0-9]+);~e', 'code2utf(\\1)', $string);

// replace literal entities
if (!isset($trans_tbl))
{
$trans_tbl = array();

foreach (get_html_translation_table(HTML_ENTITIES) as $val=>$key)
$trans_tbl[$key] = utf8_encode($val);
}

return strtr($string, $trans_tbl);
}

// Returns the utf string corresponding to the unicode value
function code2utf($num)
{
if ($num < 128) return chr($num);
if ($num < 2048) return chr(($num >> 6) + 192) . chr(($num & 63) + 128);
if ($num < 65536) return chr(($num >> 12) + 224) . chr((($num >> 6) & 63) + 128) . chr(($num & 63) + 128);
if ($num < 2097152) return chr(($num >> 18) + 240) . chr((($num >> 12) & 63) + 128) . chr((($num >> 6) & 63) + 128) . chr(($num & 63) + 128);
return '';
}
*/
/* -- */

?>


Ok, cool,

IT IS IMPORTANT TO NOTE THE USER BANNED FROM THE TRANSLATE IP WAS ALREADY BANNED IN ADSENSE, THIS SCRIPT DID NOT CAUSE THE BAN!

[Dave Hybrid]

I'll hang on before making it an official release, seems silly to update daily.

[NLP-er]

I'll hang on before making it an official release, seems silly to update daily.

Those who didn't made update yet will have only one...

[jaryx]

Note that some IP can suffer penalties for google and translation is terribly slow then (1 known case with domain banned in AdSense). During that our connection was passing out. So this update will reestablish connection even if goole gives you really long time to wait. It allows you to fululfill cache, so after that You will not need google anymore (only for new content).

Świetna robota bracie Polaku! Bez Ciebie ten zajebisty mod chyba szybko by upadł.
I nie muszę zmieniać serwera :rolleyes: Wcześniej pisałeś coś o dodawaniu do sitemapy vbseo i o własnym pajączku. Udostępnisz?

Good work. It is too early to issue a verdict, but unless your amendment, the entire operation takes a little longer?

[Dave Hybrid]

Those who didn't made update yet will have only one...

No offense but might be an idea to PM me your updates as releasing them in the thread might be confusing some people...

[NLP-er]

Świetna robota bracie Polaku! Bez Ciebie ten zajebisty mod chyba szybko by upadł.
I nie muszę zmieniać serwera :rolleyes: Wcześniej pisałeś coś o dodawaniu do sitemapy vbseo i o własnym pajączku. Udostępnisz?

Good work. It is too early to issue a verdict, but unless your amendment, the entire operation takes a little longer?

Dzięki :D Co do vbSEO Sitemap genratora, to w kt?rymś z post?w jest link do tego jak to zrobić - ja ciągle zmieniam swoją wersje, więc lepiej jak to zrobisz na bazowej, a pestka bo tam jest jasno opisane co gdzie wkleić. Niestety nie pamiętam linku, więc albo spytaj Dave, albo przeglądnij dyskusje. Co do robocika - ja to napisałem w javie, więc musiałbyś umieć jeszcze odpalić... Jakby co to PM.

Thanks for appreciate :)

[jaryx]

Dave.
How do I can add the translated pages to the vbseo sitemap? I can't find how. :erm:

[floggerss]

Hello, im form argentina. I dont understand the part of the translateflag.php... My forum is in spanish.. i have to put Spain.. but i dont understand if i only have to add the code at the end of the correct line. Explain more.

[Dave Hybrid]

Hello, im form argentina. I dont understand the part of the translateflag.php... My forum is in spanish.. i have to put Spain.. but i dont understand if i only have to add the code at the end of the correct line. Explain more.

You can just delete the spanish flag, either way the flag disappears and then you wont cause duplicate urls for your base language.

[Dave Hybrid]

Dave.
How do I can add the translated pages to the vbseo sitemap? I can't find how. :erm:

http://www.vbseo.com/f66/adding-some-additional-urls-my-sitemap-34734/

That is my thread at vbseo. It will add showthread pages in the languages you enter in the code.

For other pages ask in my thread, im sure the vbseo guys will help out again.

[Dave Hybrid]

$alanguages = array('ar', 'bg', 'next', 'etc');
if(!$archived)
foreach($alanguages as $hl) {
$vbseo_stat['t']++;
vbseo_add_url(vbseo_url_thread($threadrow, $p, $archived).'?hl='.$hl, $prior, $threadrow['lastpost'], $freq);
}

[Dave Hybrid]

FYI I'm just testing a new version that should sort the db error emails once and for all, thanks.

[floggerss]

I done everything but i have this error


Parse error: syntax error, unexpected $end in /home/reinodra/public_html/includes/functions.php(5721) : eval()'d code on line 59

Mi site:
www.floggerss.com.ar

[Sweeks]

I done everything but i have this error


Parse error: syntax error, unexpected $end in /home/reinodra/public_html/includes/functions.php(5721) : eval()'d code on line 59

Mi site:
www.floggerss.com.ar

It is meant to be the vbseo functions file within the vbseo folder not your forums functions.php :)

I just tried this our Dave and it has really crazy results on the amounts of new urls generated, ours is still running and it has already made 7 sitemaps from what was only two I think, is this normal? Not sure if it should be making this many lol

9 sitemaps total, changes listed are: +234,909 URLS, that is staggering!

[floggerss]

It is meant to be the vbseo functions file within the vbseo folder not your forums functions.php :)

g!

mmm And what can i do?

[Sweeks]

I would reupload the default vb functions file you edited, you should always back up before editing your files :) that should fix it providing you never altered the original before this time ;)

[floggerss]

Okei, but this only happen if i enable the plugings, if i turn it off, the forum works okei... if its on.. it shows you that error, in the header and if you want to translate the forum

[NLP-er]

Ok, cool,

IT IS IMPORTANT TO NOTE THE USER BANNED FROM THE TRANSLATE IP WAS ALREADY BANNED IN ADSENSE, THIS SCRIPT DID NOT CAUSE THE BAN!

Exactly - the men was already banned in AdSense, and punishment was also working in translations by very long response time. The lesson - don't piss of google :D

[NLP-er]

http://www.vbseo.com/f66/adding-some-additional-urls-my-sitemap-34734/

That is my thread at vbseo. It will add showthread pages in the languages you enter in the code.

For other pages ask in my thread, im sure the vbseo guys will help out again.

Can be helpfull to add this info in mod description - we have long conversaiton here and not anyone want o read it all ;)

[Dave Hybrid]

It is meant to be the vbseo functions file within the vbseo folder not your forums functions.php :)

I just tried this our Dave and it has really crazy results on the amounts of new urls generated, ours is still running and it has already made 7 sitemaps from what was only two I think, is this normal? Not sure if it should be making this many lol

9 sitemaps total, changes listed are: +234,909 URLS, that is staggering!

I said this script is powerful, i have 1 million urls in my sitemap... ;p

Normal URLs x28 :)

[Dave Hybrid]

It clearly states in the vbseo thread that you edit the vbseo_functions file, please pay attention.

If you edited the standard vb functions file than i would re-download the original from vb.com

[Saviour]

Dave...

Can you go to this page, please: http://glitchpc.com/forums/showthread.php?t=942

I have this installed and the page translates, but I get MySQL connection errors.

Can you please advise as soon as possible? Thank you.

[Dave Hybrid]

Dave...

Can you go to this page, please: http://glitchpc.com/forums/showthread.php?t=942

I have this installed and the page translates, but I get MySQL connection errors.

Can you please advise as soon as possible? Thank you.

It's a mysql config issue, send your hosting support a mail and explain the error, they should then fix it.

[Saviour]

Thanks, Dave...

Fixed! It was an issue with the $mysqlserver = "localhost"; edit.

Just had to change that to the database server for the MySQL database.

[floggerss]

It clearly states in the vbseo thread that you edit the vbseo_functions file, please pay attention.

If you edited the standard vb functions file than i would re-download the original from vb.com


I didnt edit anything, i only follow the instructions... i re upload the file.,. but the same error.

[floggerss]

wait, i use the same database... is the same?? i config that in the translate.php

[1Unreal]

Brilliant mod. Can't seem to get the flag bit to work though lol. Not sure why.

Voted 5 stars and nominated :)

[Dave Hybrid]

Brilliant mod. Can't seem to get the flag bit to work though lol. Not sure why.

Voted 5 stars and nominated :)

Thanks, if you need any help, PM me your URL and ill take a quick look.

[pericolosogr]

Every time i hit a flag to translate i get a blank page!
All steps done, i have a standard vbulletin forum without vbSeo!

Any solutions?

[Dave Hybrid]

Every time i hit a flag to translate i get a blank page!
All steps done, i have a standard vbulletin forum without vbSeo!

Any solutions?

Without an error message it is hard to say. If you dont get one try asking your host to turn on php errors on in the browser or ask them for your php error logs. Thanks.

[pericolosogr]

Nevermind i just saw the update!
It would be cool if you'd update immediately, just to save time to those who are experiencing this kind of problem!

[NLP-er]

Hello right now google translates non braking spaces to normal spaces what can damage some layouts. To change this behaviour edit vb Global Translator hook

and replace:

require_once("translate.php");
$output=callback($output);


to:

$output = str_replace("&nbsp;", "<&nbsp;>", $output);
require_once("translate.php");
$output=callback($output);
$output = str_replace("<&nbsp;>", "&nbsp;", $output);


Thanks :)

[1Unreal]

Thanks, if you need any help, PM me your URL and ill take a quick look.

It was ok I worked it out. I set the hook as global complete not global start :p

You should make it into a product to help us simpletons out :p

[Dave Hybrid]

v2.3 should be fully tested soon. False database error emails gone, finally. So far so good.

[Dave Hybrid]

It was ok I worked it out. I set the hook as global complete not global start :p

You should make it into a product to help us simpletons out :p

Agreed, it's on the to do list. Thanks.

[merkaz]

For those who has a blank page .

After 3 days working with NLP-er ..

This maybe becuase :

Firewall blocked google IP .

Curl functions disable on " disable_functions " On PHP ..
This functions must be enabled : curl_exec and curl_init

Thanks Dave ...
Thanks NLP-er ...

my best regards

[merkaz]

One more suggestion for this mod ..

for example :
In arabic , pages direction is rtl
In english its ltr

it's nice if the mod make the page direction Based on the langauge .

Sorry about my poor english :$ .

[NLP-er]

One more suggestion for this mod ..

for example :
In arabic , pages direction is rtl
In english its ltr

it's nice if the mod make the page direction Based on the langauge .

Sorry about my poor english :$ .

Right now if you want somesthing to stay not translated at all it is easiest to set 2 lines of code in vb Global Translator hook. First:

$output = str_replace("NOT_TRANSLATED_TEXT", "<NOT_TRANSLATED_TEXT>", $output);


just before:

$output=callback($output);


and:

$output = str_replace("<NOT_TRANSLATED_TEXT>", "NOT_TRANSLATED_TEXT", $output);


just after it. Of course have to change NOT_TRANSLATED_TEXT to somesthing real. Dave is working on making it product right now, so hope it will be configurable soon :)

[NLP-er]

Still making fun with cache by improving it so get new issue here :)

During spidering my site I realized, that sometimes google translation returns just empty result. And right now those empty results are included in cache, so If once filled - used forever...

To avoid this you must do 3 things.

1. Made change described under link - approptiate handling of non breaking spaces (not sure does necessary but I'm already working on correted code, and it works this way)
https://vborg.vbsupport.ru/showpost.php?p=1841391&postcount=189

2. In translate.php change:

/* Save to cache */
if ($enablecache) {


to:

/* Save to cache */
if ($enablecache && (strlen($ttext)>0)) {


So even if ocassionally google will joke with you, you will not cache it and just ask again during next page generation hoping this time result will be included and cached :)

Dave - it would be good to include it in new release :)

3. get rid of already cached dummy content by executing those queries:

delete from wt_cache where translated='' or translated is null;
delete from wt_cache_medium where translated='' or translated is null;
delete from wt_cache_short where translated='' or translated is null;



Note that I'm still working improving cache so future changes are possible :) Who don't like to mess in code - just wait till Dave made new official release with changes included :) For others - installing this will give me feedback about possible issues which I don't have on my forum.

Also - Dave I noted you was asking to wrote changes proposals as PM - I prefere this way because I have a chance that someone will use it and maybe discover some additional issues before official release.

[GoTTi]

the 2 tables medium and short are increasing, but nothing is in the translated fields of the tables....its just empty on my end. the site isnt translating. ill be uninstalling this i guess untill a later date.

[Dave Hybrid]

v2.3 Official Release

* More change to database optimization.

To Upgrade

Upload the new translate.php and run the following MySQL query on your DB.

delete from wt_cache where translated='' or translated is null;
delete from wt_cache_medium where translated='' or translated is null;
delete from wt_cache_short where translated='' or translated is null;

[Dave Hybrid]

Still making fun with cache by improving it so get new issue here :)
.......................................

Thanks for the DB update again, this one seems to improve speed even further and data size reduced, thanks a bunch! :up:

I'm still trying to sort the few remaining DB errors, it's a PITA to fix, I can get rid of them but the DB has to be opened and closed twice to avoid timeout so it makes everything slow... Grrrr. Ha. I'll keep trying. :)

[merkaz]

the 2 tables medium and short are increasing, but nothing is in the translated fields of the tables....its just empty on my end. the site isnt translating. ill be uninstalling this i guess untill a later date.

This maybe becuase :

Firewall blocked google IP .

Curl functions disable on " disable_functions " On PHP ..
This functions must be enabled : curl_exec and curl_init

[Sweeks]

v2.3 Official Release

* More change to database optimization.

To Upgrade

Upload the new translate.php and run the following MySQL query on your DB.

delete from wt_cache where translated='' or translated is null;
delete from wt_cache_medium where translated='' or translated is null;
delete from wt_cache_short where translated='' or translated is null;

Is this update including NLP's update? :up: seems a bit faster now too! thank you :)

[Dave Hybrid]

Is this update including NLP's update? :up: seems a bit faster now too! thank you :)

Yep, NLP-er tends to do the DB optimization as MySQL isn't my thing, I'm more features on other bugs.

Yeah, seems even better than before.

[Dave Hybrid]

Also; as you can see 717 pages indexed. site:www.teenforumz.com inurl:?hl=

If you want to see how many translated pages you have indexed goto google and type;

site:domain.com inurl:?hl=

Thanks.

[Sweeks]

Nice one, thanks for the query Dave :up: hope more gain indexing too :)

[GoTTi]

This maybe becuase :

Firewall blocked google IP .

Curl functions disable on " disable_functions " On PHP ..
This functions must be enabled : curl_exec and curl_init

curl is enabled. im thinking google blocked the site. i cant show adsense ads, thats where it started long time ago...but in earlier post i think on page 4 someone posted the translator dropdown menu thing to add to sites, and i put it on the site and it worked fine....just this mod here wont work with the site. i dunno..

[motorola]

What I supposed to do to,, run the following query's to import the tables. '' in PhP My admin? I'm a noob I dont understand,can someone explain :)

LE: Dont need hely anymore regarding this matter

Now I dont understand this.. :(


Now change $originalEncoding variable to match your forum base encoding. The default is English (iso-8859-1). You can find your default by viewing the source of one of your forum pages, it is at the top.

Next, change the $fl setting to also match your forum base language. The default again is english (en). If different swap with the relevent code from the table below.

[CThiessen]

Hi,
@ 2TX will you do an Update of your code for the current version 2.3?
I try to merge that on my own but I had no success.

The chages with:

<!--nttart-->
Do not translate this text
<!--ntend-->
and with the API Key.

What I did:
I put in: <if condition="(THIS_SCRIPT == 'showthread')"> for the flags

The I go to the showthread Template an add on top <!--nttart--> and at the end <!--ntend-->
The I edit the Postbit/Postbit Lagacy Template an ad before the content <!--ntend--> an after the content <!--nttart-->.

So only the "real" Content will be translated. This works very fast.
Show some strange Letters in the not translated parts but I don't care, I i thing Google don't care becuse most of the not translatetd words are not importet for the Content.

As I work with the marcoH1 Add on, I do the same tere so the Headlines will bo alsos transtated.

But this stops working with the Update 2.2

If you may post the code for the current version I realy apriciate.

Thanks
Christian

[NLP-er]

What I supposed to do to,, run the following query's to import the tables. '' in PhP My admin? I'm a noob I dont understand,can someone explain :)

LE: Dont need hely anymore regarding this matter

Now I dont understand this.. :(

Just open any page on your forum, on any browser go to page source and find:

<meta http-equiv="Content-Type" content="text/html; charset=THIS-IS-YOUR-ORIGINALENCODING" />

[GamerPerfection]

Updated.

site:galaxyofstars.co.uk inurl:?hl=
^^ 533 indexed. :)

[Dave Hybrid]

Updated.

site:galaxyofstars.co.uk inurl:?hl=
^^ 533 indexed. :)
Nice one, it's always worked pretty well on your site but now as it grows it will continue to be fast. :up:

[NLP-er]

Hello right now google translates non braking spaces to normal spaces what can damage some layouts. To change this behaviour edit vb Global Translator hook

and replace:

require_once("translate.php");
$output=callback($output);


to:

$output = str_replace("&nbsp;", "<&nbsp;>", $output);
require_once("translate.php");
$output=callback($output);
$output = str_replace("<&nbsp;>", "&nbsp;", $output);


Thanks :)

Because it was not included in official release once again I strongly reccomend to made this change and also execute those queries on DB:

delete from wt_cache_short where originaltext like '%&nbsp;%';
delete from wt_cache_medium where originaltext like '%&nbsp;%';
delete from wt_cache where originaltext like '%&nbsp;%';


Other wise it is possible that your cache will be populated by data like
&nbsp;X
&nbsp;&nbsp;X
&nbsp;&nbsp;&nbsp;X

So for same X translation you can have many data in DB. Making mentioned change makes it works faster, and DB is smaller. Also executing those queries on DB will remove all data like listed above and bellow, X will be translated again and after that work without dummy DB filling, and unnecessary querying google for translations. Also indexes should be smaller and faster because there is less common prefixes in originaltext.

Also in my forum many sites have translations like:
Posts: X &nbsp;Y

So now you have translations like i.e.
Posts: 1 &nbsp;213
Posts: 1 &nbsp;6
Posts: 2 &nbsp;213
Posts: 2 &nbsp;6
Posts: 1 &nbsp;100
Posts: 2 &nbsp;100
Posts: 3 &nbsp;100
Posts: 3 &nbsp;213
Posts: 3 &nbsp;6

After my changes you will have less translations. For mentioned examples it will be only:
Posts: 1
Posts: 2
Posts: 3
213
100
6

Once again - I advice those changes

[CThiessen]

Hi;
works pretty good:
Googel.de
Ergebnisse 1 - 10 von ungef?hr 7.530 aus brasil-web.de f?r inurl:?hl=
Googel.com
Ergebnisse 1 - 10 von ungef?hr 8.070 aus brasil-web.de f?r inurl:?hl=
And most of them are from the time before I add sitemap and "<if condition="(THIS_SCRIPT == 'showthread')">" to the page so Google follow the Links on its own.

Christian

[motorola]

Just open any page on your forum, on any browser go to page source and find:

<meta http-equiv="Content-Type" content="text/html; charset=THIS-IS-YOUR-ORIGINALENCODING" />


Ok,thanks.

And what I should add here.. from here,I dont understand,I know,noob question maybe

All the flags are on separate lines in alpha order, find your language and add <!-- to the start and --> to the end of the line.

eg; <!-- language flag code -->

<a rel="novbseo"href="<?php echo (strstr($_SERVER["VBSEO_URI"],'?hl='.@$_GET['hl'])) ? str_replace('?hl='.@$_GET['hl'], '', $_SERVER["VBSEO_URI"]) . '?hl=en' : str_replace('?hl='.@$_GET['hl'], '', $_SERVER["VBSEO_URI"]) . "?hl=en"; ?>"><img src="/flags/United States.gif" alt="English" border="0" /></a>

[NLP-er]

I notice that for some data cache is not used, translation is made again and again cache is populated. So there can be data duplication in DB. It is rare and didn't find out why yet (1 clue - working on that).

If you would like to check do you have data duplication in your cache execute those queries. Execute one by one - each one works on other cache table and tells you how many times and which data is duplicated (1st column duplication counter, 2nd for which originaltext, 3rd for which language):

select count(*) counter, originaltext, tl from wt_cache_short group by originaltext, tl having count(*) > 1 order by counter desc;

select count(*) counter, originaltext, tl from wt_cache_medium group by originaltext, tl having count(*) > 1 order by counter desc;

select count(*) counter, originaltext, tl from wt_cache group by originaltext, tl having count(*) > 1 order by counter desc;


If someone will find out some rule in those duplicated data it would be great :)


So till it will be solved you can just clear data duplications. I can share my queries to do that, but be aware that those can evaluating for some time and I had to connect by some mysql client on my comp, because by www client server was passing away before query finished it's job.

So if you want to delete data duplication first need to create 2 tables for temporary data:

CREATE TABLE saver (
id INT,
tl VARCHAR(10),
originaltext VARCHAR(65000)
) ENGINE = MYISAM, CHARACTER SET utf8 COLLATE utf8_general_ci;

CREATE TABLE cleaner (
id INT
) ENGINE = MYISAM, CHARACTER SET utf8 COLLATE utf8_general_ci;


And when you have those you can execute clearing quereies - note those will leave first translation in your database and only remove next translatioss for same text and language:

delete from cleaner;
delete from saver;
insert into saver (SELECT min(id) as id, tl, originaltext from wt_cache_short group by originaltext,tl having count(*) > 1);
insert into cleaner (SELECT id from wt_cache_short where (originaltext, tl) in (SELECT originaltext, tl from saver) and id not in (SELECT id from saver));
DELETE FROM wt_cache_short USING wt_cache_short INNER JOIN cleaner ON wt_cache_short.id = cleaner.id;

delete from cleaner;
delete from saver;
insert into saver (SELECT min(id) as id, tl, originaltext from wt_cache_medium group by originaltext,tl having count(*) > 1);
insert into cleaner (SELECT id from wt_cache_medium where (originaltext, tl) in (SELECT originaltext, tl from saver) and id not in (SELECT id from saver));
DELETE FROM wt_cache_medium USING wt_cache_medium INNER JOIN cleaner ON wt_cache_medium.id = cleaner.id;

delete from cleaner;
delete from saver;
insert into saver (SELECT min(id) as id, tl, originaltext from wt_cache group by originaltext,tl having count(*) > 1);
insert into cleaner (SELECT id from wt_cache where (originaltext, tl) in (SELECT originaltext, tl from saver) and id not in (SELECT id from saver));
DELETE FROM wt_cache USING wt_cache INNER JOIN cleaner ON wt_cache.id = cleaner.id;

[NLP-er]

Ok,thanks.

And what I should add here.. from here,I dont understand,I know,noob question maybe

All the flags are on separate lines in alpha order, find your language and add <!-- to the start and --> to the end of the line.

eg; <!-- language flag code -->

<a rel="novbseo"href="<?php echo (strstr($_SERVER["VBSEO_URI"],'?hl='.@$_GET['hl'])) ? str_replace('?hl='.@$_GET['hl'], '', $_SERVER["VBSEO_URI"]) . '?hl=en' : str_replace('?hl='.@$_GET['hl'], '', $_SERVER["VBSEO_URI"]) . "?hl=en"; ?>"><img src="/flags/United States.gif" alt="English" border="0" /></a>

Just remove whole line with your language flag. Same result.

[motorola]

My forum dont load now,I see just a blank page,what happens?

LE: I found if I upload file translateflags.php forum doesnt load anymore,if I delete the file from the server it works.

[TheLastSuperman]

Exactly, would you delete normal posts because your DB got a big big?

No, you'd keep them and upgrade the server if needed, so what's the difference... lol.

People worried about gaining too much content, that's a new one he he.

Not really, I help so many you would be surprised now think about all the ways this can sneak up on someone besides a post saying be careful of what might happen never hurts and is always rather helpful to those who are not even as far along in regards to vB as you are ;). You don't have to like what I say or agree with it based on your knowledge of vB but I have ran into many occasions where this would have made a difference and not for the positive so as long as others are aware IF they don't take the time to know then they might learn soon enough you see why I posted now?

Thanks!

S-MAN

[Dave Hybrid]

v2.3a Official Release

* Small fix to avoid translation of non braking spaces to normal spaces

To upgrade;

find in Global Translator Plugin:

require_once("translate.php");
$output=callback($output);


replace:

$output = str_replace("&nbsp;", "<&nbsp;>", $output);
require_once("translate.php");
$output=callback($output);
$output = str_replace("<&nbsp;>", "&nbsp;", $output);


Run this DB query:

delete from wt_cache_short where originaltext like '%&nbsp;%';
delete from wt_cache_medium where originaltext like '%&nbsp;%';
delete from wt_cache where originaltext like '%&nbsp;%';

Then optimize all 3 tables to remove overhead.

[LI_Pets]

Been watching this develop, so just one tweak is needed for vBseo?

How many languages will be used all 28?

Or do I chose them?

[da_judge]

Great addition, but im getting this problem

Installed yesterday... and i understand its got loads pages to translate...

If i translate home page... works fine... but if i try a forum or forum page all i get is

(French One)

Forum spécifié non valide. Si vous avez suivi un lien valide, s'il vous plaît aviser le administrateur

Translates into

Not valid specified forum. If you followed a link validates, please to notify the administrator

Is it a case of just waiting or is there a problem.... its been installed near 20 hours so far

Cheers

[Dave Hybrid]

Great addition, but im getting this problem

Installed yesterday... and i understand its got loads pages to translate...

If i translate home page... works fine... but if i try a forum or forum page all i get is

(French One)

Forum sp?cifi? non valide. Si vous avez suivi un lien valide, s'il vous pla?t aviser le administrateur

Translates into

Not valid specified forum. If you followed a link validates, please to notify the administrator


Is it a case of just waiting or is there a problem.... its been installed near 20 hours so far

Cheers

Sounds like a problem, what is your URL?

[da_judge]

I will PM you m8.... with test account

[CThiessen]

Hi,
Users don?t like to wait for an translation and I don?t know how long Google will wait for the translated page. Maybe if Google followed the first time a link an ever get the same timestamp in the sitemap it will take a while until Google crawl this site again.
So the best thing might be to put all the old sites in the database.
I used XLink to check my links but it doesn?t work with not translated pages if its nessary to wait for the Translation. The result is a time out.
So I check some other Link Checkers an I am satisfied with CyberSpyder Link Test (http://www.cyberspyder.com/cslnkts2.html).

So how to put all old pages in the databse.

You need:

a old computer, that you don?t need for a day or a week (belongs to the number of pages your form has)
CyberSpyder Link TestIn the Link Tester you are able to set up:

Max. Request per minute
Max parallel Test
waiting time ? set this up 120sec to wait for the translated pageTry out the best values for ?Max. Request per minute? and ?Max. Parallel Test? to not overload your Server.

Be passion and enjoy the result.
Christian

[jaryx]

Maybe I'd overlooked..., but I have not translated Relevant Replacements set in vBseo (H1, H2)
Any solution?

ps.
vbSEO 1.0 style

[da_judge]

Sounds like a problem, what is your URL?

All sort now thanks Dave.... i was using wrong files... VBSEO ones are for the paid version... not the free indexer on here

Great work Dave and Thanks

[Dave Hybrid]

All sort now thanks Dave.... i was using wrong files... VBSEO ones are for the paid version... not the free indexer on here

Great work Dave and Thanks

No, the files are for if you run standard urls or vbseo style urls.

You have vbseo but are still running the standard vbulleting url type.

[Dave Hybrid]

Hi,
Users don?t like to wait for an translation and I don?t know how long Google will wait for the translated page. Maybe if Google followed the first time a link an ever get the same timestamp in the sitemap it will take a while until Google crawl this site again.
So the best thing might be to put all the old sites in the database.
I used XLink to check my links but it doesn?t work with not translated pages if its nessary to wait for the Translation. The result is a time out.
So I check some other Link Checkers an I am satisfied with CyberSpyder Link Test (http://www.cyberspyder.com/cslnkts2.html).

So how to put all old pages in the databse.

You need:

a old computer, that you don?t need for a day or a week (belongs to the number of pages your form has)
CyberSpyder Link TestIn the Link Tester you are able to set up:

Max. Request per minute
Max parallel Test
waiting time ? set this up 120sec to wait for the translated pageTry out the best values for ?Max. Request per minute? and ?Max. Parallel Test? to not overload your Server.

Be passion and enjoy the result.
Christian

Google and the other search engines crawl and index the pages just fine, after the initial translation the pages load as fast as regular pages. On one site I have nearly a 100k of these pages indexed and bringing traffic. That said this could speed up the indexing a bit by pre-translating them so the load instant the first time Google visits, which isn't a bad thing. Thanks.

[valendono]

if we are upgrading from the old version.. can u give us the steps or u want us to remove everything then start from 0 ?

[Dave Hybrid]

if we are upgrading from the old version.. can u give us the steps or u want us to remove everything then start from 0 ?

What version are you upgrading from?

The second post here details from v1 to v2

Then the various v2 upgrades are scattered in the thread.

[NLP-er]

What version are you upgrading from?

The second post here details from v1 to v2

Then the various v2 upgrades are scattered in the thread.

Could be good idea to put links to those posts in mod description... This is really long conversaiton here and will be even longer :)

[Megatr0n]

Hi,

I got it working on my site but check out what it looks like when I translate to Japanese?

www.gupshupp.com

[Dave Hybrid]

Hi,

I got it working on my site but check out what it looks like when I translate to Japanese?

www.gupshupp.com

http://www.gupshupp.com/forums/?hl=ja

Looks perfectly fine to me.

[Dave Hybrid]

Hi,
Users don?t like to wait for an translation and I don?t know how long Google will wait for the translated page. Maybe if Google followed the first time a link an ever get the same timestamp in the sitemap it will take a while until Google crawl this site again.
So the best thing might be to put all the old sites in the database.
I used XLink to check my links but it doesn?t work with not translated pages if its nessary to wait for the Translation. The result is a time out.
So I check some other Link Checkers an I am satisfied with CyberSpyder Link Test (http://www.cyberspyder.com/cslnkts2.html).

So how to put all old pages in the databse.

You need:

a old computer, that you don?t need for a day or a week (belongs to the number of pages your form has)
CyberSpyder Link TestIn the Link Tester you are able to set up:

Max. Request per minute
Max parallel Test
waiting time ? set this up 120sec to wait for the translated pageTry out the best values for ?Max. Request per minute? and ?Max. Parallel Test? to not overload your Server.

Be passion and enjoy the result.
Christian

Is it me or is that software really slow, It take ages to crawl normal pages too.

[merkaz]

Please see screenprint:

i think you must have JP language in your PC ..
I think it works fine with google and people how has JP language .

[merkaz]

Hello Dave ,

" $enablesession " Not work with me now ..
Can you check on it ?

[Dave Hybrid]

Hello Dave ,

" $enablesession " Not work with me now ..
Can you check on it ?

Why are you editing that, it did not say to do so in the instructions.

[merkaz]

Why are you editing that, it did not say to do so in the instructions.

If someone translate the page then went to another one it back to original language ..

" $enablesession " save the session in the visitor language . isn't ?

[NLP-er]

I notice that for some data cache is not used, translation is made again and again cache is populated. So there can be data duplication in DB. It is rare and didn't find out why yet (1 clue - working on that).

So ok - my clue is - hazard. When 2 pages with same not translated text are generating in same time it is possible that booth will find in cache nothing, booth will translate and booth will put it to cache. It can be easily resolved just by changing INDEX to UNIQUE. It will resolve data duplication, but...

I don't want to do this because afraid of errors when 2nd translation will be inserted. Because I cannot reproduce it in browser, I don't know how it will behave. Possible behavior is that 2nd translation just get error during insertion and just go further and everything is fine for end user (spider). But other possibility is that during that error 2nd page generation will fail (i.e. white screen). Note that even in this situation it can happen only once per page, and only when we have this hazard situation.

So does someone knows what really will happen (white page, error on page, or just silent mysql error and normal generation)? :)

[NLP-er]

Ok - already found good solution for that :) So I will give you better solution today :)

[Sweeks]

It seems to have ceased working for me, I may try a reinstall, what is best Dave? Thanks :up:

[vivoperdio]

Could you please add Indonesian Translation ?

Thank you.

[NLP-er]

Hello. Just worked out for you another data base update which will made it without data duplication and also faster :)

So really on one table still you can have duplicated data, because of mysql unique limitations I didn't set it in wt_cache, but this table has less data than other updated tables.

Also collate was changed - believe it works faster when makes collation by binary values, that by some additional rules like in previous DB.

To have faster DB without data duplication You have to.

For new installations
Here is new DB:

CREATE TABLE wt_cache (
id INT AUTO_INCREMENT NOT NULL PRIMARY KEY,
tl VARCHAR(10),
originaltext VARCHAR(65000),
translated TEXT,
INDEX(originaltext(323), tl)
) ENGINE = MYISAM, CHARACTER SET utf8 COLLATE utf8_bin;

CREATE TABLE wt_cache_medium (
id INT AUTO_INCREMENT NOT NULL PRIMARY KEY,
tl VARCHAR(10),
originaltext VARCHAR(255),
translated VARCHAR(1000),
UNIQUE (originaltext, tl)
) ENGINE = MYISAM, CHARACTER SET utf8 COLLATE utf8_bin;

CREATE TABLE wt_cache_short (
id INT AUTO_INCREMENT NOT NULL PRIMARY KEY,
tl VARCHAR(10),
originaltext VARCHAR(50),
translated VARCHAR(255),
UNIQUE (originaltext, tl)
) ENGINE = MYISAM, CHARACTER SET utf8 COLLATE utf8_bin;


Also change code in translate.php
this one:

/* Check cache for translation */
if ($enablecache) {
$sql=null;
$length = strlen($text);
if ($length<=50) {
$sql = mysql_query("SELECT translated FROM wt_cache_short WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1");
} else if ($length > 255) {
$sql = mysql_query("SELECT translated FROM wt_cache WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1");
} else {
$sql = mysql_query("SELECT translated FROM wt_cache_medium WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1");
}

while($t = mysql_fetch_array($sql)) {
return $lsto.$t['translated'].$rsto;
}
}
/* -- if not found, proceed with Google Translate */


To this one:

/* Check cache for translation */
if ($enablecache) {
$sql=null;
$length = strlen($text);

if ($length<=50) {
$sql = mysql_query("SELECT translated FROM wt_cache_short WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."'") or die('Query to short cache failed: ' . mysql_error());
} else if ($length > 255) {
$sql = mysql_query("SELECT translated FROM wt_cache WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1") or die('Query to normal cache failed: ' . mysql_error());
} else {
$sql = mysql_query("SELECT translated FROM wt_cache_medium WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."'") or die('Query to medium cache failed: ' . mysql_error());
}

while($t = mysql_fetch_array($sql)) {
return $lsto.$t['translated'].$rsto;
}
}
/* -- if not found, proceed with Google Translate */


And change this one:

////////////////////////////////////////////////////
if ($enablecache) {
mysql_connect ($mysqlserver, $dbusername, $dbpassword);
mysql_select_db ($dbname);
mysql_set_charset('utf8');
}
////////////////////////////////////////////////////


To this one (less white screns):

////////////////////////////////////////////////////
if ($enablecache) {
establishConnection();
}


function establishConnection() {
global $dbusername, $dbpassword, $mysqlserver, $dbname;
mysql_connect ($mysqlserver, $dbusername, $dbpassword) or die('Could not connect: ' . mysql_error());
mysql_select_db ($dbname) or die('Could not select database');
mysql_set_charset('utf8');
}
////////////////////////////////////////////////////


And last change. This one:

/* Save to cache */
if ($enablecache && (strlen($ttext)>0)) {
$length = strlen($text);
if ($length<=50) {
mysql_query("INSERT INTO wt_cache_short SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'");
} else if ($length > 255) {
mysql_query("INSERT INTO wt_cache SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'");
} else {
mysql_query("INSERT INTO wt_cache_medium SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'");
}
}
/* -- */


To this one:

/* Save to cache */
if ($enablecache && (strlen($ttext)>0)) {
$length = strlen($text);

if (!mysql_ping()) {
mysql_close();
establishConnection();
}

if ($length<=50) {
mysql_query("INSERT IGNORE INTO wt_cache_short SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'") or die('Insert to short cache failed: ' . mysql_error());
} else if ($length > 255) {
mysql_query("INSERT INTO wt_cache SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'") or die('Insert to normal cache failed: ' . mysql_error());
} else {
mysql_query("INSERT IGNORE INTO wt_cache_medium SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'") or die('Insert to medium cache failed: ' . mysql_error());
}
}
/* -- */


So whole new translate.php is:

<?php
////////////////////////////////////////////////////
// Global Translator API
////////////////////////////////////////////////////

global $enablesession, $enablecache, $originalEncoding, $fl;
global $dbusername, $dbpassword, $mysqlserver, $dbname;

////////////////////////////////////////////////////
// SETTINGS
////////////////////////////////////////////////////
$enablesession = false; //ignore
$enablecache = true; //true - enable cache setting, false - disable cache setting
$originalEncoding = 'iso-8859-1'; //refer to your forum source code for your base encoding
$fl = 'en'; //current language (original) - refer table for language variables

$dbusername = "SET_IT";
$dbpassword = "SET_IT";
$mysqlserver = "localhost";
$dbname = "SET_IT";

////////////////////////////////////////////////////

if ($enablesession) {
@session_name("iPWTLang");
@session_start();
}


set_time_limit(0);


////////////////////////////////////////////////////
if ($enablecache) {
establishConnection();
}


function establishConnection() {
global $dbusername, $dbpassword, $mysqlserver, $dbname;
mysql_connect ($mysqlserver, $dbusername, $dbpassword) or die('Could not connect: ' . mysql_error());
mysql_select_db ($dbname) or die('Could not select database');
mysql_set_charset('utf8');
}
////////////////////////////////////////////////////

function api_strip ($dt) { $dt = str_replace("\\\\", '\\', $dt); return ($dt); }

function emodfix ($dt) { $dt = str_replace('\"', '"', $dt); return ($dt); }

function remove_tags ($html) {
/* Remove reserved triple tags in html */
return preg_replace ("|<<<([^><]*)>>>|e", "emodfix('\\1')", $html);
}

function add_tags ($buffer) {
/* Inject triple tags to html to preserve html content from translation using ob_start */
return preg_replace("/(^|>\/?)([^><]*)($|\/?<)/e",
"emodfix('\\1').'<<<'.emodfix('\\2').'>>>'.emodfix('\\3')",
$buffer);
}

function getServerLoad($windows=false)
{
//MY CODE TO DISABLE SERVER LOAD CHEcKING
return 1;
//END

$os = strtolower(PHP_OS);
if (strpos($os, "win") === false) {
if (file_exists("/proc/loadavg")) {
$data = file_get_contents("/proc/loadavg");
$load = explode(' ', $data);
return $load[0];

} elseif (function_exists("shell_exec")) {

$load = explode(' ', `uptime`);
return $load[count($load)-1];

} else {

return false;
}

} elseif($windows) {

if(class_exists("COM")) {
$wmi = new COM("WinMgmts:\\\\.");
$cpus = $wmi->InstancesOf("Win32_Processor");

$cpuload = 0;
$i = 0;

if(version_compare('4.50.0', PHP_VERSION) == 1) {
// PHP 4
while ($cpu = $cpus->Next()) {
$cpuload += $cpu->LoadPercentage;
$i++;
}

} else {

// PHP 5
foreach ( $cpus as $cpu ) {
$cpuload += $cpu->LoadPercentage;
$i++;
}
}

$cpuload = round($cpuload / $i, 2);
return "$cpuload%";

} else {

return false;
}
}
}

function translate($text, $fl, $tl){
if (trim($text) == null) return $text; //skip translation if string empty

/* Retain left and right spaces after translation */
$lsto = substr($text, 0, strlen($text) - strlen(ltrim($text)));
$rsto = substr($text, strlen(rtrim($text)), strlen($text) - strlen(rtrim($text)));
/* -- */

/* Declare global */
global $enablecache,$overloadval,$disinterval;
if(floatval(getServerLoad()) >= 1.5) // Numeral = server load on which to crash
{

// Write current time to file.
$fp = fopen("overloadcache.txt","w");
fwrite($fp, time() . "");
fclose($fp);


if($nfh= @fopen("overloadcache.txt","r"))
{
$overloadtime = fgets($nfh);
if($overloadtime > time()-30) // Numeral = # seconds after overload not to cache!
{
$enablecache = false;
}
fclose($nfh);
}
}

/* Check for server overloads (modification by DeViAnThans3)*/
global $originalEncoding;
if ($originalEncoding != 'utf-8') {
$text = iconv($originalEncoding, 'utf-8', trim($text));
} else {
$text = trim($text);
}

/* Check cache for translation */
if ($enablecache) {
$sql=null;
$length = strlen($text);
/*
if (!mysql_ping()) {
mysql_close();
establishConnection();
}
*/
if ($length<=50) {
$sql = mysql_query("SELECT translated FROM wt_cache_short WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."'") or die('Query to short cache failed: ' . mysql_error());
} else if ($length > 255) {
$sql = mysql_query("SELECT translated FROM wt_cache WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1") or die('Query to normal cache failed: ' . mysql_error());
} else {
$sql = mysql_query("SELECT translated FROM wt_cache_medium WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."'") or die('Query to medium cache failed: ' . mysql_error());
}

while($t = mysql_fetch_array($sql)) {
return $lsto.$t['translated'].$rsto;
}
}
/* -- if not found, proceed with Google Translate */

/* -Establish cURL connection to Google Translate API server- */
$ch = @curl_init();
//@curl_setopt($ch, CURLOPT_URL, "http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&langpair=".urlencode($fl.'|'.$tl)."&q=".urlencode($text));

@curl_setopt($ch, CURLOPT_URL, "http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&langpair=".urlencode($fl.'|'.$tl)."&q=".urlencode($text));

//$postParams = "v=1.0&langpair=".$fl.'|'.$tl."&q=".iconv('iso-8859-2', 'utf-8', $text);
//@curl_setopt($ch, CURLOPT_URL, "http://ajax.googleapis.com/ajax/services/language/translate");
//@curl_setopt ($Curl_Session, CURLOPT_POST, 1);
//@curl_setopt ($Curl_Session, CURLOPT_POSTFIELDS, $postParams);

@curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax)");
@curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
@curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
$in = @curl_exec ($ch);
/* -End connection- */

preg_match('/{"translatedText":"(.*?)"}/', $in, $out);
//$ttext = conv(api_strip($out[1]));
$ttext = str_replace ('\u0026', '&' , api_strip($out[1]));
//$ttext = api_strip($out[1]);
//$ttext = unicode_encode(api_strip($out[1]), 'utf-8');

/* Save to cache */
if ($enablecache && (strlen($ttext)>0)) {
$length = strlen($text);

if (!mysql_ping()) {
mysql_close();
establishConnection();
}

if ($length<=50) {
mysql_query("INSERT IGNORE INTO wt_cache_short SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'") or die('Insert to short cache failed: ' . mysql_error());
} else if ($length > 255) {
mysql_query("INSERT INTO wt_cache SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'") or die('Insert to normal cache failed: ' . mysql_error());
} else {
mysql_query("INSERT IGNORE INTO wt_cache_medium SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'") or die('Insert to medium cache failed: ' . mysql_error());
}
}
/* -- */

return $lsto.$ttext.$rsto;
}


function callback($buffer){
global $fl;
$modifyhref = false;
//-------------------------------------------------------------------------------
global $enablesession;

// Get language setting from queries:
$lang = stripslashes(@$_GET['hl']);

if ($enablesession) {
// Save setting to server session [1]
// If language setting not found in queries, check session data: [2]
if (checklang($lang)) $_SESSION['lang'] = $lang; else $lang = @$_SESSION['lang'];
}

if (!checklang($lang) || $lang == $fl) { // If invalid language setting or language = original language, skip translation.

$tp = $buffer;
$tp = preg_replace("/<disp_lang>/is", checklang($fl), $tp);
return remove_tags($tp);

} else {
// Extract BUFFER
$tp = preg_replace("/(^|>\/?)([^><]*)($|\/?<)/e",
"emodfix('\\1').translate(emodfix('\\2'),'$fl','$la ng').emodfix('\\3')",
$buffer);

if ($modifyhref) {
$tp = preg_replace('/(<a [^><]*href\=")([^"]*)("[^><]*>)/e',
"emodfix('\\1').langconv('$lang',emodfix('\\2')).em odfix('\\3')",
$tp);

$tp = preg_replace('/(<form [^><]*action\=")([^"]*)("[^><]*>)/e',
"emodfix('\\1').langconv('$lang',emodfix('\\2')).em odfix('\\3'))",
$tp);
}

$tp = preg_replace("/<disp_lang>/is", checklang($lang), $tp);
$tp = remove_tags($tp);
return $tp;

}
//---end function
}

function langconv($hl, $href){
if ($href && substr($href, 0, 7) != 'http://' && substr($href, 0, 11) != 'javascript:' && strpos($href, "#") === false) {
if(strpos($href, "?") === false) return $href.'?hl='.urlencode($hl); else return $href.'&hl='.urlencode($hl);
} else { return $href; }
}

function checklang($hl){
$langvar = array ("sq"=>"Albanian","ar"=>"Arabic","bg"=>"Bulgarian","ca"=>"Catalan","zh-CN"=>"Chinese","hr"=>"Croatian","cs"=>"Czech","da"=>"Danish","nl"=>"Dutch","en"=>"English","et"=>"Estonian","tl"=>"Filipino","fi"=>"Finnish","fr"=>"French","gl"=>"Galician","de"=>"German","el"=>"Greek","iw"=>"Hebrew","hi"=>"Hindi","hu"=>"Hungarian","id"=>"Indonesian","it"=>"Italian","ja"=>"Japanese","ko"=>"Korean","lv"=>"Latvian","lt"=>"Lithuanian","mt"=>"Maltese","no"=>"Norwegian","pl"=>"Polish","pt"=>"Portuguese","ro"=>"Romanian","ru"=>"Russian","sr"=>"Serbian","sk"=>"Slovak","sl"=>"Slovenian","es"=>"Spanish","sv"=>"Swedish","zh-TW"=>"Taiwanese","th"=>"Thai","tr"=>"Turkish","uk"=>"Ukrainian","vi"=>"Vietnamese");
foreach ($langvar as $i => $val){ if ($i == $hl) return $val; }
return false;
}

function clearlangsetting(){
global $enablesession;
if ($enablesession) $_SESSION['lang'] = null; /* -Clear lang session data on server- */
}

?>


Hope Dave will include it in next release.

Now - for already installed products
Here we have 2 ways. Easy one, and good one.

In easy one - just drop all tables from DB, set it from new script and change translate.php like described.

In good one - you want to keep already cached data, and this will need some changes.
1. change translate.php like described
2. dissable cache - set $enablecache=false; in translate.php
3. change collate by executing:

Alter table wt_cache_short collate utf8_bin;
ALTER TABLE wt_cache_short CHANGE tl tl VARCHAR( 10 ) CHARACTER SET utf8 COLLATE utf8_bin NULL DEFAULT NULL;
ALTER TABLE wt_cache_short CHANGE originaltext originaltext VARCHAR( 50 ) CHARACTER SET utf8 COLLATE utf8_bin NULL DEFAULT NULL;
ALTER TABLE wt_cache_short CHANGE translated translated VARCHAR( 255 ) CHARACTER SET utf8 COLLATE utf8_bin NULL DEFAULT NULL;

Alter table wt_cache_medium collate utf8_bin;
ALTER TABLE wt_cache_medium CHANGE tl tl VARCHAR( 10 ) CHARACTER SET utf8 COLLATE utf8_bin NULL DEFAULT NULL;
ALTER TABLE wt_cache_medium CHANGE originaltext originaltext VARCHAR( 255 ) CHARACTER SET utf8 COLLATE utf8_bin NULL DEFAULT NULL;
ALTER TABLE wt_cache_medium CHANGE translated translated VARCHAR( 1000 ) CHARACTER SET utf8 COLLATE utf8_bin NULL DEFAULT NULL;

Alter table wt_cache collate utf8_bin;
ALTER TABLE wt_cache CHANGE tl tl VARCHAR( 10 ) CHARACTER SET utf8 COLLATE utf8_bin NULL DEFAULT NULL;
ALTER TABLE wt_cache CHANGE translated translated TEXT CHARACTER SET utf8 COLLATE utf8_bin NULL DEFAULT NULL;
ALTER TABLE wt_cache CHANGE translated translated VARCHAR( 65000 ) CHARACTER SET utf8 COLLATE utf8_bin NULL DEFAULT NULL;


4. Delete data duplication like described here (note this is very important - otherwise unique indexes will not be set): https://vborg.vbsupport.ru/showpost.php?p=1844479&postcount=282

Note that this step if time consuming and it is a chance that you will need to connect to DB with other client that WWW (just check it first). Remember - always you can use the easy way ;)

5. Optimize tables:

OPTIMIZE TABLE wt_cache, wt_cache_medium, wt_cache_short;


6. Reindex to unique:

alter table wt_cache_short drop index originaltext;
create UNIQUE INDEX originaltext on wt_cache_short (originaltext, tl);

alter table wt_cache_medium drop index originaltext;
create UNIQUE INDEX originaltext on wt_cache_medium (originaltext, tl);


7. enable cache - set $enablecache=true; in translate.php


And that is. As I wrote data duplication will disappear for ever from wt_cache_short and wt_cache_medium. Also it will be save in case of hazard described in my 2 previous posts. Also according to my automatic tests - page generation works faster.

So - enjoy :D

[CThiessen]

Is it me or is that software really slow, It take ages to crawl normal pages too.
Hello,
Yes it is slow. But store all results in a local Database. So you are able to stop and continue the test at any time. I used Xenu to check Links but that one considers the translated pages an even some Links to social networks as broken because they do not answered in time.
This one I set up on my Computer at work and leave it running over the weekend. I do set it up to waiting time 120 (90 should be enough) an 30 workers. To stop a test and run a report even takes some time. But it?s OK for me as I don?t need to start from the beginning, I going to continue the test every COB until the next morning.

Christian

[masterweb]

Great hack!, i downloaded it 2 days ago due i prefer test a mod before replying with my feedbacks and what i saw is:

After installing it on my Demo board and following the instructions carefully (at first instructions seems complicated but even if is a long procedure if you follow each steep as mentioned it works without problems), i uploaded the flag's folder on root and made the necessary changes to the style...at the end it works without problems and it translated some pages on a decent timeframe.

Then i installed it on my real board and as mentioned above following each steep present on the instructions it works fine, congratulations dude and thanks for this release, now i have just a question:

The main language of my board is the English (i deleted the EN line on the translate.php) but there are sections on Spanish and Italian too so, must i delete also these lines on the translate.php?. Thanks once again.-

[Dave Hybrid]

It seems to have ceased working for me, I may try a reinstall, what is best Dave? Thanks :up:

What is the problem, looks ok to me...

[Dave Hybrid]

Hello. Just worked out for you another data base update which will made it without data duplication and also faster :)

So really on one table still you can have duplicated data, because of mysql unique limitations I didn't set it in wt_cache, but this table has less data than other updated tables.

Also collate was changed - believe it works faster when makes collation by binary values, that by some additional rules like in previous DB.

To have faster DB without data duplication You have to.

For new installations
Here is new DB:

CREATE TABLE wt_cache (
id INT AUTO_INCREMENT NOT NULL PRIMARY KEY,
tl VARCHAR(10),
originaltext VARCHAR(65000),
translated TEXT,
INDEX(originaltext(50), tl)
) ENGINE = MYISAM, CHARACTER SET utf8 COLLATE utf8_bin;

CREATE TABLE wt_cache_medium (
id INT AUTO_INCREMENT NOT NULL PRIMARY KEY,
tl VARCHAR(10),
originaltext VARCHAR(255),
translated VARCHAR(1000),
UNIQUE (originaltext, tl)
) ENGINE = MYISAM, CHARACTER SET utf8 COLLATE utf8_bin;

CREATE TABLE wt_cache_short (
id INT AUTO_INCREMENT NOT NULL PRIMARY KEY,
tl VARCHAR(10),
originaltext VARCHAR(50),
translated VARCHAR(255),
UNIQUE (originaltext, tl)
) ENGINE = MYISAM, CHARACTER SET utf8 COLLATE utf8_bin;


Also change code in translate.php
this one:

/* Check cache for translation */
if ($enablecache) {
$sql=null;
$length = strlen($text);
if ($length<=50) {
$sql = mysql_query("SELECT translated FROM wt_cache_short WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1");
} else if ($length > 255) {
$sql = mysql_query("SELECT translated FROM wt_cache WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1");
} else {
$sql = mysql_query("SELECT translated FROM wt_cache_medium WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1");
}

while($t = mysql_fetch_array($sql)) {
return $lsto.$t['translated'].$rsto;
}
}
/* -- if not found, proceed with Google Translate */


To this one:

/* Check cache for translation */
if ($enablecache) {
$sql=null;
$length = strlen($text);

if ($length<=50) {
$sql = mysql_query("SELECT translated FROM wt_cache_short WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."'") or die('Query to short cache failed: ' . mysql_error());
} else if ($length > 255) {
$sql = mysql_query("SELECT translated FROM wt_cache WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1") or die('Query to normal cache failed: ' . mysql_error());
} else {
$sql = mysql_query("SELECT translated FROM wt_cache_medium WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."'") or die('Query to medium cache failed: ' . mysql_error());
}

while($t = mysql_fetch_array($sql)) {
return $lsto.$t['translated'].$rsto;
}
}
/* -- if not found, proceed with Google Translate */


And change this one:

////////////////////////////////////////////////////
if ($enablecache) {
mysql_connect ($mysqlserver, $dbusername, $dbpassword);
mysql_select_db ($dbname);
mysql_set_charset('utf8');
}
////////////////////////////////////////////////////


To this one (less white screns):

////////////////////////////////////////////////////
if ($enablecache) {
establishConnection();
}


function establishConnection() {
global $dbusername, $dbpassword, $mysqlserver, $dbname;
mysql_connect ($mysqlserver, $dbusername, $dbpassword) or die('Could not connect: ' . mysql_error());
mysql_select_db ($dbname) or die('Could not select database');
mysql_set_charset('utf8');
}
////////////////////////////////////////////////////


And last change. This one:

/* Save to cache */
if ($enablecache && (strlen($ttext)>0)) {
$length = strlen($text);
if ($length<=50) {
mysql_query("INSERT INTO wt_cache_short SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'");
} else if ($length > 255) {
mysql_query("INSERT INTO wt_cache SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'");
} else {
mysql_query("INSERT INTO wt_cache_medium SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'");
}
}
/* -- */


To this one:

/* Save to cache */
if ($enablecache && (strlen($ttext)>0)) {
$length = strlen($text);

if (!mysql_ping()) {
mysql_close();
establishConnection();
}

if ($length<=50) {
mysql_query("INSERT IGNORE INTO wt_cache_short SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'") or die('Insert to short cache failed: ' . mysql_error());
} else if ($length > 255) {
mysql_query("INSERT INTO wt_cache SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'") or die('Insert to normal cache failed: ' . mysql_error());
} else {
mysql_query("INSERT IGNORE INTO wt_cache_medium SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'") or die('Insert to medium cache failed: ' . mysql_error());
}
}
/* -- */


So whole new translate.php is:

<?php
////////////////////////////////////////////////////
// Global Translator API
////////////////////////////////////////////////////

global $enablesession, $enablecache, $originalEncoding, $fl;
global $dbusername, $dbpassword, $mysqlserver, $dbname;

////////////////////////////////////////////////////
// SETTINGS
////////////////////////////////////////////////////
$enablesession = false; //ignore
$enablecache = true; //true - enable cache setting, false - disable cache setting
$originalEncoding = 'iso-8859-1'; //refer to your forum source code for your base encoding
$fl = 'en'; //current language (original) - refer table for language variables

$dbusername = "SET_IT";
$dbpassword = "SET_IT";
$mysqlserver = "localhost";
$dbname = "SET_IT";

////////////////////////////////////////////////////

if ($enablesession) {
@session_name("iPWTLang");
@session_start();
}


set_time_limit(0);


////////////////////////////////////////////////////
if ($enablecache) {
establishConnection();
}


function establishConnection() {
global $dbusername, $dbpassword, $mysqlserver, $dbname;
mysql_connect ($mysqlserver, $dbusername, $dbpassword) or die('Could not connect: ' . mysql_error());
mysql_select_db ($dbname) or die('Could not select database');
mysql_set_charset('utf8');
}
////////////////////////////////////////////////////

function api_strip ($dt) { $dt = str_replace("\\\\", '\\', $dt); return ($dt); }

function emodfix ($dt) { $dt = str_replace('\"', '"', $dt); return ($dt); }

function remove_tags ($html) {
/* Remove reserved triple tags in html */
return preg_replace ("|<<<([^><]*)>>>|e", "emodfix('\\1')", $html);
}

function add_tags ($buffer) {
/* Inject triple tags to html to preserve html content from translation using ob_start */
return preg_replace("/(^|>\/?)([^><]*)($|\/?<)/e",
"emodfix('\\1').'<<<'.emodfix('\\2').'>>>'.emodfix('\\3')",
$buffer);
}

function getServerLoad($windows=false)
{
//MY CODE TO DISABLE SERVER LOAD CHEcKING
return 1;
//END

$os = strtolower(PHP_OS);
if (strpos($os, "win") === false) {
if (file_exists("/proc/loadavg")) {
$data = file_get_contents("/proc/loadavg");
$load = explode(' ', $data);
return $load[0];

} elseif (function_exists("shell_exec")) {

$load = explode(' ', `uptime`);
return $load[count($load)-1];

} else {

return false;
}

} elseif($windows) {

if(class_exists("COM")) {
$wmi = new COM("WinMgmts:\\\\.");
$cpus = $wmi->InstancesOf("Win32_Processor");

$cpuload = 0;
$i = 0;

if(version_compare('4.50.0', PHP_VERSION) == 1) {
// PHP 4
while ($cpu = $cpus->Next()) {
$cpuload += $cpu->LoadPercentage;
$i++;
}

} else {

// PHP 5
foreach ( $cpus as $cpu ) {
$cpuload += $cpu->LoadPercentage;
$i++;
}
}

$cpuload = round($cpuload / $i, 2);
return "$cpuload%";

} else {

return false;
}
}
}

function translate($text, $fl, $tl){
if (trim($text) == null) return $text; //skip translation if string empty

/* Retain left and right spaces after translation */
$lsto = substr($text, 0, strlen($text) - strlen(ltrim($text)));
$rsto = substr($text, strlen(rtrim($text)), strlen($text) - strlen(rtrim($text)));
/* -- */

/* Declare global */
global $enablecache,$overloadval,$disinterval;
if(floatval(getServerLoad()) >= 1.5) // Numeral = server load on which to crash
{

// Write current time to file.
$fp = fopen("overloadcache.txt","w");
fwrite($fp, time() . "");
fclose($fp);


if($nfh= @fopen("overloadcache.txt","r"))
{
$overloadtime = fgets($nfh);
if($overloadtime > time()-30) // Numeral = # seconds after overload not to cache!
{
$enablecache = false;
}
fclose($nfh);
}
}

/* Check for server overloads (modification by DeViAnThans3)*/
global $originalEncoding;
if ($originalEncoding != 'utf-8') {
$text = iconv($originalEncoding, 'utf-8', trim($text));
} else {
$text = trim($text);
}

/* Check cache for translation */
if ($enablecache) {
$sql=null;
$length = strlen($text);
/*
if (!mysql_ping()) {
mysql_close();
establishConnection();
}
*/
if ($length<=50) {
$sql = mysql_query("SELECT translated FROM wt_cache_short WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."'") or die('Query to short cache failed: ' . mysql_error());
} else if ($length > 255) {
$sql = mysql_query("SELECT translated FROM wt_cache WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1") or die('Query to normal cache failed: ' . mysql_error());
} else {
$sql = mysql_query("SELECT translated FROM wt_cache_medium WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."'") or die('Query to medium cache failed: ' . mysql_error());
}

while($t = mysql_fetch_array($sql)) {
return $lsto.$t['translated'].$rsto;
}
}
/* -- if not found, proceed with Google Translate */

/* -Establish cURL connection to Google Translate API server- */
$ch = @curl_init();
//@curl_setopt($ch, CURLOPT_URL, "http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&langpair=".urlencode($fl.'|'.$tl)."&q=".urlencode($text));

@curl_setopt($ch, CURLOPT_URL, "http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&langpair=".urlencode($fl.'|'.$tl)."&q=".urlencode($text));

//$postParams = "v=1.0&langpair=".$fl.'|'.$tl."&q=".iconv('iso-8859-2', 'utf-8', $text);
//@curl_setopt($ch, CURLOPT_URL, "http://ajax.googleapis.com/ajax/services/language/translate");
//@curl_setopt ($Curl_Session, CURLOPT_POST, 1);
//@curl_setopt ($Curl_Session, CURLOPT_POSTFIELDS, $postParams);

@curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax)");
@curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
@curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
$in = @curl_exec ($ch);
/* -End connection- */

preg_match('/{"translatedText":"(.*?)"}/', $in, $out);
//$ttext = conv(api_strip($out[1]));
$ttext = str_replace ('\u0026', '&' , api_strip($out[1]));
//$ttext = api_strip($out[1]);
//$ttext = unicode_encode(api_strip($out[1]), 'utf-8');

/* Save to cache */
if ($enablecache && (strlen($ttext)>0)) {
$length = strlen($text);

if (!mysql_ping()) {
mysql_close();
establishConnection();
}

if ($length<=50) {
mysql_query("INSERT IGNORE INTO wt_cache_short SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'") or die('Insert to short cache failed: ' . mysql_error());
} else if ($length > 255) {
mysql_query("INSERT INTO wt_cache SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'") or die('Insert to normal cache failed: ' . mysql_error());
} else {
mysql_query("INSERT IGNORE INTO wt_cache_medium SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'") or die('Insert to medium cache failed: ' . mysql_error());
}
}
/* -- */

return $lsto.$ttext.$rsto;
}


function callback($buffer){
global $fl;
$modifyhref = false;
//-------------------------------------------------------------------------------
global $enablesession;

// Get language setting from queries:
$lang = stripslashes(@$_GET['hl']);

if ($enablesession) {
// Save setting to server session [1]
// If language setting not found in queries, check session data: [2]
if (checklang($lang)) $_SESSION['lang'] = $lang; else $lang = @$_SESSION['lang'];
}

if (!checklang($lang) || $lang == $fl) { // If invalid language setting or language = original language, skip translation.

$tp = $buffer;
$tp = preg_replace("/<disp_lang>/is", checklang($fl), $tp);
return remove_tags($tp);

} else {
// Extract BUFFER
$tp = preg_replace("/(^|>\/?)([^><]*)($|\/?<)/e",
"emodfix('\\1').translate(emodfix('\\2'),'$fl','$la ng').emodfix('\\3')",
$buffer);

if ($modifyhref) {
$tp = preg_replace('/(<a [^><]*href\=")([^"]*)("[^><]*>)/e',
"emodfix('\\1').langconv('$lang',emodfix('\\2')).em odfix('\\3')",
$tp);

$tp = preg_replace('/(<form [^><]*action\=")([^"]*)("[^><]*>)/e',
"emodfix('\\1').langconv('$lang',emodfix('\\2')).em odfix('\\3'))",
$tp);
}

$tp = preg_replace("/<disp_lang>/is", checklang($lang), $tp);
$tp = remove_tags($tp);
return $tp;

}
//---end function
}

function langconv($hl, $href){
if ($href && substr($href, 0, 7) != 'http://' && substr($href, 0, 11) != 'javascript:' && strpos($href, "#") === false) {
if(strpos($href, "?") === false) return $href.'?hl='.urlencode($hl); else return $href.'&hl='.urlencode($hl);
} else { return $href; }
}

function checklang($hl){
$langvar = array ("sq"=>"Albanian","ar"=>"Arabic","bg"=>"Bulgarian","ca"=>"Catalan","zh-CN"=>"Chinese","hr"=>"Croatian","cs"=>"Czech","da"=>"Danish","nl"=>"Dutch","en"=>"English","et"=>"Estonian","tl"=>"Filipino","fi"=>"Finnish","fr"=>"French","gl"=>"Galician","de"=>"German","el"=>"Greek","iw"=>"Hebrew","hi"=>"Hindi","hu"=>"Hungarian","id"=>"Indonesian","it"=>"Italian","ja"=>"Japanese","ko"=>"Korean","lv"=>"Latvian","lt"=>"Lithuanian","mt"=>"Maltese","no"=>"Norwegian","pl"=>"Polish","pt"=>"Portuguese","ro"=>"Romanian","ru"=>"Russian","sr"=>"Serbian","sk"=>"Slovak","sl"=>"Slovenian","es"=>"Spanish","sv"=>"Swedish","zh-TW"=>"Taiwanese","th"=>"Thai","tr"=>"Turkish","uk"=>"Ukrainian","vi"=>"Vietnamese");
foreach ($langvar as $i => $val){ if ($i == $hl) return $val; }
return false;
}

function clearlangsetting(){
global $enablesession;
if ($enablesession) $_SESSION['lang'] = null; /* -Clear lang session data on server- */
}

?>


Hope Dave will include it in next release.

Now - for already installed products
Here we have 2 ways. Easy one, and good one.

In easy one - just drop all tables from DB, set it from new script and change translate.php like described.

In good one - you want to keep already cached data, and this will need some changes.
1. change translate.php like described
2. change collate by executing:

Alter table wt_cache_short collate utf8_bin;
ALTER TABLE wt_cache_short CHANGE tl tl VARCHAR( 10 ) CHARACTER SET utf8 COLLATE utf8_bin NULL DEFAULT NULL;
ALTER TABLE wt_cache_short CHANGE originaltext originaltext VARCHAR( 50 ) CHARACTER SET utf8 COLLATE utf8_bin NULL DEFAULT NULL;
ALTER TABLE wt_cache_short CHANGE translated translated VARCHAR( 255 ) CHARACTER SET utf8 COLLATE utf8_bin NULL DEFAULT NULL;

Alter table wt_cache_medium collate utf8_bin;
ALTER TABLE wt_cache_medium CHANGE tl tl VARCHAR( 10 ) CHARACTER SET utf8 COLLATE utf8_bin NULL DEFAULT NULL;
ALTER TABLE wt_cache_medium CHANGE originaltext originaltext VARCHAR( 255 ) CHARACTER SET utf8 COLLATE utf8_bin NULL DEFAULT NULL;
ALTER TABLE wt_cache_medium CHANGE translated translated VARCHAR( 1000 ) CHARACTER SET utf8 COLLATE utf8_bin NULL DEFAULT NULL;

Alter table wt_cache collate utf8_bin;
ALTER TABLE wt_cache CHANGE tl tl VARCHAR( 10 ) CHARACTER SET utf8 COLLATE utf8_bin NULL DEFAULT NULL;
ALTER TABLE wt_cache CHANGE translated translated TEXT CHARACTER SET utf8 COLLATE utf8_bin NULL DEFAULT NULL;
ALTER TABLE wt_cache CHANGE translated translated VARCHAR( 65000 ) CHARACTER SET utf8 COLLATE utf8_bin NULL DEFAULT NULL;


3. Delete data duplication like described here (note this is very important - otherwise unique indexes will not be set): https://vborg.vbsupport.ru/showpost.php?p=1842327&postcount=214

Note that this step if very time consuming and probably you will need to connect to DB with other client that WWW. Remember - always you can use the easy way ;)

4. Optimize tables:

OPTIMIZE TABLE wt_cache, wt_cache_medium, wt_cache_short;


5. Reindex to unique:

alter table wt_cache_short drop index originaltext;
create UNIQUE INDEX originaltext on wt_cache_short (originaltext, tl);

alter table wt_cache_medium drop index originaltext;
create UNIQUE INDEX originaltext on wt_cache_medium (originaltext, tl);


And that is. As I wrote data duplication will disappear for ever from wt_cache_short and wt_cache_medium. Also it will be save in case of hazard described in my 2 previous posts. Also according to my automatic tests - page generation works faster.

So - enjoy :D

Error

SQL query:

CREATE UNIQUE INDEX originaltext ON wt_cache_medium(
originaltext,
tl
);

MySQL said: Documentation
#1062 - Duplicate entry 'I even learned a few bits, the number range in particular will c' for key 2

***********

Edit

Seems ok now, just run the query again. Would still appreciate an explanation of the error, thanks mate.

[Dave Hybrid]

I don't know what you did but it's slow, really slow, unbearable. Something isnt right with your changes.

[Dave Hybrid]

In your version is causes crazy high server load, and cache false doesnt seem to work anymore.

[NLP-er]

I don't know what you did but it's slow, really slow, unbearable. Something isnt right with your changes.
Of course - you just wrote that you didn't made UNIQUE INDEX - so you just came back to time which you had in your very first release.

Once again:
Delete data duplication like described here (note this is very important - otherwise unique indexes will not be set)

And you had data duplication:
#1062 - Duplicate entry

So you don't have index now - it have to be slow :p

My changes are ok and make it works faster - just have to make all steps.

If you had error with data duplication - drop duplicated data. If you did it and have same problem - drop data duplication again - it just means that in the meantime between dropping data duplication and trying to create UNIQUE INDEX, some new data came with duplications (google doesn't sleep).

Before you will drop duplicated data again it is wise to set normal index again:

create INDEX originaltext on wt_cache_short (originaltext, tl);


then drop data duplication and reindex to unique as described in previous post :)

As I wrote - you always have the easy way. You want keep your cache - made everything as described and don't complain that with my update is something wrong if you didn't make it.

[NLP-er]

In your version is causes crazy high server load, and cache false doesnt seem to work anymore.
Time consuming is deletion of data duplication and also reindexing. It was mentioned in description - mysql have some work and it is only during the changes - script is working faster. Also always have the easy way :p

$enablecache works fine - possible that mysql still works on some of your queries from deleting data duplication and slows down the system.

But it is good idea - disable cache before deleting data duplication and enable it again after all changes :) You will not have problems with data duplication in the meantime. I will add those 2 points to update description