vb.org Archive
Page 4 of 18 « First 23 4 5614 Last »
Show 40 post(s) from this thread on one page

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   Modification Graveyard (https://vborg.vbsupport.ru/forumdisplay.php?f=224)
-   -   Miscellaneous Hacks - vB Global Translator - Multiply your indexed pages & put search traffic on autopilot (https://vborg.vbsupport.ru/showthread.php?t=217329)

NLP-er 06-30-2009 02:54 PM
Quote:
Originally Posted by GoTTi (Post 1840177)
ok i left it installed overnight and i checked the mysql tables and this is what it looks like

https://vborg.vbsupport.ru/attachmen...1&d=1246373739

still not loading anything in the cache table. and when i click any flags, nothing is loading for me. any ideas?
You can always set $enablecache = false; in translate.php - then you will know does your page translates at all. I'm quite sure that cache is not an issue here and we can be sure by temporary disabling it.

NLP-er 06-30-2009 02:58 PM
Quote:
Originally Posted by Geraldm (Post 1840203)
Well I installed the new version 2.0 script yesterday.

My forum currently has 4,745 threads and 17,052 posts and today my DB looks like this:

http://www.delphi-php.net/images/records24.jpg

34.7MB in 24 hours ?!!?!? I'll let you know how big this DB gets after a week.....

Cheers, and thanks for the great script!

Regards,
Gerald.
Well - in vb you have 1 content. Here it is multiplied for each translation. As long as you have space on server - who cares?... ;)

Dave Hybrid 06-30-2009 05:09 PM
Exactly, would you delete normal posts because your DB got a big big?

No, you'd keep them and upgrade the server if needed, so what's the difference... lol.

People worried about gaining too much content, that's a new one he he.

T2x 06-30-2009 05:16 PM
Quote:
Originally Posted by tpearl5 (Post 1840175)
You certainly can't be hurting anything by including an API key.

Here's another question... How about displaying flags based on where the IP is from? I understand this would involve a small index of IP's, but perhaps it could be an exclusion. For example, 'if not from US, display flags'

edit - nevermind - that would be a very large index either way that would have to be referenced on every visit, which sounds very expensive

Nah its actually very easy to do. I have an ip database on my site which performs a similar function.

I would recommend using javascript instead to read the browser language.

Code:
var nav_lng="nolang"; //default language string
if(navigator.language)nav_lng=navigator.language;
else if(navigator.browserLanguage)nav_lng=navigator.browserLanguage;
else if(navigator.userLanguage)nav_lng=navigator.userLanguage;
else if(navigator.systemLanguage)nav_lng=navigator.systemLanguage;
else if(window.navigator.language)nav_lng=window.navigator.language;
for example, for english visitors the nav_lng variable will then contain either en-US or en-GB(for users from the UK or AU)


I actually have dynamic javascript translation going on as well right now using this and it works fine.

burlesque 06-30-2009 05:55 PM
I was also getting the blank page when I clicked on a flag, however when I set the caching to 'false' it translated fine.

However, when I add set the cache to 'true' I get the white page again.

But if I set the cache to false once again, everything translates fine, until I set the cache to 'true'...

.. and then I get a blank page again.

The db has been installed correctly but I'm at a loss as to why this happens.

music legend 06-30-2009 06:54 PM
Quote:
Originally Posted by burlesque (Post 1840287)
I was also getting the blank page when I clicked on a flag, however when I set the caching to 'false' it translated fine.

However, when I add set the cache to 'true' I get the white page again.

But if I set the cache to false once again, everything translates fine, until I set the cache to 'true'...

.. and then I get a blank page again.

The db has been installed correctly but I'm at a loss as to why this happens.
where is the caching option you turned off?

NLP-er 06-30-2009 07:01 PM
Quote:
Originally Posted by Dave Hybrid (Post 1840265)
Exactly, would you delete normal posts because your DB got a big big?

No, you'd keep them and upgrade the server if needed, so what's the difference... lol.

People worried about gaining too much content, that's a new one he he.
:D exactly I'm very happy when new posts arrives on my forum - more is better, or people will go somewhere where they will find what they are looking for...

burlesque 06-30-2009 07:01 PM
in translate.php

Code:
$enablesession = false; //ignore
$enablecache = false; //true - enable cache setting, false - disable cache setting

NLP-er 06-30-2009 07:07 PM
Quote:
Originally Posted by burlesque (Post 1840287)
I was also getting the blank page when I clicked on a flag, however when I set the caching to 'false' it translated fine.

However, when I add set the cache to 'true' I get the white page again.

But if I set the cache to false once again, everything translates fine, until I set the cache to 'true'...

.. and then I get a blank page again.

The db has been installed correctly but I'm at a loss as to why this happens.
In such case cache is an issue here. Can you PM to me some more details? I will gladly see what is gooing on :)

music legend 06-30-2009 07:12 PM
yeah that fixed my white screen as well when i turned it to false it worked

if its set to true i just get the white screen

burlesque 06-30-2009 07:19 PM
Quote:
Originally Posted by NLP-er (Post 1840328)
In such case cache is an issue here. Can you PM to me some more details? I will gladly see what is gooing on :)
PM Sent :)

music legend 06-30-2009 07:52 PM
Quote:
Originally Posted by burlesque (Post 1840334)
PM Sent :)
let me know if yall figure out the problem because im sure its the same one i am having

Dave Hybrid 06-30-2009 08:25 PM
v2.1

* Meta description and meta keywords now translated in the output source code, better for SEO and search engine indexing
* Note no change to files

To update just replace your vB Global Translator plugin code with what is below.

Also change the Execution Order: to 50

Code:
// Enable UTF-8 characters
if(isset($_GET['hl'])) {
$output = str_replace('lang="en"', 'lang="'.$_GET['hl'].'"', $output);
 header ('Content-type: text/html; charset=utf-8');
// Keep remainders of <style and <script tags!
    $time = time(); // Unique Tag Identifier!
    preg_match_all('|<script[^>]*>(.*?)</script>|si', $output,$scripttags, PREG_SET_ORDER);
    for($i=0;$i<count($scripttags);$i++)
    {
        $output = str_replace($scripttags[$i][0], "<a name=\"$i\"></a>",$output);
    }     
    //preg_match_all("<style[^>]*>.*</style>",$buffer,$styletags);
    preg_match_all('|<style[^>]*>(.*?)</style>|si',$output,$styletags);
    for($i=0;$i<count($styletags);$i++)
    {
        $output = str_replace($styletags[1][$i], "<a name=\"s$i\"></a>",$output);
    }
 require_once("translate.php");
$output=callback($output);
// Place back remainders of <style and <script tags!
    for($i=0;$i<count($styletags);$i++)
    {
        $output = str_replace("<a name=\"s$i\"></a>", $styletags[1][$i], $output);
    }
    for($i=0;$i<count($scripttags);$i++)
    {
        $output = str_replace("<a name=\"$i\"></a>",$scripttags[$i][0] ,$output);
    }

// Translate META tags.
preg_match_all("|<meta[^>]+name=\"([^\"]*)\"[^>]" . "+content=\"([^\"]*)\"[^>]+>|i",$output, $out,PREG_PATTERN_ORDER);
$meta = array();
for ($i=0;$i < count($out[1]);$i++) {
        if (strtolower($out[1][$i]) == "keywords") $meta['keywords'] = $out[2][$i];
        if (strtolower($out[1][$i]) == "description") $meta['description'] = $out[2][$i];
    }

// Start replacing the original META tags with translated tags.
$output=str_replace("<meta name=\"description\" content=\"".$meta['description']."\" />","<meta name=\"description\" content=\"".html_entity_decode(callback($meta['description']))."\" />",$output);
$output=str_replace("<meta name=\"keywords\" content=\"".$meta['keywords']."\" />","<meta name=\"keywords\" content=\"".html_entity_decode(callback($meta['keywords']))."\" />",$output);
}

Sweeks 06-30-2009 08:37 PM
Nice one this mod just keeps getting better :up:

Dave Hybrid 06-30-2009 08:39 PM
Thank you, more to come. ;P

Dave Hybrid 06-30-2009 08:40 PM
Quote:
Originally Posted by NLP-er (Post 1840328)
In such case cache is an issue here. Can you PM to me some more details? I will gladly see what is gooing on :)
Thanks for helping out. :up:

Sweeks 06-30-2009 08:42 PM
Maybe throw in that API key option just to put everyone at peace if it doesnt have a performance hit :)

Geraldm 06-30-2009 09:44 PM
Hi just tried v2.1 and it's not translating the meta keywords or meta Description as described??

Dave, you know my forum as I've PMed you it in the past ... can you please take a look? :)

Thanks,
Gerald.

Dave Hybrid 06-30-2009 09:51 PM
Quote:
Originally Posted by Geraldm (Post 1840406)
Hi just tried v2.1 and it's not translating the meta keywords or meta Description as described??

Dave, you know my forum as I've PMed you it in the past ... can you please take a look? :)

Thanks,
Gerald.
You pasted the new plugin code and changed the Execution order?

PM me the url again please.

Dave Hybrid 06-30-2009 09:58 PM
Quote:
Originally Posted by Geraldm (Post 1840406)
Hi just tried v2.1 and it's not translating the meta keywords or meta Description as described??

Dave, you know my forum as I've PMed you it in the past ... can you please take a look? :)

Thanks,
Gerald.
I managed to find your old PM, it looks fine to me mate.

Geraldm 06-30-2009 10:24 PM
Quote:
Originally Posted by Dave Hybrid (Post 1840410)
I managed to find your old PM, it looks fine to me mate.
Ahh it was only translating the keywords and not the description ...... I re-read your instructions and noticed I missed the bit about changing the execution order :) I've now changed it to 50 and yes it's now translating the keywords and desc.... Great stuff! :)

Thank you!

Dave Hybrid 06-30-2009 10:36 PM
Great, you're welcome. :up:

GoTTi 06-30-2009 11:16 PM
Quote:
Originally Posted by NLP-er (Post 1840219)
You can always set $enablecache = false; in translate.php - then you will know does your page translates at all. I'm quite sure that cache is not an issue here and we can be sure by temporary disabling it.
its doing the same thing. i put false on the .php and reuploaded, reloaded the pages, and clicked a flag and all its doing is loading, then it stops.

GoTTi 06-30-2009 11:35 PM
i just installed a testboard, clean, no mods on it and only installed this mod here, made a post, clicked a flag and nothing is happening. what could possibly be causing this to not work?

NLP-er 06-30-2009 11:57 PM
Since some installations still have problems using cache and we are working on that, for those who has running vb global translator I have some optimizations :)

1. Query optimization - change in template.php code between /* Check cache for translation */ and /* -- if not found, proceed with Google Translate */ to:
Code:
/* Check cache for translation */
if ($enablecache) {
  $sql=null;
  $length = strlen($text);
  if ($length<=50) {
    $sql = mysql_query("SELECT translated FROM wt_cache_short WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1"); 
  } else if ($length > 255) {
    $sql = mysql_query("SELECT translated FROM wt_cache WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1");
  } else {
    $sql = mysql_query("SELECT translated FROM wt_cache_medium WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1");
  }
 
        while($t = mysql_fetch_array($sql)) {
                return $lsto.$t['translated'].$rsto;
        }
}
/* -- if not found, proceed with Google Translate */
Unnecessary data was taken from DB. Hope Dave will make new release quickly and include this change :)

2. Index changes. Since for someone DB was growing fast I made some tests. It is enough to change indexes.

For installed products run those queries:
Code:
alter table wt_cache_short drop index tl;
create index originaltext on wt_cache_short (originaltext, tl);

alter table wt_cache_medium drop index tl;
create index originaltext on wt_cache_medium (originaltext, tl);

alter table wt_cache drop index tl;
create index originaltext on wt_cache (originaltext(50), tl);
For new installations it will be:
Code:
CREATE TABLE wt_cache (
id INT AUTO_INCREMENT NOT NULL PRIMARY KEY,
tl VARCHAR(10),
originaltext VARCHAR(65000),
translated TEXT,
INDEX (originaltext (50), tl)
) ENGINE = MYISAM, CHARACTER SET utf8 COLLATE utf8_general_ci;

CREATE TABLE wt_cache_medium (
id INT AUTO_INCREMENT NOT NULL PRIMARY KEY,
tl VARCHAR(10),
originaltext VARCHAR(255),
translated VARCHAR(1000),
INDEX (originaltext, tl)
) ENGINE = MYISAM, CHARACTER SET utf8 COLLATE utf8_general_ci;

CREATE TABLE wt_cache_short (
id INT AUTO_INCREMENT NOT NULL PRIMARY KEY,
tl VARCHAR(10),
originaltext VARCHAR(50),
translated VARCHAR(255),
INDEX (originaltext, tl)
) ENGINE = MYISAM, CHARACTER SET utf8 COLLATE utf8_general_ci;
Hope Dave will change installation manual to this one.

I realized that column tl has only 28 possible values (or less if you don’t use all flags) and that is not too good for indexes. Only by changing order from (tl, originaltext) to (originaltext, tl) my database size changed from 51MB to 44,3MB without any performance lost and even with improvement. Changing it only to (originaltext) have worst results that (originaltext, tl).

Here are some results of my tests (I have weak internet connection so don't look on times but on differences between performance in different configurations):
WITH index (tl,originaltext) DB size=51MB TESTS RESULTS:
MIN: 4484 MAX: 13266 AVG: 7823
MIN: 3188 MAX: 9703 AVG: 5848
MIN: 3797 MAX: 9594 AVG: 5683
TOTAL AVG: 6451

WITH index (originaltext,tl) DB size=44,3MB TESTS RESULTS:
MIN: 2859 MAX: 10172 AVG: 6232
MIN: 2890 MAX: 9094 AVG: 6109
MIN: 3468 MAX: 9500 AVG: 5813
TOTAL AVG: 6051

WITH index (originaltext) DB size=43,2MB TESTS RESULTS:
MIN: 4454 MAX: 10406 AVG: 5998
MIN: 3703 MAX: 13922 AVG: 6341
MIN: 2922 MAX: 14578 AVG: 6888
TOTAL AVG: 6409

Times are in ms. In each test 20 different cached translated pages was generated. For each setting 3 tests series at row was executed.

NLP-er 06-30-2009 11:58 PM
For those who have problems with using cache at all - right now I suspect that there is problem with making connection to DB. Even when all settings are ok - It seems that script is waiting for connection and DB holds it. Is someone with this problem has a clue what to change (maybe some number of connections limit) It would be good if share solution with others. As long as I'm not able to reproduce this on my side, or someone will give me access to his forum, I'm not even sure does this is a problem - and it is only a suspicion.

NLP-er 07-01-2009 12:11 AM
Quote:
Originally Posted by GoTTi (Post 1840444)
i just installed a testboard, clean, no mods on it and only installed this mod here, made a post, clicked a flag and nothing is happening. what could possibly be causing this to not work?
If it is clean, I think you could PM me access data to your adminCP and FTP. Then I can check what is going on :) Right now I can only tell that somesthing wrong ;)

alqloob alsahya 07-01-2009 12:14 AM
nice

Excellent subject of very beautiful

Thank you

GoTTi 07-01-2009 03:30 AM
my site is blocked from google adsense, would that matter in this situation?

GoTTi 07-01-2009 03:33 AM
Quote:
Originally Posted by NLP-er (Post 1840466)
If it is clean, I think you could PM me access data to your adminCP and FTP. Then I can check what is going on :) Right now I can only tell that somesthing wrong ;)
ive sent u a PM with the login.

Dave Hybrid 07-01-2009 09:50 AM
v2.2

* More improvements to database code, better speed and lower data size
* Flag files combined size reduced from 40kb to 3kb for faster page load speed

To upgrade download new package, upload new translate.php and /flags/ folder.

Run the following MySQL query on your DB via phpmyadmin.

Code:
alter table wt_cache_short drop index tl;
create index originaltext on wt_cache_short (originaltext, tl);

alter table wt_cache_medium drop index tl;
create index originaltext on wt_cache_medium (originaltext, tl);

alter table wt_cache drop index tl;
create index originaltext on wt_cache (originaltext(50), tl);
Thanks!

GamerPerfection 07-01-2009 10:28 AM
I been on holiday since 21st Jun.
You mean I have to do this all over again?

Sweeks 07-01-2009 11:37 AM
Quote:
Originally Posted by Dave Hybrid (Post 1840647)
v2.2

* More improvements to database code, better speed and lower data size
* Flag files combined size reduced from 40kb to 3kb for faster page load speed

To upgrade download new package, upload new translate.php and /flags/ folder.

Run the following MySQL query on your DB via phpmyadmin.

Code:
alter table wt_cache_short drop index tl;
create index originaltext on wt_cache_short (originaltext, tl);

alter table wt_cache_medium drop index tl;
create index originaltext on wt_cache_medium (originaltext, tl);

alter table wt_cache drop index tl;
create index originaltext on wt_cache (originaltext(50), tl);
Thanks!
Yet another excellent update, much smoother now Dave :up:

Dave Hybrid 07-01-2009 12:26 PM
Quote:
Originally Posted by clarkey25 (Post 1840655)
I been on holiday since 21st Jun.
You mean I have to do this all over again?
If you want to run the better version yes, it's called progression. ;P

GamerPerfection 07-01-2009 01:46 PM
Installed.

I noticed the overloadcache file is no longer in the zip file, can I just delete the old file that is still on my server?

NLP-er 07-01-2009 01:48 PM
Since some users complains that when cache enabled white screan apperas I made solution for that. Testes on one server with problem, if doesn't solve yours it will give at least some more information now.

Note that some IP can suffer penalties for google and translation is terribly slow then (1 known case with domain banned in AdSense). During that our connection was passing out. So this update will reestablish connection even if goole gives you really long time to wait. It allows you to fululfill cache, so after that You will not need google anymore (only for new content).

NOTE: I know also one case when google give other penalty that time to wait. The penalty was - empty response to all queries. In such case It is between you and google ;)

Change translate.php to this one and set up DB and other params.

Dave - It would be good to have it in official release :)

Code:
<?php
////////////////////////////////////////////////////
// Global Translator API
////////////////////////////////////////////////////

global $enablesession, $enablecache, $originalEncoding, $fl;
global $dbusername, $dbpassword, $mysqlserver, $dbname;

////////////////////////////////////////////////////
//        SETTINGS
////////////////////////////////////////////////////
$enablesession = false; //ignore
$enablecache = true; //true - enable cache setting, false - disable cache setting
$originalEncoding = 'iso-8859-1'; //refer to your forum source code for your base encoding
$fl = 'en'; //current language (original) - refer table for language varibles

$dbusername = "SET IT";
$dbpassword = "SET IT";
$mysqlserver = "localhost";
$dbname = "SET IT";

////////////////////////////////////////////////////

if ($enablesession) {
        @session_name("iPWTLang");
        @session_start();
}


set_time_limit(0);


////////////////////////////////////////////////////
if ($enablecache) {
  establishConnection();
}


function establishConnection() {
  global $dbusername, $dbpassword, $mysqlserver, $dbname;
  mysql_connect ($mysqlserver, $dbusername, $dbpassword) or die('Could not connect: ' . mysql_error());
  mysql_select_db ($dbname) or die('Could not select database');
  mysql_set_charset('utf8');
}
 ////////////////////////////////////////////////////

function api_strip ($dt) { $dt = str_replace("\\\\", '\\', $dt); return ($dt); }

function emodfix ($dt) { $dt = str_replace('\"', '"', $dt); return ($dt); }

function remove_tags ($html) {
/* Remove reserved triple tags in html */
return preg_replace ("|<<<([^><]*)>>>|e", "emodfix('\\1')", $html);
}

function add_tags ($buffer) {
/* Inject triple tags to html to preserve html content from translation using ob_start */
return preg_replace("/(^|>\/?)([^><]*)($|\/?<)/e",
            "emodfix('\\1').'<<<'.emodfix('\\2').'>>>'.emodfix('\\3')",
            $buffer);
}

function getServerLoad($windows=false)
{
//MY CODE TO DISABLE SERVER LOAD CHEcKING
  return 1;
//END

    $os = strtolower(PHP_OS);
    if (strpos($os, "win") === false) {
        if (file_exists("/proc/loadavg")) {
            $data = file_get_contents("/proc/loadavg");
            $load = explode(' ', $data);
            return $load[0];

        } elseif (function_exists("shell_exec")) {

            $load = explode(' ', `uptime`);
            return $load[count($load)-1];

        } else {

            return false;
        }

    } elseif($windows) {

        if(class_exists("COM")) {
            $wmi = new COM("WinMgmts:\\\\.");
            $cpus = $wmi->InstancesOf("Win32_Processor");

            $cpuload = 0;
            $i = 0;

            if(version_compare('4.50.0', PHP_VERSION) == 1) {
                // PHP 4
                while ($cpu = $cpus->Next()) {
                    $cpuload += $cpu->LoadPercentage;
                    $i++;
                }

            } else {

                // PHP 5
                foreach ( $cpus as $cpu ) {
                    $cpuload += $cpu->LoadPercentage;
                    $i++;
                }
            }

            $cpuload = round($cpuload / $i, 2);
            return "$cpuload%";

        } else {

            return false;
        }
    }
}

function translate($text, $fl, $tl){
if (trim($text) == null) return $text; //skip translation if string empty

/* Retain left and right spaces after translation */
$lsto = substr($text, 0, strlen($text) - strlen(ltrim($text)));
$rsto = substr($text, strlen(rtrim($text)), strlen($text) - strlen(rtrim($text)));
/* -- */

/* Declare global */
global $enablecache,$overloadval,$disinterval;
if(floatval(getServerLoad()) >= 1.5)    // Numeral = server load on which to crash
{

    // Write current time to file.
    $fp = fopen("overloadcache.txt","w");
    fwrite($fp, time() . "");
    fclose($fp);


  if($nfh= @fopen("overloadcache.txt","r"))
  {
      $overloadtime = fgets($nfh);
      if($overloadtime > time()-30)  // Numeral = # seconds after overload not to cache!
      {
          $enablecache = false;
      }
      fclose($nfh);
  }
}

/* Check for server overloads  (modification by DeViAnThans3)*/
global $originalEncoding;
if ($originalEncoding != 'utf-8') {
  $text = iconv($originalEncoding, 'utf-8', trim($text));
} else {
  $text = trim($text);
}

/* Check cache for translation */
if ($enablecache) {
  $sql=null;
  $length = strlen($text);
/*
  if (!mysql_ping()) {
    mysql_close();
    establishConnection();
  }
*/ 
  if ($length<=50) {
    $sql = mysql_query("SELECT translated FROM wt_cache_short WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1") or die('Query to short cache failed: ' . mysql_error());
  } else if ($length > 255) {
    $sql = mysql_query("SELECT translated FROM wt_cache WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1") or die('Query to normal cache failed: ' . mysql_error());
  } else {
    $sql = mysql_query("SELECT translated FROM wt_cache_medium WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1") or die('Query to medium cache failed: ' . mysql_error());
  }
 
        while($t = mysql_fetch_array($sql)) {
                return $lsto.$t['translated'].$rsto;
        }
}
/* -- if not found, proceed with Google Translate */

/* -Establish cURL connection to Google Translate API server- */
$ch = @curl_init();
//@curl_setopt($ch, CURLOPT_URL, "http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&langpair=".urlencode($fl.'|'.$tl)."&q=".urlencode($text));

@curl_setopt($ch, CURLOPT_URL, "http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&langpair=".urlencode($fl.'|'.$tl)."&q=".urlencode($text));

//$postParams = "v=1.0&langpair=".$fl.'|'.$tl."&q=".iconv('iso-8859-2', 'utf-8', $text);
//@curl_setopt($ch, CURLOPT_URL, "http://ajax.googleapis.com/ajax/services/language/translate");
//@curl_setopt ($Curl_Session, CURLOPT_POST, 1);
//@curl_setopt ($Curl_Session, CURLOPT_POSTFIELDS, $postParams);

@curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax)");
@curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
@curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
$in = @curl_exec ($ch);
/* -End connection- */

preg_match('/{"translatedText":"(.*?)"}/', $in, $out);
//$ttext = conv(api_strip($out[1]));
$ttext = str_replace ('\u0026', '&' , api_strip($out[1]));
//$ttext = api_strip($out[1]);
//$ttext = unicode_encode(api_strip($out[1]), 'utf-8');

/* Save to cache */
if ($enablecache) {
  $length = strlen($text);

  if (!mysql_ping()) {
    mysql_close();
    establishConnection();
  }
 
  if ($length<=50) {
    mysql_query("INSERT INTO wt_cache_short SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'") or die('Insert to short cache failed: ' . mysql_error());
  } else if ($length > 255) {
    mysql_query("INSERT INTO wt_cache SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'") or die('Insert to normal cache failed: ' . mysql_error());
  } else {
    mysql_query("INSERT INTO wt_cache_medium SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'") or die('Insert to medium cache failed: ' . mysql_error());
  }
}
/* -- */

return $lsto.$ttext.$rsto;
}


function callback($buffer){
global $fl;
$modifyhref = false;
//-------------------------------------------------------------------------------
global $enablesession;

// Get language setting from queries:
$lang = stripslashes(@$_GET['hl']);

if ($enablesession) {
        // Save setting to server session [1]
        // If language setting not found in queries, check session data: [2]
        if (checklang($lang)) $_SESSION['lang'] = $lang; else $lang = @$_SESSION['lang'];
}

if (!checklang($lang) || $lang == $fl) { // If invalid language setting or language = original language, skip translation.

$tp = $buffer;
$tp = preg_replace("/<disp_lang>/is", checklang($fl), $tp);
return remove_tags($tp);

} else {
    // Extract BUFFER
$tp = preg_replace("/(^|>\/?)([^><]*)($|\/?<)/e",
            "emodfix('\\1').translate(emodfix('\\2'),'$fl','$lang').emodfix('\\3')",
            $buffer);

if ($modifyhref) {
$tp = preg_replace('/(<a [^><]*href\=")([^"]*)("[^><]*>)/e',
            "emodfix('\\1').langconv('$lang',emodfix('\\2')).emodfix('\\3')",
            $tp);

$tp = preg_replace('/(<form [^><]*action\=")([^"]*)("[^><]*>)/e',
            "emodfix('\\1').langconv('$lang',emodfix('\\2')).emodfix('\\3'))",
            $tp);
}

$tp = preg_replace("/<disp_lang>/is", checklang($lang), $tp);
    $tp = remove_tags($tp);
return $tp;

}
//---end function
}

function langconv($hl, $href){
if ($href && substr($href, 0, 7) != 'http://' && substr($href, 0, 11) != 'javascript:' && strpos($href, "#") === false) {
        if(strpos($href, "?") === false) return $href.'?hl='.urlencode($hl); else return $href.'&hl='.urlencode($hl);
} else { return $href; }
}

function checklang($hl){
$langvar = array ("sq"=>"Albanian","ar"=>"Arabic","bg"=>"Bulgarian","ca"=>"Catalan","zh-CN"=>"Chinese","hr"=>"Croatian","cs"=>"Czech","da"=>"Danish","nl"=>"Dutch","en"=>"English","et"=>"Estonian","tl"=>"Filipino","fi"=>"Finnish","fr"=>"French","gl"=>"Galician","de"=>"German","el"=>"Greek","iw"=>"Hebrew","hi"=>"Hindi","hu"=>"Hungarian","id"=>"Indonesian","it"=>"Italian","ja"=>"Japanese","ko"=>"Korean","lv"=>"Latvian","lt"=>"Lithuanian","mt"=>"Maltese","no"=>"Norwegian","pl"=>"Polish","pt"=>"Portuguese","ro"=>"Romanian","ru"=>"Russian","sr"=>"Serbian","sk"=>"Slovak","sl"=>"Slovenian","es"=>"Spanish","sv"=>"Swedish","zh-TW"=>"Taiwanese","th"=>"Thai","tr"=>"Turkish","uk"=>"Ukrainian","vi"=>"Vietnamese");
foreach ($langvar as $i => $val){ if ($i == $hl) return $val; }
return false;
}

function clearlangsetting(){
global $enablesession;
if ($enablesession) $_SESSION['lang'] = null; /* -Clear lang session data on server- */
}

/* Unicode UTF-8 Support Libraries */
/*
function conv($str) {
$str = preg_replace("/\\\u([a-zA-Z0-9]{4})/e",
        "unichr('\\1')",
        $str);
return html_entity_decode_utf8($str);
//return $str;
}


function unichr($hex) {
$dec = hexdec($hex);

  if ($dec < 128) {
    $utf = chr($dec);
  } else if ($dec < 2048) {
    $utf = chr(192 + (($dec - ($dec % 64)) / 64));
    $utf .= chr(128 + ($dec % 64));
  } else {
    $utf = chr(224 + (($dec - ($dec % 4096)) / 4096));
    $utf .= chr(128 + ((($dec % 4096) - ($dec % 64)) / 64));
    $utf .= chr(128 + ($dec % 64));
  }
  return $utf;
}


function html_entity_decode_utf8($string)
{
    static $trans_tbl;

    // replace numeric entities
    $string = preg_replace('~&#x([0-9a-f]+);~ei', 'code2utf(hexdec("\\1"))', $string);
    $string = preg_replace('~&#([0-9]+);~e', 'code2utf(\\1)', $string);

    // replace literal entities
    if (!isset($trans_tbl))
    {
        $trans_tbl = array();

        foreach (get_html_translation_table(HTML_ENTITIES) as $val=>$key)
            $trans_tbl[$key] = utf8_encode($val);
    }

    return strtr($string, $trans_tbl);
}

// Returns the utf string corresponding to the unicode value
function code2utf($num)
{
    if ($num < 128) return chr($num);
    if ($num < 2048) return chr(($num >> 6) + 192) . chr(($num & 63) + 128);
    if ($num < 65536) return chr(($num >> 12) + 224) . chr((($num >> 6) & 63) + 128) . chr(($num & 63) + 128);
    if ($num < 2097152) return chr(($num >> 18) + 240) . chr((($num >> 12) & 63) + 128) . chr((($num >> 6) & 63) + 128) . chr(($num & 63) + 128);
    return '';
}
*/
/* -- */

?>

Dave Hybrid 07-01-2009 01:51 PM
Quote:
Originally Posted by clarkey25 (Post 1840742)
Installed.

I noticed the overloadcache file is no longer in the zip file, can I just delete the old file that is still on my server?
You need to delete everything and fresh install, that included all files, flags and a brand new database and plugins. Everything.

Dave Hybrid 07-01-2009 01:53 PM
Quote:
Originally Posted by NLP-er (Post 1840744)
Since some users complains that when cache enabled white screan apperas I made solution for that. Testes on one server with problem, if doesn't solve yours it will give at least some more information now.

Note that some IP can suffer penalties for google and translation is terribly slow then (1 known case with domain banned in AdSense). During that our connection was passing out. So this update will reestablish connection even if goole gives you really long time to wait. It allows you to fululfill cache, so after that You will not need google anymore (only for new content).

NOTE: I know also one case when google give other penalty that time to wait. The penalty was - empty response to all queries. In such case It is between you and google ;)

Change translate.php to this one and set up DB and other params.

Dave - It would be good to have it in official release :)

Code:
<?php
////////////////////////////////////////////////////
// Global Translator API
////////////////////////////////////////////////////

global $enablesession, $enablecache, $originalEncoding, $fl;
global $dbusername, $dbpassword, $mysqlserver, $dbname;

////////////////////////////////////////////////////
//        SETTINGS
////////////////////////////////////////////////////
$enablesession = false; //ignore
$enablecache = true; //true - enable cache setting, false - disable cache setting
$originalEncoding = 'iso-8859-1'; - //refer to your forum source code for your base encoding
$fl = 'en'; //current language (original) - refer table for language varibles

$dbusername = "SET IT";
$dbpassword = "SET IT";
$mysqlserver = "localhost";
$dbname = "SET IT";

////////////////////////////////////////////////////

if ($enablesession) {
        @session_name("iPWTLang");
        @session_start();
}


set_time_limit(0);


////////////////////////////////////////////////////
if ($enablecache) {
  establishConnection();
}


function establishConnection() {
  global $dbusername, $dbpassword, $mysqlserver, $dbname;
  mysql_connect ($mysqlserver, $dbusername, $dbpassword) or die('Could not connect: ' . mysql_error());
  mysql_select_db ($dbname) or die('Could not select database');
  mysql_set_charset('utf8');
}
 ////////////////////////////////////////////////////

function api_strip ($dt) { $dt = str_replace("\\\\", '\\', $dt); return ($dt); }

function emodfix ($dt) { $dt = str_replace('\"', '"', $dt); return ($dt); }

function remove_tags ($html) {
/* Remove reserved triple tags in html */
return preg_replace ("|<<<([^><]*)>>>|e", "emodfix('\\1')", $html);
}

function add_tags ($buffer) {
/* Inject triple tags to html to preserve html content from translation using ob_start */
return preg_replace("/(^|>\/?)([^><]*)($|\/?<)/e",
            "emodfix('\\1').'<<<'.emodfix('\\2').'>>>'.emodfix('\\3')",
            $buffer);
}

function getServerLoad($windows=false)
{
//MY CODE TO DISABLE SERVER LOAD CHEcKING
  return 1;
//END

    $os = strtolower(PHP_OS);
    if (strpos($os, "win") === false) {
        if (file_exists("/proc/loadavg")) {
            $data = file_get_contents("/proc/loadavg");
            $load = explode(' ', $data);
            return $load[0];

        } elseif (function_exists("shell_exec")) {

            $load = explode(' ', `uptime`);
            return $load[count($load)-1];

        } else {

            return false;
        }

    } elseif($windows) {

        if(class_exists("COM")) {
            $wmi = new COM("WinMgmts:\\\\.");
            $cpus = $wmi->InstancesOf("Win32_Processor");

            $cpuload = 0;
            $i = 0;

            if(version_compare('4.50.0', PHP_VERSION) == 1) {
                // PHP 4
                while ($cpu = $cpus->Next()) {
                    $cpuload += $cpu->LoadPercentage;
                    $i++;
                }

            } else {

                // PHP 5
                foreach ( $cpus as $cpu ) {
                    $cpuload += $cpu->LoadPercentage;
                    $i++;
                }
            }

            $cpuload = round($cpuload / $i, 2);
            return "$cpuload%";

        } else {

            return false;
        }
    }
}

function translate($text, $fl, $tl){
if (trim($text) == null) return $text; //skip translation if string empty

/* Retain left and right spaces after translation */
$lsto = substr($text, 0, strlen($text) - strlen(ltrim($text)));
$rsto = substr($text, strlen(rtrim($text)), strlen($text) - strlen(rtrim($text)));
/* -- */

/* Declare global */
global $enablecache,$overloadval,$disinterval;
if(floatval(getServerLoad()) >= 1.5)    // Numeral = server load on which to crash
{

    // Write current time to file.
    $fp = fopen("overloadcache.txt","w");
    fwrite($fp, time() . "");
    fclose($fp);


  if($nfh= @fopen("overloadcache.txt","r"))
  {
      $overloadtime = fgets($nfh);
      if($overloadtime > time()-30)  // Numeral = # seconds after overload not to cache!
      {
          $enablecache = false;
      }
      fclose($nfh);
  }
}

/* Check for server overloads  (modification by DeViAnThans3)*/
global $originalEncoding;
if ($originalEncoding != 'utf-8') {
  $text = iconv($originalEncoding, 'utf-8', trim($text));
} else {
  $text = trim($text);
}

/* Check cache for translation */
if ($enablecache) {
  $sql=null;
  $length = strlen($text);
/*
  if (!mysql_ping()) {
    mysql_close();
    establishConnection();
  }
*/ 
  if ($length<=50) {
    $sql = mysql_query("SELECT translated FROM wt_cache_short WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1") or die('Query to short cache failed: ' . mysql_error());
  } else if ($length > 255) {
    $sql = mysql_query("SELECT translated FROM wt_cache WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1") or die('Query to normal cache failed: ' . mysql_error());
  } else {
    $sql = mysql_query("SELECT translated FROM wt_cache_medium WHERE originaltext='".addslashes($text)."' AND tl='".addslashes($tl)."' LIMIT 1") or die('Query to medium cache failed: ' . mysql_error());
  }
 
        while($t = mysql_fetch_array($sql)) {
                return $lsto.$t['translated'].$rsto;
        }
}
/* -- if not found, proceed with Google Translate */

/* -Establish cURL connection to Google Translate API server- */
$ch = @curl_init();
//@curl_setopt($ch, CURLOPT_URL, "http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&langpair=".urlencode($fl.'|'.$tl)."&q=".urlencode($text));

@curl_setopt($ch, CURLOPT_URL, "http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&langpair=".urlencode($fl.'|'.$tl)."&q=".urlencode($text));

//$postParams = "v=1.0&langpair=".$fl.'|'.$tl."&q=".iconv('iso-8859-2', 'utf-8', $text);
//@curl_setopt($ch, CURLOPT_URL, "http://ajax.googleapis.com/ajax/services/language/translate");
//@curl_setopt ($Curl_Session, CURLOPT_POST, 1);
//@curl_setopt ($Curl_Session, CURLOPT_POSTFIELDS, $postParams);

@curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax)");
@curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
@curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
$in = @curl_exec ($ch);
/* -End connection- */

preg_match('/{"translatedText":"(.*?)"}/', $in, $out);
//$ttext = conv(api_strip($out[1]));
$ttext = str_replace ('\u0026', '&' , api_strip($out[1]));
//$ttext = api_strip($out[1]);
//$ttext = unicode_encode(api_strip($out[1]), 'utf-8');

/* Save to cache */
if ($enablecache) {
  $length = strlen($text);

  if (!mysql_ping()) {
    mysql_close();
    establishConnection();
  }
 
  if ($length<=50) {
    mysql_query("INSERT INTO wt_cache_short SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'") or die('Insert to short cache failed: ' . mysql_error());
  } else if ($length > 255) {
    mysql_query("INSERT INTO wt_cache SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'") or die('Insert to normal cache failed: ' . mysql_error());
  } else {
    mysql_query("INSERT INTO wt_cache_medium SET tl='".addslashes($tl)."', originaltext='".addslashes($text)."', translated='".addslashes(trim($ttext))."'") or die('Insert to medium cache failed: ' . mysql_error());
  }
}
/* -- */

return $lsto.$ttext.$rsto;
}


function callback($buffer){
global $fl;
$modifyhref = false;
//-------------------------------------------------------------------------------
global $enablesession;

// Get language setting from queries:
$lang = stripslashes(@$_GET['hl']);

if ($enablesession) {
        // Save setting to server session [1]
        // If language setting not found in queries, check session data: [2]
        if (checklang($lang)) $_SESSION['lang'] = $lang; else $lang = @$_SESSION['lang'];
}

if (!checklang($lang) || $lang == $fl) { // If invalid language setting or language = original language, skip translation.

$tp = $buffer;
$tp = preg_replace("/<disp_lang>/is", checklang($fl), $tp);
return remove_tags($tp);

} else {
    // Extract BUFFER
$tp = preg_replace("/(^|>\/?)([^><]*)($|\/?<)/e",
            "emodfix('\\1').translate(emodfix('\\2'),'$fl','$lang').emodfix('\\3')",
            $buffer);

if ($modifyhref) {
$tp = preg_replace('/(<a [^><]*href\=")([^"]*)("[^><]*>)/e',
            "emodfix('\\1').langconv('$lang',emodfix('\\2')).emodfix('\\3')",
            $tp);

$tp = preg_replace('/(<form [^><]*action\=")([^"]*)("[^><]*>)/e',
            "emodfix('\\1').langconv('$lang',emodfix('\\2')).emodfix('\\3'))",
            $tp);
}

$tp = preg_replace("/<disp_lang>/is", checklang($lang), $tp);
    $tp = remove_tags($tp);
return $tp;

}
//---end function
}

function langconv($hl, $href){
if ($href && substr($href, 0, 7) != 'http://' && substr($href, 0, 11) != 'javascript:' && strpos($href, "#") === false) {
        if(strpos($href, "?") === false) return $href.'?hl='.urlencode($hl); else return $href.'&hl='.urlencode($hl);
} else { return $href; }
}

function checklang($hl){
$langvar = array ("sq"=>"Albanian","ar"=>"Arabic","bg"=>"Bulgarian","ca"=>"Catalan","zh-CN"=>"Chinese","hr"=>"Croatian","cs"=>"Czech","da"=>"Danish","nl"=>"Dutch","en"=>"English","et"=>"Estonian","tl"=>"Filipino","fi"=>"Finnish","fr"=>"French","gl"=>"Galician","de"=>"German","el"=>"Greek","iw"=>"Hebrew","hi"=>"Hindi","hu"=>"Hungarian","id"=>"Indonesian","it"=>"Italian","ja"=>"Japanese","ko"=>"Korean","lv"=>"Latvian","lt"=>"Lithuanian","mt"=>"Maltese","no"=>"Norwegian","pl"=>"Polish","pt"=>"Portuguese","ro"=>"Romanian","ru"=>"Russian","sr"=>"Serbian","sk"=>"Slovak","sl"=>"Slovenian","es"=>"Spanish","sv"=>"Swedish","zh-TW"=>"Taiwanese","th"=>"Thai","tr"=>"Turkish","uk"=>"Ukrainian","vi"=>"Vietnamese");
foreach ($langvar as $i => $val){ if ($i == $hl) return $val; }
return false;
}

function clearlangsetting(){
global $enablesession;
if ($enablesession) $_SESSION['lang'] = null; /* -Clear lang session data on server- */
}

/* Unicode UTF-8 Support Libraries */
/*
function conv($str) {
$str = preg_replace("/\\\u([a-zA-Z0-9]{4})/e",
        "unichr('\\1')",
        $str);
return html_entity_decode_utf8($str);
//return $str;
}


function unichr($hex) {
$dec = hexdec($hex);

  if ($dec < 128) {
    $utf = chr($dec);
  } else if ($dec < 2048) {
    $utf = chr(192 + (($dec - ($dec % 64)) / 64));
    $utf .= chr(128 + ($dec % 64));
  } else {
    $utf = chr(224 + (($dec - ($dec % 4096)) / 4096));
    $utf .= chr(128 + ((($dec % 4096) - ($dec % 64)) / 64));
    $utf .= chr(128 + ($dec % 64));
  }
  return $utf;
}


function html_entity_decode_utf8($string)
{
    static $trans_tbl;

    // replace numeric entities
    $string = preg_replace('~&#x([0-9a-f]+);~ei', 'code2utf(hexdec("\\1"))', $string);
    $string = preg_replace('~&#([0-9]+);~e', 'code2utf(\\1)', $string);

    // replace literal entities
    if (!isset($trans_tbl))
    {
        $trans_tbl = array();

        foreach (get_html_translation_table(HTML_ENTITIES) as $val=>$key)
            $trans_tbl[$key] = utf8_encode($val);
    }

    return strtr($string, $trans_tbl);
}

// Returns the utf string corresponding to the unicode value
function code2utf($num)
{
    if ($num < 128) return chr($num);
    if ($num < 2048) return chr(($num >> 6) + 192) . chr(($num & 63) + 128);
    if ($num < 65536) return chr(($num >> 12) + 224) . chr((($num >> 6) & 63) + 128) . chr(($num & 63) + 128);
    if ($num < 2097152) return chr(($num >> 18) + 240) . chr((($num >> 12) & 63) + 128) . chr((($num >> 6) & 63) + 128) . chr(($num & 63) + 128);
    return '';
}
*/
/* -- */

?>
Ok, cool,

IT IS IMPORTANT TO NOTE THE USER BANNED FROM THE TRANSLATE IP WAS ALREADY BANNED IN ADSENSE, THIS SCRIPT DID NOT CAUSE THE BAN!

Dave Hybrid 07-01-2009 01:55 PM
I'll hang on before making it an official release, seems silly to update daily.

NLP-er 07-01-2009 02:38 PM
Quote:
Originally Posted by Dave Hybrid (Post 1840747)
I'll hang on before making it an official release, seems silly to update daily.
Those who didn't made update yet will have only one...


All times are GMT. The time now is 04:59 PM.
Page 4 of 18 « First 23 4 5614 Last »
Show 40 post(s) from this thread on one page

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.02146 seconds
  • Memory Usage 2,001KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (10)bbcode_code_printable
  • (20)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (1)pagenav_pagelinkrel
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (40)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 
Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 
Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete