Forum Display Enhancements - vB Enterprise Translator

2.3.15 RELEASED - TRANSLATIONS NEVER WERE SO FAST!!!

Great thanks to all who nominate vBET and allow us to win MOTM :up:

This mod translates your forum using FREE translations for over 30 languages from Yandex API. It is 100% FREE in version described here (2.x).
Note: using this mod (vBET 2.x) is free and bound by license agreement (see below)

Yandex translation gives you 1 000 000 FREE characters per DAY to translate. Please note that Yandex do not supports all language pairs, but vBET detects automatically which are supported for your forum language.

Version for vBulletin 4.0 is here: vBET 2.4.x for vBulletin 4.0

vBET 3.x released!
Please note that vBET 3.x is developed as separate product line and it is available here: vBET 3.x. Full version is paid, it supports 53 languages each to each translation, using: Google, Microsoft, Apertium, Yandex, SDL BeGlobal and DetectLanguage. It has lot of optimizations for booth performance and translations costs.

LIVE DEMO - note that demo shows functionality of full version (3.x/4.x), which has more features than 2.x
See also screen shots below, for configuration options

Features:
- Translation by Yandex (in PRO version supports 53 languages using several translation providers)
- Optimized cache, algorhitms and more performance options - PRO version only
- Translates URLs and RSS - PRO version only
- Supports multi language communities (allows to write in different languages) - PRO version only
- Links appropriate for SEO consensus (configurable) - even without any SEO mod
- Long texts are translated no matter how big those are
- Translation cache (after first translation results are cached and next page generation is much faster)
- Doesn't translate BB codes for code, php and html (must refresh post cache to see effect on existing posts Admin CP -> Maintenance -> Update Counters -> Rebuild Post Cache)
- Easy installation - just upload files and import product file
- Highly configurable and easy configuration - by Admin CP (see screenschots)
- Configurable place for flags - simply select in options where you want to have translation flags. NO MANUAL TEMPLATE MODIFICATION. Supported: Footer-Bottom, Footer-Top, Navbar-Bottom, Navbar-Top, Custom
- Possible custom place for flags - in such case just edit template which you want and add $vbenterprisetranslatorflags
- Translates also archive pages
- Reverse page direction for Arabic, Hebrew and Persian languages
- Not translated save areas - if you need something to be not translated simply edit appropriate template and put it between
<!--START vBET NOT TRANSLATED AREA--> and <!--END vBET NOT TRANSLATED AREA--> (note: you cannot use any mod which removes comments to use this functionality)

Installation:
1. Download zip package (last release - see index at the end of file name)
2. Upload to your forum directory on server everything inside upload directory
3. Import in vB Admin CP -> Plugins & Products -> Manage Products -> Add/Import Product -> Choose product file, set Allow Overwrite to yes and click Import
4. Click here: "Mark as Installed" Thanks!
5. Configure in AdminCP -> vBulletin Options -> vBulletin Options -> vB Enterprise Translator
You MUST configure at least Forum language and Yandex Translation API key. NOTE that there are some other important options - especially if your forum is in subdirectory or if you use vbSEO.

For MORE INFO and troubleshooting please read first post There is FAQ. So save your time for writing questions which are already answered and get your solution faster

Known issues:
- link for page on which you are doesn't track translation, cause it has same URL as flag for forum default language

ENJOY! :up:

And thanks for nominations

Already donated (great thanks:up:
PossumX, adwade, roymogg, zipperty, CoZmicShReddeR, RedHacker, john devost, mandingo, chris, smnoel, Audax666, Damas, TurkeySub, nectons, Scalemotorcars, TheWite, Cybershaolin, burlesque, Ransy72560, chick, jaryx, linuxututs, cormee, dai-kun x2 :up:, imported_silkroad, Megatr0n, KrisP, CareyCrew, ghisirds, vitoreis, cyc, mosz, CThiessen, actolearn, jaconi, andie215, yahoooh, bollie

LICENSE AGREEMENT - using this mod you agree for those conditions:
- you are NOT allowed to remove credit link under any conditions
- removing credit link means BREAKING LICENSE AGREEMENT and you lose your rights to use this mod
- using it after that is illegal, it is software PIRACY and it will be notified in vBulletin community and other places, including appropriate government/police units in your country
- author of this mod reserves the right to change the terms of this license agreement at any time. Failure to receive notification of a change does not make those changes invalid. A current copy of this license is and will be available here.

Please note that in full vBET version there is possibility to buy branding free license. There is no such possibility in free version included here.

Version history:
2.3.15 Changes:
- Fixed canonical link
- Fixed issue with SSL verification
2.3.14 Changes:
- Added 48 new languages pairs
- Updated link to Yandex in configuration
- When translation limit is reached now we show original text instead of empty message
2.3.13 Updated link to Yandex API key
2.3.12 Google changed to Yandex translations
2.3.11 Added "Affiliate Program" parameter
2.3.10 Footer bug corrected
2.3.9 userip added to Google translation request + small bug fixed
2.3.8 Small bug fixed - now all supported languages can be chosen as forum default language
2.3.7 Bugs fixed:
- weird redirections with vbet_lang
- double language in URL
- changing text direction for specific languages
Added support for:
- usernames not translated in who is online
- supported URL replacements for canonical URL
2.3.6 Important fix! - there was issue with sending referrer header which is necessary by Google conditions. This one was fixed + issue with # in translation links
2.3.5 Important change - According to Google conditions referrer header must be added to each translation query - otherwise you can be banned by Google. Install this version to avoid any troubles.
Changes:
- Added referrer header for Google translation queries
- Corrected bug for returning to forum main page from translated page
Also according to actual Google conditions your "Powered by Google" information should have Google logo. So please change text for flags in your vBET configuration to something like this:

Automatic Translations (Powered by <span style="vertical-align:sub;"><img src="http://www.google.com/uds/css/small-logo.png" alt="Powered by Google" border="0"/></span>):

You can customize it but "Powered by Google" should stay.
2.3.4 Important bug fixed! Corrected translation of meta keywords and description
2.3.3 Important bug fixed! And also some other bugs fixed
- Fixed bug with shifted results caching
- Fixed bug for URLs to javascript functions
- Fixed bug for not translating usernames in postbit
- Little cosmetic changes
2.3.2 Really little changes.
- Changed default value of "Links title" attribute, to be more accurate for Google conditions
- Little template changed for flags, so "Link title" font will be smaller (just like vB info in footer)
- Small bug fixed
2.3.1 Some little bug fixes. Confirmed working with vbSEO 3.3.1 and confirmed working SEO links on forums with vbSEO and subdirectory in URL
2.3.0 Great news for those without vbSEO
- Supports translation SEO links even without vbSEO
- Translate Google Maps (configurable)
- Little optimizations (i.e. included only necessary things for not translated pages)
- Supports several cache cleaning strategies (optimization for really big forums) and automatically disables cache for cleaning time to avoid read/write conflicts
2.2.1 Small bug fixed with returning to default language for main page and SEO links (happens only for specific vbSEO configuration, so you don't need to update if 2.2.0 is working for you)
2.2.0 Translation tracking is working even when redirection by some mod occures (configurable reaction time)
2.1.0 9 new translation languages added! Also:
- Redirect to user language functionality - configurable (disabled, forced, after asking user does he want to be redirected).
- Fixed bug for SEO links with forums acting like directories by id.
- Corrected several flags
Update note: if you use SEO links you must change .htaccess file according to description in first post. If you changed any vbseo.php rules ? un-change those.
2.0.3 Quick bug fixing (issue with bitfields in options panel)
2.0.2 Added option to set your forum main domain for tracking URL's in subdomains
2.0.1 Small bug fixed and added direction changing suport also for Hebrew
2.0.0 Some small or medium changes and one big - for vbSEO users now SEO consensus links available.
What's new:
- Links appropriate for SEO consensus (configurable)
- Persian support added
- Doesn't translate BB codes for code, php and html (must refresh post cache to see effect on existing posts Admin CP -> Maintenance -> Update Counters -> Rebuild Post Cache)
- Reverse page direction for Arabic and Persian languages
- Option to set forum directory if not linked to domain
- Few bugs fixed
1.1.1 Some little changes and bugs fixed
- Re-encode not translated parts, so there is no strange signs in scripts, images title etc.
- Corrected flag for Sweden
- Few harmless bug fixed in translation tracking and cache disabling
- Respecting Google conditions default flags title have link to Google (configurable)
1.1.0 Lot of new features and no longer in beta stage
- Long texts are translated no matter how big those are
- Cache is working for chosen languages (you simply select for each language does translations should be cached)
- Option to add Google Translate API key for translation requests
- Configurable cache time to live and scheduled task which automatically remove old cache.
- Option to show flags only for guests
- Option to easily remove flags border (without any style changing)
- Option to not translate usernames
- During translation changes param lang in html tag
- Option to translate messages changed by vBSEO to JavaScript (it has such functionality for SEO purposes)
1.0.5 Flags showing correctly for forums in subdirectory + corrected returning to base language
1.0.4 Bug fixed with translation of '0' string. Also added little feature - if some parts of your forum uses vbSEO and some are not then now flags will have appropriate URL's everywhere
1.0.3 Big bug fixed Mod doesn't mess in template anymore (it did in some specific cases)
1.0.2 Small bug fixed - error with array merge solved
1.0.1 Small bug fixed - now choosing supported languages in options works fine
1.0.0 Great mod released

[NLP-er]

Originally Posted by imported_silkroad

Also, I wanted to add that if anyone has any ideas how to improve the issues that this mod seems to perform (2) a lot of joins without indexes and (2) write a lot of tmp tables to disk (see screen shots), that would be great :-)

Actually joins are made only if you use selecting by serie (which is configurable) and are joining by serie which is indexed and is a number (so it is very fast to compare).

I see your screen shots and I'm very surprised - you are sure that this is about this mod

Look yourself to product file - each table has index for serie, and look in to vbenterprisetranslator_functions.php on SELECT queries - all 3 which joins tables are using serie to join... Maybe I miss something but I'm still sure that all joins in my mod are made with indexes. So - unless you show me exactly which query causes it I assume that the reason can be in other mod or vB.

About BLOBS. DB is optimized to handle long textes - it means what don't need be in BLOB isn't. Short texts are put into small and medium cache. Long - must be in blob and there is no other way. In SQL you will find VARCHAR(65000) but mySQL will change it to some BLOB or TEXT field. Medium cache is as long as it is possible to be possible to create unique indexes - one char more and unique will not be possible. Also VARCHAR(328) in medium cache statys as varchar. It appears that in older version of mySQL like 4.1.1 it is changed VARCHAR(328) to BLOB, but if someone choose to use old soft which is slower - for me means that he don't need speed, so I optimized it for those who use higher versions

Still - if someone have idea to improve - I will gladly read what solution will make it faster

[NLP-er]

Originally Posted by imported_silkroad

Sorry! I thought you used the idea from the code he posted before on the topic, in the hl mod.

Nope Didn't know about this post. Just look in Google documentation, found key param and changed very few lines of code

[imported_silkroad]

Originally Posted by NLP-er

AI see your screen shots and I'm very surprised - you are sure that this is about this mod

Ah! That is from the old hl mod (or maybe another mod) .... !! But the issue only started with we installed the "old hl mod" cause I monitor DB performance rather carefully.....

I look forward to seeing the new performance after porting!

Thanks.

FYI, here is part of the T2x Google API code from "before" :

@curl_setopt($ch, CURLOPT_URL, "http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&key=blahbalhjfkljfdlkjfdlkjfdkjfdkjfkdjfkdfjdkpNbYUjebhSesaThcIIYMEFOUC-VHLBB1KJQkw&langpair=".urlencode($fl.'|'.$tl)."&q=".urlencode($text));

[NLP-er]

Originally Posted by T2x

Okay I have updated my code on page 13 to the latest after all of the fixes for the various bugs

Still a few to sort out but its looking very nice

I works on main page now, but all images dissapear. Only on main page - in other places images are ok.

[NLP-er]

Just want to all let you know that I'm ready to make 2.0.0 release

I changed code and now links will be generated according to SEO consensus (/en/address) - configurable. Also added some other requeste features

Just waiting till T2x fix his code to work appropriatelly in main page and in forums in subdirectories Great work T2x :up:

Also new release will not require any vbseo.php changes

[imported_silkroad]

Originally Posted by NLP-er

Just want to all let you know that I'm ready to make 2.0.0 release

Amazing!

This will be MOTY (Mod of the Year!)

[wolfstream]

Originally Posted by NLP-er

What is wrong with the paths?

Judging by your own base install, everything.
Use one of your own translation links, on the base install, you will see massive image problems and what appear to be script problems (stats loading, etc). This is because of how you have set this up, the paths are pointing to /en/images instead of /images/ , etc.

Sure, this could be just an install thing, but when you provide a demo, it should work, otherwise, people won't install the product.

[yahoooh]

Originally Posted by T2x

Hey guys, I decided to do some work and make this mod comply with vBSEO's link consensus idea.

What this will do is change all of the translation likes like so:

For example if a page.html is translated into danish, the url will normally be:

www.website.com/page.html?language=da

This will change the format to

www.website.com/da/page.html

This is much better for SEO so I am sure all of you with vBSEO will like this.

WARNING: this will only work properly if your forum is installed at the BASE of your domain, so something like www.website.com/index.php, if your forums are installed at www.website.com/forums/index.php some parts of my code will not work properly for you!!

All you need to do is open up your vbseo.php and add this code right BELOW error_reporting(0);

Make sure it is ABOVE the includes

if(isset($_GET['vbseourl']))
{
	$processuri = 0;
	if(preg_match('/^..(\/|$)/',$_GET['vbseourl']))
	{
		$processuri = 1;
		$_GET['language'] = substr($_GET['vbseourl'],0,2);
		$_GET['vbseourl'] = substr($_GET['vbseourl'],3);
	}
	else if(preg_match('/^\/..(\/|$)/',$_GET['vbseourl']))
	{
		$processuri = 1;
		$_GET['language'] = substr($_GET['vbseourl'],1,2);
		$_GET['vbseourl'] = substr($_GET['vbseourl'],3);
	}
	else if(substr($_GET['vbseourl'],0,5) == "zh-CN" || substr($_GET['vbseourl'],0,5) == "zh-TW")
	{
		$processuri = 1;
		$_GET['language'] = substr($_GET['vbseourl'],0,5);
		$_GET['vbseourl'] = substr($_GET['vbseourl'],6);
	}
	else if(substr($_GET['vbseourl'],1,5) == "zh-CN" || substr($_GET['vbseourl'],1,5) == "zh-TW")
	{
		$processuri = 1;
		$_GET['language'] = substr($_GET['vbseourl'],1,5);
		$_GET['vbseourl'] = substr($_GET['vbseourl'],6);
	}

	if($processuri == 1)
	{
		if(substr($_SERVER['REQUEST_URI'],0,2)=='//')
			$_SERVER['REQUEST_URI'] = substr($_SERVER['REQUEST_URI'],1);
		$tempuri = isset($_SERVER['HTTP_X_REWRITE_URL']) ? $_SERVER['HTTP_X_REWRITE_URL'] :
		(isset($_SERVER['HTTP_X_ORIGINAL_URL']) ? $_SERVER['HTTP_X_ORIGINAL_URL'] :
		(isset($_SERVER['REQUEST_URI']) ? $_SERVER['REQUEST_URI'] :
		(isset($_ENV['REQUEST_URI']) ? $_ENV['REQUEST_URI'] : $_SERVER['SCRIPT_NAME']))
		);
		if (strstr($tempuri, 'vbseo.php') && $_GET['vbseourl'])
			$tempuri = preg_replace('#vbseo\.php.*#', $_GET['vbseourl'] . ($_SERVER['QUERY_STRING']?'?'.$_SERVER['QUERY_STRING']:''), $tempuri);
		$tempuri = preg_replace('#\#.*$#', '', $tempuri);
		$_SERVER['REQUEST_URI'] = $tempuri;
		if(preg_match('/^\/..(\/|$)/',$_SERVER['REQUEST_URI']))
			$_SERVER['REQUEST_URI'] = substr($_SERVER['REQUEST_URI'],3);
		else if(substr($_SERVER['REQUEST_URI'],1,5) == "zh-CN" || substr($_SERVER['REQUEST_URI'],1,5) == "zh-TW")
			$_SERVER['REQUEST_URI'] = substr($_SERVER['REQUEST_URI'],6);
		$_SERVER['HTTP_X_REWRITE_URL'] = $_SERVER['REQUEST_URI'];
	}
}
else
{
	$processuri = 0;
	if(substr($_SERVER['REQUEST_URI'],0,2)=='//')
		$_SERVER['REQUEST_URI'] = substr($_SERVER['REQUEST_URI'],1);
	$tempuri = isset($_SERVER['HTTP_X_REWRITE_URL']) ? $_SERVER['HTTP_X_REWRITE_URL'] :
	(isset($_SERVER['HTTP_X_ORIGINAL_URL']) ? $_SERVER['HTTP_X_ORIGINAL_URL'] :
	(isset($_SERVER['REQUEST_URI']) ? $_SERVER['REQUEST_URI'] :
	(isset($_ENV['REQUEST_URI']) ? $_ENV['REQUEST_URI'] : $_SERVER['SCRIPT_NAME']))
	);
	$tempuri = preg_replace('#\#.*$#', '', $tempuri);

	if(preg_match('/^..(\/|$)/',$tempuri))
	{
		$processuri = 1;
		$_GET['language'] = substr($tempuri,0,2);
	}
	else if(preg_match('/^\/..(\/|$)/',$tempuri))
	{
		$processuri = 1;
		$_GET['language'] = substr($tempuri,1,2);
	}
	else if(substr($tempuri,0,5) == "zh-CN" || substr($tempuri,0,5) == "zh-TW")
	{
		$processuri = 1;
		$_GET['language'] = substr($tempuri,0,5);
	}
	else if(substr($tempuri,1,5) == "zh-CN" || substr($tempuri,1,5) == "zh-TW")
	{
		$processuri = 1;
		$_GET['language'] = substr($tempuri,1,5);
	}

	if($processuri == 1)
	{
		$_SERVER['REQUEST_URI'] = $tempuri;
		if(preg_match('/^\/..(\/|$)/',$_SERVER['REQUEST_URI']))
			$_SERVER['REQUEST_URI'] = substr($_SERVER['REQUEST_URI'],3);
		else if(substr($_SERVER['REQUEST_URI'],1,5) == "zh-CN" || substr($_SERVER['REQUEST_URI'],1,5) == "zh-TW")
			$_SERVER['REQUEST_URI'] = substr($_SERVER['REQUEST_URI'],6);
		$_SERVER['HTTP_X_REWRITE_URL'] = $_SERVER['REQUEST_URI'];
	}
}

If you want to 301 all of your old pages to the new link format, you can put this in your .htaccess above all of your existing rules

RewriteCond %{QUERY_STRING} (hl=|language=)(zh-TW|zh-CN|..)
RewriteRule ^(.*)$ http://%{HTTP_HOST}/%2/$1? [L,R=301]

Please note that this code assumes you are running vBSEO.

This code will fix translation for the base of your website if you want the following format. www.website.com/de or www.website.com/de/ . this only applies to your main page.

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^/?((zh-CN|zh-TW|..))/?$ vbseo.php?vbseourl=$1/index.php [L,QSA]

If you want to generate proper sitemaps with all of your new language urls, all you need to do is open up your vbseo_sitemap_functions.php in your vbseo_sitemap directory then:

find this code

}else
    		    	$freq = $vboptions['vbseo_sm_freq_t'];
    		    	     
            	vbseo_log_entry("[showthread] forum_id: $forumid, thread_id: $threadrow[threadid], total posts: $totalposts, pages: $totalpages, views: $threadrow[views] $prior");

    			for($p=1; $p<=$totalpages; $p++)
    			{

Add this code right after it:

$alanguages = array('ar', 'bg', 'zh-CN', 'zh-TW', 'hr', 'cs', 'da', 'nl', 'fi', 'fr', 'de', 'el', 'iw', 'hu', 'it', 'ja', 'ko', 'no', 'pl', 'pt', 'ro', 'ru', 'sr', 'sk', 'es', 'sv', 'th', 'tr');
					if(!$archived)
					foreach($alanguages as $hl) {
					$vbseo_stat['t']++; 
					$currentthreadurl = vbseo_url_thread($threadrow, $p, $archived);
					$strlenhost = strlen($vbseo_vars['bburl']."/");
					vbseo_add_url(substr($currentthreadurl,0,$strlenhost).$hl."/".substr($currentthreadurl,$strlenhost), 0.3, $threadrow['lastpost'], $freq);
					}

make sure its before:

$vbseo_stat[$archived?'at':'t'] += vbseo_add_2urls(
    				vbseo_url_thread($threadrow, $p, $archived),
    				vbseo_url_thread($threadrow, $p, $archived, true),
    				$prior,
    				$threadrow['lastpost'],
					$freq
    			);
				
				}

    			if($showpost)
    			{

            	$getposts = $db->query("

If you notice there is an array in the code that looks like this

$alanguages = array('ar', 'bg', 'zh-CN', 'zh-TW', 'hr', 'cs', 'da', 'nl', 'fi', 'fr', 'de', 'el', 'iw', 'hu', 'it', 'ja', 'ko', 'no', 'pl', 'pt', 'ro', 'ru', 'sr', 'sk', 'es', 'sv', 'th', 'tr');

You should put the code of every language that you want added to your sitemap file in this array. Mine may differ from the languages you support so make sure you check it against your installation.


Just a note of caution: My fixes assume you are running a standard installation of apache with vBSEO. They may not work properly on some systems due to differences in the way that vBSEO handles some server variables. I recommend that you back up any files before editting them that way if you encounter problems you can restore your old versions.

I will be working on this code more so if any of you have issues please let me know and I will work on them.

If NLP-er would like he can provide a modified version of his flags in order to work with the new format or I will write up the code later today. This is necessary so that the flags show the proper links.

Anyway enjoy guys

how to make it work with addon not depend on vbseo like vba links and vba dyna which be only show flags when the requist_url is the setting

[yahoooh]

i hope Nlp can fix the issue related what i talk with him about vbseo_url and requist_url for pages in vba links and vba dyna

any solution to make it work in requist_url and make vbseo as requist_url by some change in vbseo files
it is eazy then to make all the site work

[imported_silkroad]

Originally Posted by yahoooh

i hope Nlp can fix the issue related what i talk with him about vbseo_url and requist_url for pages in vba links and vba dyna

any solution to make it work in requist_url and make vbseo as requist_url by some change in vbseo files
it is eazy then to make all the site work

FWIW, we run vBa Links and everything works fine (with the old hl mod) and the new t2x_vbseo.php code.

Seems it should be OK with the new language= mod (this mod), since it is better :-)