![]() |
Hey champ
wget fetches one link at a time serially that takes 1-5 seconds to load each while "Website downloader for windows" is downloading 20 links parallel at a time. |
Quote:
Second, most people do not want to put the type of load you are talking about on a production server. For example, our forum gets more than 4,000,000 page views a month, so there is NO WAY we would run any process that spiders 20 links in parallel (we even regulate how GoogleBot spiders our site). As I said earlier KrisP, you don't understand the details and implications of what you are talking about. You don't understand the basics of performance management, system management, and seem to be a "windows user fan" who just wants to make a statement about how great windows it versus a unix/linux command line tool. So be it, but you are simply spreading misinformation here. Of course, GoogleBot uses a variety of wget for all their spidering, so I guess you will argue with Google as well ;) I promise you Google does not use a windows spidering tool, ROTFL. If you want to run 1000 parallel instances of wget, do it. Anyone who understands basic command line scripting could write this script in less than 10 minutes. Like I said earlier you have a (1) configuration problem and you also have a (2) lack experience and knowledge issue, obvious by your posts. That is no problem. We are all glad to help you, but do not post misinformation here (please!) about how to use wget and how great windows is over unix/linux command line knowledge and expertise. |
Quote:
|
Quote:
|
Quote:
|
Quote:
vbenterprisetranslator_flag vbenterprisetranslator_flags_begin vbenterprisetranslator_flags_end Begin and end opens and closes div, and flag is one link. So you can customize it as you wish. |
Quote:
Yes, if this change is not made all links i sitemap will look like this, fx. : http:/www.mysite.com/showthread.php?t=205435?language=en instead of http:/www.mysite.com/showthread.php?t=205435&language=en BTW: There is also the posibillity to generate default sitemap and then search/replace: Find: </loc> Replace: &language=fi</loc> (except first link <loc>http://www.mysite.com/?language=sv</loc> that must be changed back to ?) And save as xx.xml for each language to get an overview of indexing different languages. (of course you have to do this manually every time you want sitemap updated - or you can just do it the first time and then change to big sitemap later) |
Is it possible to keep UI untranslated?
|
Quote:
I have a script that copies the vbseo_sitemaps and generates sitemaps for about 15 languages, moves them to their own subdirectories (one for each language) and then pings google after each one is updated. From experience, the size of your forum makes a big (very big) difference in how you approach indexing translations in Google. If you have a large forum with hundreds of thousands of post, you will hurt your traffic if you translate all languages and create sitemaps for all languages, because Google will not index all, and you will then have lots of links from low traffic languages. It is better, for large sites, to only index the top 10 languages and definitely do not make all the flags viable or you will have less traffic than targeting the major languages with high Internet usage. |
I see your point. However from my perspective I am doing a little experiment of creating some language specific domains (uploading a copy of the forum to these) - only showing one language per language specific domain. That is why I need seperate sitemaps. Hopefully this strategy will give good indexing.
I use language= links and this robots.txt to make spiders only spider the specific language (if some of you see a flaw in the sitemap, I would of course be very interested to know :) ) Quote:
I dont use the folder structure ("seo links") because I couldnt find a way to use robots.txt to narrow down to one language only using that type of links. |
All times are GMT. The time now is 02:14 AM. |
Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.
X vBulletin 3.8.12 by vBS Debug Information | |
---|---|
|
|
![]() |
|
Template Usage:
Phrase Groups Available:
|
Included Files:
Hooks Called:
|