[imported_silkroad]

Originally Posted by KrisP

Hey champ

wget fetches one link at a time serially that takes 1-5 seconds to load each while "Website downloader for windows" is downloading 20 links parallel at a time.

First of all, if you want wget to download 20 links in parallel, you can simply run twenty instances. If you want wget to download 100 links in parallel, you can simply run 100 instances, etc.

Second, most people do not want to put the type of load you are talking about on a production server. For example, our forum gets more than 4,000,000 page views a month, so there is NO WAY we would run any process that spiders 20 links in parallel (we even regulate how GoogleBot spiders our site).

As I said earlier KrisP, you don't understand the details and implications of what you are talking about. You don't understand the basics of performance management, system management, and seem to be a "windows user fan" who just wants to make a statement about how great windows it versus a unix/linux command line tool. So be it, but you are simply spreading misinformation here.

Of course, GoogleBot uses a variety of wget for all their spidering, so I guess you will argue with Google as well I promise you Google does not use a windows spidering tool, ROTFL.

If you want to run 1000 parallel instances of wget, do it. Anyone who understands basic command line scripting could write this script in less than 10 minutes.

Like I said earlier you have a (1) configuration problem and you also have a (2) lack experience and knowledge issue, obvious by your posts. That is no problem. We are all glad to help you, but do not post misinformation here (please!) about how to use wget and how great windows is over unix/linux command line knowledge and expertise.