Googlebot!

[filburt1]

I dumped this in my phpinclude template. Does it look like it will properly e-mail me if Googlebot is encountered, then not e-mail me again for two hours even if it is still there?

PHP Code:

if (strpos($_SERVER['HTTP_USER_AGENT'], 'Googlebot') != false)
{
    $result = mysql_query("SELECT lastvisit + INTERVAL 2 HOUR > NOW() AS withintwohours FROM nonstandard_visitors WHERE useragent = 'Googlebot'");
    $s = mysql_fetch_row($result);
    if ($s[0] == '1')
    {
        mail($technicalemail, 'Googlebot is at the site!', 'Googlebot is at the site at ' . strftime('%X') . '.
            There will be no more e-mails until at least two hours have passed.');
        $result = mysql_query("UPDATE nonstandard_visitors SET lastvisit = NOW() WHERE useragent = 'Googlebot'");
    }
} 


The schema for nonstandard_visitors is:

mysql> describe nonstandard_visitors;
+-----------+---------------+------+-----+---------+----------------+
| Field     | Type          | Null | Key | Default | Extra          |
+-----------+---------------+------+-----+---------+----------------+
| id        | int(11)       |      | PRI | NULL    | auto_increment |
| useragent | varchar(255)  |      |     |         |                |
| lastvisit | timestamp(14) | YES  |     | NULL    |                |
+-----------+---------------+------+-----+---------+----------------+
3 rows in set (0.00 sec)

[DrkFusion]

Sure why not

*doesn't know what ^^ is talking about*

Your forums are still not being being listed :-/

-Arunan

[filburt1]

GB crawled them before the modifications were complete, and still hasn't even added stuff to the index.

I'm confident

[Erwin]

Google looks like it's restricting pages from one site to only up to 1000 pages. It used to be much higher. You can test that script easily by changing the HTTP bit to the one that looks for an Opera browser, then surf your site with Opera. It looks okay from here.

[filburt1]

How does it get to the 1000th? Go to one page, parse everything, then back to the orignal, next link, everything on that page, etc., or recursively until it just buries itself in the first link?

Of course I think 1000 will still be fine for me; I don't have that many forums or threads to account for 1000 indicies

[Erwin]

They may have worked out .htaccess aliasing, because with my site, and other sites like eva2000's, Google no longer list all the aliased page - some, but not all. It prefers 3rd-party links like Photopost (with mine) or Gallery links (with eva2000)'s site. eva2000's site used to have over 20,000 pages in, but now down to only 900 odd pages. Same with mine.

To see how many pages your site has in Google, run this search:

domainname site:domainname.com

(replace domainname with your site's domain name)

[Erwin]

Originally posted by filburt1
Of course I think 1000 will still be fine for me; I don't have that many forums or threads to account for 1000 indicies

But if you have made it so that EVERY page of a thread is indexable, and you have over 350,000 posts... That's a lot of potential pages in Google. I remember fastforward saying that he had over 75,000 pages in and they were only the first page of every thread, not every page.

[filburt1]

Well apparently my code doesn't work because GB is there now and I didn't get an e-mail

[DrkFusion]

How do you know if GB is there?

-Drk

[filburt1]

He happened to be in WOL