Remove Spiders from Who's Online

[mfyvie]

*** Staff note: The author of this modification has passed away in a diving accident (http://www.englishforum.ch/announcements/20529-mourning-loss-friend-leader-innovator-genuinely-nice-guy.html). We wish his family all strength in dealing with this traggic issue. ***

Remove Spiders from Who's Online

The problem

The majority of the the "guests" on your forum are probably spiders. While you might think that it is impressive to show so many guests using your forum, this does not reflect the true number of people online. The who's online page does offer visitors the chance to change the display based on members, guests, or spiders, however the totals (and the record number of users online at one time) do not exclude spiders.

This mod does not remove spiders from your forum, it simply stops them being counted in the totals.

Some search engine spiders such as Yahoo's Slurp can initiate hundreds of connections at the same time (from different IP addresses). Each one of these connections will be included in your totals as a "guest".



What does this modification do?
Completely remove all known spiders (spiders found in your includes/xml/spiders_vbulletin.xml file) from the who's online display
Updates the true guest totals displayed on the main forum page
Resets your maximum users online record on installation
Includes changes for vBadvanced's online users module
Doesn't offer separate spider statistics or configuration, it just makes it appear like those spiders aren't on your forum (even though they are)
Easy access through template conditionals and variables to display different behaviours based on whether the connection is from a spider or not.

Who should use this modification?

Anyone who wants to their forum to show the true number of users online in the statistics

I run a big system, will this slow my board down?

Absolutely not. It's been written with big boards in mind. Even if you use a very large spiders_vbulletin.xml file, this file will only be checked once for each new session created. After that, whether the session is used by a spider is recorded in the session table. For users running vBulletin 3.6.6 or later, it will actually increase the speed of the who's online page, since known spiders are no longer compared against the list of known spider user agents. No new database queries are added, as existing queries are used or modified instead.

Hasn't this been done before?

There are a couple, but I've taken a slightly different approach with this mod - my focus was on simplicity and performance. You might also want to check out some other mods here (https://vborg.vbsupport.ru/showthread.php?t=145328) and here (https://vborg.vbsupport.ru/showthread.php?t=119134), to see if they suit your purposes better.



Installation instructions
If using vBadvanced, see the instructions below for a manual edit of your onlineusers.php file
Install the enclosed .xml file via AdminCP -> Plugins & Products -> Manage Products -> Add/Import Product. Remember to click overwrite if upgrading
Wait a few minutes - existing sessions must have some activity before they are reclassifed as a genuine member or guest
(optional) Edit your admincp quick statistics file (instructions below)
(optional) Reset your maximum online statistics again (if you feel the need, but it is done automatically during installation)Configuration instructions

Nothing to configure! Just see the note below about spiders_vbulletin.xml

How can I make this more effective?

Spiders are identified via the includes/xml/spiders_vbulletin.xml file. However, the version that ships with vBulletin is quite small, and will recognise only the most popular spiders. Anything else will still show up in your statistics as a guest, making it hard to tell real guests from spiders.

Therefore it is recommended to update your spiders_vbulletin.xml if using this mod. You can obtain a much better version of this file from Christian Stadler here (http://www.ragnarokonline.de/spiderlist). Please contribute new user agent strings directly to him and encourage him to keep providing new updates of this file.

Will using a big spiders_vbulletin.xml slow things down? Theoretically yes, but in practical terms now. Lookups are only one once for each session when it is created. If you are running a version earlier than vBulletin 3.6.6 it will mean that more work must be done on the who's online page, but nowhere else.

Which versions of vBulletin will this work on?

This mod is designed to work for 3.6.6 and above, but there is some additional code included to handle 3.6.0 - 3.6.5 as well.

Optional extras

If you are using vBAdvanced then you'll want to modify your online users module. The following change should work for but versions 2.x and 3.x. Go to your modules/onlineusers.php file and locate this line:
WHERE session.lastactivity > " . (TIMENOW - $vbulletin->options['cookietimeout']) . "Directly under this line, insert the following line (on a line by itself):
" . ($killspiders ? " AND " . TABLE_PREFIX . "session.spider = -1 " : null) . "Please don't forget to include the quotes exactly as they appear above.

If you'd like to also include the quick stats in your admincp with this modification open your admincp/index.php file and locate this line:
$guestsarry = $db->query_first("SELECT COUNT(host) AS sessions FROM " . TABLE_PREFIX . "session WHERE userid = 0 AND lastactivity > $datecut");replace it with:
$guestsarry = $db->query_first("SELECT COUNT(host) AS sessions FROM " . TABLE_PREFIX . "session WHERE userid = 0 " . (($vbulletin->products['whosonline_kill_spiders'] AND $vbulletin->options['enablespiders']) ? ' AND spider = -1 ' : null) . "AND lastactivity > $datecut");Would you like to reset your maximum online users statistic (the one on the front page or who's online page that tells you the most number of users ever online)? go to the AdminCP -> Maintenance -> Execute SQL Query and run the following query:
DELETE FROM datastore WHERE title = 'maxloggedin' LIMIT 1

There is an additional file, spider_test.php that can be used to test the operation of this mod, as well as giving you a handy overview of which guests have not been classified as spiders, but probably should be. You can then use the user agent strings to expand your spiders_vbulletin.xml file. You can run spider_test.php from any forum directory, just upload it and then type the path to the file into your browser.

Support

Will be provided to those who click install

Version history

1.0 (15.07.2007) Initial version

Disclaimer: No actual spiders where harmed during the development of this modification.

[mfyvie]

Reserved

[TTG]

Yahoo slurp are the biggest PITB but I just use a robots.txt to reduce the times they are allowed to visit. Works well enough for me. Whats the benefit in this compared to a simple robots.txt ?
User-agent: *
Crawl-delay: 30
Disallow:
User-agent: Slurp
Crawl-delay: 1500

[mfyvie]

Robots.txt works only for those bots that respect it, and identifying every single bot to put into a robots.txt can also be time consuming. There is also a performance overhead associated with robots.txt. I block all my bad bots directly in the webserver, but all the "good bots" to crawl my site.

However, everything that I've written above (and your question) relating to blocking bots from the site.

This mod simply removes the spiders from the totals, allowing you to get a better idea of how many "real" guests you have on your forum. If you keep statistics and graphs as I do this gives you a clearer picture of growth, etc.

It also saves questions from users saying "Wow, there were 800 guests online last night, this forum is really popular".

[iogames]

What can I say??? :rolleyes:

[FleaBag]

Hi Mark! A lovely little hack, when you said you have a bunch o' "why didn't anyone else think of that" hacks, you were right!

I will install sometime soon! And on another note, apologies I have not replied to your email (I'm not ignorant, honest) - I just forgot I'd emailed you from my domain account (and not my Gmail account) - I couldn't work out where your email had gone!

[keyness]

You know there are some good mods, for example in my forum it seems like this: (14 members & 16 guests & 1 logged out & 32 bots) But I really would like to remove bots from who's online without reducing number in forumhome. Could it be possible?

[mfyvie]

You know there are some good mods, for example in my forum it seems like this: (14 members & 16 guests & 1 logged out & 32 bots) But I really would like to remove bots from who's online without reducing number in forumhome. Could it be possible?

You could selectively disable the plugins from this mod to remove spiders from who's online, but leave them on the main forum page if you want.

[mfyvie]

Some of you might be wondering what is so different about this mod, so I'll take a little bit of time to explain how you might use it to do some different stuff.

This mod actually classifies each session as spider or normal user when the session is created and stores the result in the session table. Why? Because it makes it very easy for us to check the "spider status" on any session from any page, WITHOUT having to compare the user agent against a list of known user agents every time.

Why is this better? Two reasons, the first is speed and the second is flexibility. By default the mod will suppress the spiders from the statistics (it won't block anything). But we could use this mod to do more if we wanted.

If you look in the session table you'll see an extra column called "spider", the values are -1 (normal user), 0 (not yet determined - you'll rarely see this) and 1 (spider). Now you can easily change things depending on whether a spider is viewing the page or not. You could put something in a template like this:

<if condition="$session[spider] == 1">Come into my parlour, said the spider to the fly</if>Now that text would appear only to spiders. Along the same lines you could show text only to real users like this:

<if condition="$session[spider] == -1">Don't tell the spiders, but they can't see this text.</if>You could also use it inside any plugin simply by testing the value of $vbulletin->session->vars['spider'] like this:

if ($vbulletin->session->vars['spider'] == 1)
{
//Some stuff to do when we have a spider visiting
}Putting a variable into the session table to use elsewhere is the same kind of approach used in GLA (some of you might have seen this recently).

So with a little bit of imagination some of you might also be able to see that this mod could help you do all sorts of new things (I've seen many requests for example from people who want to display different links or pages to spiders than normal users - this would be simple using the above techniques).

What if you wanted to get some quick statistics to show how many users or guests are online. Easy - now you can use a simple SQL query like this:

SELECT COUNT(*) FROM session WHERE userid = 0 AND spider =-1This would show the number of non-spider guests in the connection table (but you'd have to remember to add something onto the end to limit the last activity to the last 15 minutes (or whatever you have it configured to), so the final code in a plugin might look something like this:

$guests = implode("", $vbulletin->db->query_first("SELECT COUNT(*) FROM session WHERE userid = 0 AND spider =-1 AND lastactivity > " . (TIMENOW - $vbulletin->options['cookietimeout']) . "))Again you could vary it to something like userid > 0 for members, or spider = 1 for the spiders. This way you can get the totals quickly from any page with just a single line of code and a single query.

[Quantnet]

thanks

[ERuiz]

I installed this mod and made the corresponding changes to the onlineusers.php file, but when I access my homepage, it says there are some guests on the who's online module block but if I go to the forum who's online display, it says there are no guests. What could be wrong?

[mfyvie]

I installed this mod and made the corresponding changes to the onlineusers.php file, but when I access my homepage, it says there are some guests on the who's online module block but if I go to the forum who's online display, it says there are no guests. What could be wrong?

Point 3 of the installation instructions - wait a few minutes ;)

When you first install the mod you will get different values for the numbers of guests in your vbadvanced block, the who's online total, and the total listed on the main forum index. Once your cookie timeout has expired (15-30 minutes) these values should converge to be the same. The reason is that you may be seeing old sessions which (because they are not active) have not been reclassified to either spider or non-spider in the session table.

[ERuiz]

Point 3 of the installation instructions - wait a few minutes ;)

When you first install the mod you will get different values for the numbers of guests in your vbadvanced block, the who's online total, and the total listed on the main forum index. Once your cookie timeout has expired (15-30 minutes) these values should converge to be the same. The reason is that you may be seeing old sessions which (because they are not active) have not been reclassified to either spider or non-spider in the session table.

Thanks for the quick response. Ok, I will give it 1 hour, LOL. Thanks again...

[ERuiz]

I did the edit for the control panel option and I get an error... It says it can't find the session table.

I had to manually add vb3_ (my prefix) to the session.spider portion of the code you provided and this got rid of the error.

I am using 3.6.7 and VBA RC1

[mfyvie]

I had to manually add vb3_ (my prefix) to the session.spider portion of the code you provided and this got rid of the error.

Thanks for pointing that out - I checked my example above and changed it slightly. I dropped off the "sessions." part which means it should work without a prefix. I also noticed a small problem with that example - the value to test for was 0, it should have been -1. You should change yours accordingly otherwise it will report your number of guests as 0.

[keyness]

There is something wrong in this mod mfyvie, I've installed it 30 minutes ago, but when I click to who's online, I just can see 5 members and 1 guest, but I am pretty sure, there are at least 10 active users for last 2 minutes. I'll wait several minutes too.

[efil]

Hi,
When i use vBadvanced onlineusers.php edit, I get this database error:
Database error in vBulletin 3.6.7:

Invalid SQL:

SELECT session.userid, username, usergroupid, (user.options & 512) AS invisible, IF(displaygroupid=0, user.usergroupid, displaygroupid) AS displaygroupid
FROM session AS session
LEFT JOIN user AS user USING (userid)
WHERE session.lastactivity > 1184558427
AND " . TABLE_PREFIX . "session.spider = 0

ORDER BY username ASC;

MySQL Error : You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'session.spider = 0

ORDER BY username ASC' at line 5
Error Number : 1064

[mfyvie]

There is something wrong in this mod mfyvie, I've installed it 30 minutes ago, but when I click to who's online, I just can see 5 members and 1 guest, but I am pretty sure, there are at least 10 active users for last 2 minutes. I'll wait several minutes too.

How are you sure that there are 10 active users? The other 4 may be spiders, or they may be inactive. Which view was this? Check both the front page of your forum, and the who's online display - in the first 15-30 minutes these may report different numbers. If you still don't believe it, you can manually check the session table in the database (however, this also shows sessions that are inactive). In the column "spider" you'll see -1, 0 or 1. 1 means spider, -1 means non-spider and 0 means that it hasn't decided yet (happens on new installs for inactive sessions). I wouldn't be so quick to pronounce something wrong with the mod, it could simply be that you have less "real users" than you thought ;)

Hi, When i use vBadvanced onlineusers.php edit, I get this database error:

I've altered the syntax of that example I posted slightly use double quotes instead of single quotes. If you just change the two single quotes into double quotes it should work (or recopy the line from my example in the mod text at the top). I've tested this on my own forum and it worked.

[Hornstar]

Thanks, I may consider installing this, as it might offer an improvement on performance.

[projectego]

Great idea. Thanks a bunch!

/me clicks install

[keyness]

How are you sure that there are 10 active users? The other 4 may be spiders, or they may be inactive. Which view was this? Check both the front page of your forum, and the who's online display - in the first 15-30 minutes these may report different numbers. If you still don't believe it, you can manually check the session table in the database (however, this also shows sessions that are inactive). In the column "spider" you'll see -1, 0 or 1. 1 means spider, -1 means non-spider and 0 means that it hasn't decided yet (happens on new installs for inactive sessions). I wouldn't be so quick to pronounce something wrong with the mod, it could simply be that you have less "real users" than you thought ;)

Come on mfyvie, I'm not a newbie at all :D

Well, I also have "who has wisited today" add-on, it shows the user's last activity time when you hover on nickname. I've counted more than 10 while who's online page shows only 5 members and 1 guest.

[mfyvie]

Come on mfyvie, I'm not a newbie at all :D

Well, I also have "who has wisited today" add-on, it shows the user's last activity time when you hover on nickname. I've counted more than 10 while who's online page shows only 5 members and 1 guest.

I'm not suggesting you are a newbie. I thought myself that I wasn't showing enough guests, but when I checked it was correct.

Are you saying the display of the members or the guests is incorrect? At first you said it was guests, then you bring up the point about your who has visited today mod which gives you information on members. My mod does nothing at all to the display of members.

If you still doubt the mod, best to look into the session table. Which version of vb are you running? It makes a difference because two different sets of code run depending on the version.

[efil]

Hi,
Now there is no database error , but despite there is 6 members , i can see in vBadvanced just 1, me.
It's working good at forumhome.

[mfyvie]

Sorry, I thought I'd changed the example to -1, it was still on 0 (now fixed). In your vbadanced module the line should read spider = -1, not spider = 0. Change that and it should work the way you want it to.

[efil]

Sorry, I thought I'd changed the example to -1, it was still on 0 (now fixed). In your vbadanced module the line should read spider = -1, not spider = 0. Change that and it should work the way you want it to.

Thank you!
Now it's working.

[keyness]

Are you saying the display of the members or the guests is incorrect? At first you said it was guests, then you bring up the point about your who has visited today mod which gives you information on members. My mod does nothing at all to the display of members.

If you still doubt the mod, best to look into the session table. Which version of vb are you running? It makes a difference because two different sets of code run depending on the version.

Well both are incorrect. For more than 30 minutes, it just showed maximum 5 members and only 1 guest (and another proof, more than 20 guests enter forum just via google in an hour)

Maybe it's because i use too many mods. By the way I'm using classic vbulletin 3.6.7 version.

[mfyvie]

Keyness, the only way for you to verify this is to look into the session table. There you will see the guests and how they are classified (by checking the spider column). -1 means a real guest, 0 means not decided, and 1 means a spider.

I have many guests who arrive on my forum by google, but in reality they don't do anything and they don't move around the site. In this case their status may stay on "0" and they won't get shown in the totals. For example, I have a lot of hits where google has brought someone directly to an attachment or an image has been sideloaded because an image in one of my threads has been directly linked in another forum's thread. This type of thing creates sessions in your table, but these users show no activity on your forum in reality.

I don't think any other mods are affecting it, because those totals are nothing more than the number of rows returned from the mysql query that checks the spider status of each connection.

For example, on my session table right now I have 89 guests with spider status -1 (real guest), 4 with status 0 (unknown) and 81 with status 1 (spider). Of course different forums will vary in how many real guests they have versus spiders.

Just bear in mind that a visitor coming in via google could have come in for any reason - it could even be that they found an image via google images and that image was displayed on the google images page - this will appear like a visit, but it isn't really.

If you want to be really sure you could do something like a grep on your web server logs for each IP address you want to test. I did this when I was testing the mod, because like you, I thought it might not be working. During this process I actually discovered how many other forums were sideloading my attachments and I reconfigured my web server to block them!

[testebr]

What software did you use to generate those graphs (https://vborg.vbsupport.ru/attachment.php?attachmentid=67003&d=1184507800) and how to?

[mfyvie]

The software is called MRTG. It's free, but to be honest it isn't that simple to setup and get running. Once you get it running then you need some extra code to provide the vbulletin specific stuff, I was thinking of releasing that as a mod, but I didn't want to deal with the million requests about how to get MRTG running :(

[Kohhal]

This is the who's online hack I've been waiting for, much simplier and truer than others out there. Like you, I just want to hide the spiders, my users won't care to see what spiders are on the site so no reason to display them IMO, great hack, installing now :)

[SnitchSeeker]

This looks like an EXCELLENT mod!

But honestly, it's more than I want. For example, I don't want to reset my max users online stats and I already have a mod (vbBOL) that shows spiders as spiders in the "who's online" page.

Mainly, I am looking for some way of hiding certain html code in my templates from spiders (or showing them custom code), as you show in this example:<if condition="$session[spider] == 1">Come into my parlour, said the spider to the fly</if>

Can I install this without resetting stats? Or do you know of another way that I can change code in templates for spiders?

I don't mind not showing spiders as guests in the "Currently Active Users" stats, I just don't want to ruin anything else.

I think it would be cool to show how many bots/spiders are online in the "Currently Active Users" stats. As an addition after "__guests" like (42 members, 192 guests and 63 spiders)

[mfyvie]

Well both are incorrect. For more than 30 minutes, it just showed maximum 5 members and only 1 guest (and another proof, more than 20 guests enter forum just via google in an hour)

Ok, I've now attached an additional file called spider_test.php, you can run this from any forum directory. It will test your installation and basically show you the things that I have been describing to you about this issue (regarding when I asked you to verify your doubts by looking into the session table). Since this file shows you directly what is in your session table, but classified according the spider type, you should be able to see that the statistics being reported are the true statistics.

This file is also useful for anyone else who doubts that this mod is working, or for anyone who wants to find new spider strings to add to their spiders_vbulletin.xml.

This is the who's online hack I've been waiting for, much simplier and truer than others out there. Like you, I just want to hide the spiders, my users won't care to see what spiders are on the site so no reason to display them IMO, great hack, installing now :)

Thanks for your encouraging words. Within an hour of this mod being released 3 people rated it "Terrible" via the rating system, but nobody had actually downloaded it. I for one use the rating system when I'm deciding whether to install a mod, so I was puzzled as to why 3 people would have done this (without even looking at the mod). But anyway, I'm glad there are people out there that do understand why this mod is a little different and like it for exactly what it is.

This looks like an EXCELLENT mod!

But honestly, it's more than I want. For example, I don't want to reset my max users online stats and I believe I already have a mod to not show spiders as guests.

Mainly, I am looking for some way of hiding certain html code in my templates from spiders (or showing them custom code), as you show in this example:<if condition="$session[spider] == 1">Come into my parlour, said the spider to the fly</if>How can I do that without resetting or changing the other stuff?

Well actually, it sounds like this mod is not more than you want, it's exactly what you've been looking for, and it does exactly what you need. If you believe that you already have a mod that excludes spiders, then disable it - because this mod does that as well, but probably with less overhead to your forum.

It seems like your only beef with this mod is that it will reset your max online statistics when it installs? The reason it does this is because your max online statistics are probably false! Even if other mods remove the spiders from your display, are they stopping the artificially high number from inflating your max online statistics? Even if they stop it at the front end, have they also thought to stop it in the back end as well? Because as soon as you login to your admincp the wrong statistics will be recorded! My mod stops the admincp from updating this number, even if you haven't done the optional manual edit in the admincp.

But if you still want to disable the reset of the maxonline, simply download the .xml file and remove the entire block that looks like this:

<code version="*">
<installcode><![CDATA[// Each time we install we'll zero out the max online counter.

$vbulletin->maxloggedin['maxonline'] = $totalonline;
$vbulletin->maxloggedin['maxonlinedate'] = TIMENOW;
build_datastore('maxloggedin', serialize($vbulletin->maxloggedin), 1);]]></installcode>
<uninstallcode />
</code>Delete it, then save the .xml file. Now when you install it, it won't touch your (falsely inflated) max online stats. ;)

[Kohhal]

One thing I've noticed, I reset the max users online, used to be ridiculously high and inaccurate, immediatly I get a new figure for the time that I did the reset , BUT, it still seems to be including the spiders in this figure as the forum shows 10 members and maybe 10 guests, but most users online after reset shows 100+ ?

[SnitchSeeker]

Delete it, then save the .xml file. Now when you install it, it won't touch your (falsely inflated) max online stats.
Thanks!

My stats are falsely inflated, but not by more than a couple hundred at most. If I reset it, the figure would be incredibly low and inaccurate, so I don't know which is worse. My boards were WAY more popular 2 years ago than today. The reason I know this (other than the inflated "most users" stat) is we had over 100 new users registering each day. Now we have maybe 30 new users on average per day. "Most users online" currently shows 1,786 (dated 2 years ago) and most of late is around 1,000 less. :(

I just noticed I am running 3.6.5 and not 3.6.6. Do you think it will be ok for me to install it?

PS. I think it would be cool to show how many bots/spiders are online in the "Currently Active Users" stats. As an addition after "__guests" like (42 members, 192 guests and 63 spiders)

[mfyvie]

One thing I've noticed, I reset the max users online, used to be ridiculously high and inaccurate, immediatly I get a new figure for the time that I did the reset , BUT, it still seems to be including the spiders in this figure as the forum shows 10 members and maybe 10 guests, but most users online after reset shows 100+ ?

Ok, well there are only 3 places I know of where this can happen. The admincp (which my mod stops from updating the total), the who's online display, the main forum display, and the vbadvanced module (if you are running vbadvanced). If for example you are running vbadvanced and forgot to edit this module, it will be reseting the totals for you.

Can you confirm that you aren't running vbadvanced, and that you don't have any other areas of your forum where the maxonline is displayed (or updated)? Also, which version of vb are you running? If you are running pre 3.6.6 there is different code, but I wasn't able to test this because I'm running 3.6.7.

My stats are falsely inflated, but not by more than a couple hundred at most. If I reset it, the figure would be incredibly low and inaccurate, so I don't know which is worse. My boards were WAY more popular 2-3 years ago than today. The reason I know this (other than the inflated "most users" stat) is we had over 100 new users registering each day. Now we have maybe 30-50 on average. "Most users online" currently shows 1,786 (dated 2 years ago) and most of late is around 1,000 less. :(

I just noticed I am running 3.6.5 and not 3.6.6. Do you think it will be ok for me to install it?

It should work, though I'm not able to test it - there's a note in the installation instructions about pre 3.6.6 sites.

PS. I think it would be cool to show how many bots/spiders are online in the "Currently Active Users" stats. As an addition after "__guests" like (42 members, 192 guests and 63 spiders)

Actually this mod was designed not to even offer that as an option. I'm reusing the same query, so there's no additional overhead. To start getting fancy means to depart from the original design goal of this mod - simplicity and speed. I've never understood why people want to know the number of spiders on their board - it's a totally meaningless number - you could have 500 yahoo slurp spiders crawling one page per session, or you could have a single spider crawl 500 pages in a single session. What's more important to check is the spider activity in your weblogs - it really doesn't have any place in your online users in vbulletin.

[R@V3N]

Very nice mod. Just what I needed. Someone decided it would be fun to spam he online count and I like to report real numbers. Keep up the good work. :)

[Charlie98902]

I tried this out on my forum and there seems to be a error on IE 7 users the error # is 163 I think. There is more text and I'll gladly install it again to be verbatim if you need it.

[mfyvie]

I tried this out on my forum and there seems to be a error on IE 7 users the error # is 163 I think. There is more text and I'll gladly install it again to be verbatim if you need it.

Do you mean you saw this error when you tried to install, or when it was running? This mod doesn't actually directly change anything that is output to HTML, it simply changes things in the background. I've no idea what might have caused this? File permissions?

[koxito]

Wow .. a very good mod, installed on my board! :)

[TheBlackPoet]

hey... i love it.... i hate the deceptive countup on my site.... real members and real visitors work better for me to guage what's what!! so thanks..

[Charlie98902]

When it is running it may be conflicting with one of my other mods, it was just a want not a I need this really, so I can live without it.

[Kohhal]

Ok, well there are only 3 places I know of where this can happen. The admincp (which my mod stops from updating the total), the who's online display, the main forum display, and the vbadvanced module (if you are running vbadvanced). If for example you are running vbadvanced and forgot to edit this module, it will be reseting the totals for you.

Can you confirm that you aren't running vbadvanced, and that you don't have any other areas of your forum where the maxonline is displayed (or updated)? Also, which version of vb are you running? If you are running pre 3.6.6 there is different code, but I wasn't able to test this because I'm running 3.6.7.

I'm not running vbadvanced and I've no hacks or anything installed that should affect this. Only couple of minor hacks installed.

I don't have any non-default displays of max users online anywhere so dunno if that could be it.

I'm running 3.6.7 PL1

[SnitchSeeker]

I've never understood why people want to know the number of spiders on their board - it's a totally meaningless number - you could have 500 yahoo slurp spiders crawling one page per session, or you could have a single spider crawl 500 pages in a single session. What's more important to check is the spider activity in your weblogs - it really doesn't have any place in your online users in vbulletin.I suppose you're right. I was thinking mainly for my own benefit so that if the site is slow I can see if it's due to a high number of spiders on. I've never been good at checking logs. They're complicated to understand for me at least. I'm not very technical savvy.

[rungok]

Thanx for this mod. But I have a problem wich you have talked about before. When I insert " . ($killspiders ? " AND " . TABLE_PREFIX . "session.spider = -1 " : null) . " in the onlineusers.php I get this fault:
Database error in vBulletin 3.6.5:

Invalid SQL:

SELECT session.userid, username, usergroupid, (user.options & 512) AS invisible, IF(displaygroupid=0, user.usergroupid, displaygroupid) AS displaygroupid
FROM vb_session AS session
LEFT JOIN vb_user AS user USING (userid)
WHERE session.lastactivity > 1185130825
AND vb_session.spider = -1
ORDER BY invisible ASC, username ASC;

MySQL Error : Unknown table 'vb_session' in where clause
Error Number : 1109
Date : Sunday, July 22nd 2007 @ 09:30:25 PM
Script : http://www.nettkafeen.no/forums/portal.php
Referrer : http://www.nettkafeen.no/forums/index.php
IP Address : 81.167.190.76
Username : Mr.Man
Classname : vb_database
and the vb_session table exist.

[rungok]

Solved it. I just pasted this line without the table prefix, and it worked.
" . ($killspiders ? " AND session.spider = -1 " : null) . "
All of my tables have table-prefix _vb, but the rest of the original query refers to this table without it.

[AndyP]

Is anyone else trying to use this mod with PaulM's most ever mod (https://vborg.vbsupport.ru/showthread.php?t=122424)?

I'm currently getting low counts for active users, that is, spiders aren't included.
But the most ever guest count is a lot higher than I'd expect. It must be including spiders, and in turn affecting the most ever users online stat.

Are they compatable? To me it doesn't look like it.

[lebanon]

I just added ur mod, however whats weird is my record of users went down from 1600 to 172 members online and today as the record.
My who is online shows guests but the main page shows 0 guests,
Am only confused cause i cant see in the code what could mess up the record query !

[mfyvie]

Solved it. I just pasted this line without the table prefix, and it worked.
" . ($killspiders ? " AND session.spider = -1 " : null) . "All of my tables have table-prefix _vb, but the rest of the original query refers to this table without it.

Thanks for that. Very strange though - I have to check more into that syntax. I don't use table prefixes, or mysql 5 - are you using mysql 5 and if so, is strict mode set?

Is anyone else trying to use this mod with PaulM's most ever mod (https://vborg.vbsupport.ru/showthread.php?t=122424)?

I'm currently getting low counts for active users, that is, spiders aren't included.
But the most ever guest count is a lot higher than I'd expect. It must be including spiders, and in turn affecting the most ever users online stat.

Are they compatable? To me it doesn't look like it.

From the looks of it they wouldn't be compatible. The reason is that Paul's mod will not be differentiating spiders in the totals, and is probably re-writing the max online statistic.

I just added ur mod, however whats weird is my record of users went down from 1600 to 172 members online and today as the record.
My who is online shows guests but the main page shows 0 guests,
Am only confused cause i cant see in the code what could mess up the record query !

Sounds like the mod is functioning exactly as designed. I suggest you give it time (see other posts on this thread which cover the same subject) and the number of guests will increase to the correct number. If in doubt -use the test script I included to verify that everything is working as you expect.

[AndyP]

Thanks mfyvie, that's what I thought. It would be great if they did work together though, as now it looks like I might have to drop one of them, unless I can work out a way to edit it myself.

[mfyvie]

Thanks mfyvie, that's what I thought. It would be great if they did work together though, as now it looks like I might have to drop one of them, unless I can work out a way to edit it myself.

Take a look into Paul's mod and see if you can see the part where it does a query to get the number of guests. It should just be a matter of adding "AND spider = -1" to the "WHERE" section of the query. After that you'll probably want to manually reset the max online totals again.

[AndyP]

There wasn't a where statement.

However, I got it to fix the most guests online record from PaulM's mod by amending the execution order of your mod at the forumhome_complete hook to be earlier than Paul's. This way your mod was amending the max number of guests before Paul's mod used it.

Unfortunately the max users ever online is still including spiders and I don't believe PaulM's mod affected this.

Example:
The most ever members online was 16, Today at 09:06 PM. The most ever guests online was 4, Today at 09:08 PM.
=> Most users ever online was 35, Today at 09:10 PM.

[Ronseal]

This Mod appears to be working fine on my Forum Index, but still shows far too many guests on the VBadvanced portal page....... is it another case where I just have to wait?

I've added the code as advised to onlineusers.php but it doesn't seem to have made any diffence.

[Ronseal]

Sorry to bump the thread but has anybody any ideas why this Mod doesn't work on my portal page?

[mfyvie]

This Mod appears to be working fine on my Forum Index, but still shows far too many guests on the VBadvanced portal page....... is it another case where I just have to wait?

I've added the code as advised to onlineusers.php but it doesn't seem to have made any diffence.

Hi Ron,

Sorry I can't really help with that. This mod doesn't officially support vbadvanced mod. I just added that code as a courtesy. However, others have got it to work. If you see no difference at all you may have inserted the code into the wrong file, or the wrong section of the file. I suggest you go back and start from the beginning and try and debug it yourself. Sorry.

[Ronseal]

Thanks for the reply mfyvie but I have to admit I'm stumped, I've added the code as advised but it doesn't appear to correct the count on the vbadvanced portal page. Can anybody else spot anything I've done wrong, here's where I've added the code and I've also tried it without the line spaces above and below?
FROM " . TABLE_PREFIX . "session AS session
LEFT JOIN " . TABLE_PREFIX . "user AS user USING (userid)
WHERE session.lastactivity > " . (TIMENOW - $vbulletin->options['cookietimeout']) . "

" . ($killspiders ? " AND " . TABLE_PREFIX . "session.spider = -1 " : null) . "

ORDER BY username ASC
");
Many thanks
Ron

[mfyvie]

Hi Ron,

I can only see two possibilities -

1. You may be editing the wrong file. Try to deliberately insert a syntax error and see what happens. If nothing changes, you may be editing the wrong file. Try renaming the file, just to make sure that everything crashes (then rename it back)
2. The variable $killspiders must be set to true to see any difference. This variable is set by the mod itself earlier on, so should be true (unless you've disabled it). I could not tell you why it is not set to true, but you could try to force is by inserting the following line earlier in the file:

$killspiders = true;


Apart from those tips I really have no idea without looking at and playing with your system directly.

[Ronseal]

mfyvie, you've done it! :D
I simply added the killspiders code you've mentioned at the top of the 'onlineusers.php' file and hey-presto it works perfect. I'm not sure where this setting is supposed to be set, but for the time being untill I find out where, everything works perfect. Although just to clarify I hope this doesn't prevent spiders from crawling my site and indexing it, I just want to remove spiders from the online users count.

Many thanks mfyvie!

[mfyvie]

Ron, I'm glad you used this trick to at least verify that your edits are working.

Here is the code that sets this variable - it runs from the hook global_start, and therefore should be included on all vbadvanced pages as well.

$killspiders = $vbulletin->options['enablespiders'];

I suspect that you may have disabled the vbulletin setting "Enable spider display". If vbulletin does not identify connections as spiders or not, this mod cannot work correctly.

[Ronseal]

'Enable Spider Display' is set to 'Yes' and always has been as far as I recall, and I'm not sure what you mean by the hook 'global_start'. I've searched for a reference to this code in a few of my vBadvanced CMPS templates and I can't find any sign of it so I assume I'm looking in the wrong place.

[mfyvie]

Hi Ron, if you look in your vbadvanced files you'll see a line that reads like this:

require_once('./global.php');

If you then look in global.php you'll find a line that looks like this:

($hook = vBulletinHook::fetch_hook('global_start')) ? eval($hook) : false;

The code in my previous post fires at global_start, so in other words, if that setting is set to on as you say, then the variable should be set, and you shouldn't have to set it manually. Why it doesn't work is a mystery to me :)

[Ronseal]

Hi mfyvie, As you've already anticipated my 'global.php' file does indeed have the line
($hook = vBulletinHook::fetch_hook('global_start')) ? eval($hook) : false;
the same as yours, so it'll have to remain a mystery for the moment why is doesn't work. However I do run vBSEO 3.0.0 on my site so perhaps that has some involvement somewhere. I haven't explored all the options in vBSEO but if I find any potential conflict I'll let you know.... but for the meantime your manual edit to the 'onlineusers.php' appears to work fine.

Thanks again
Ron

[James Argo]

/\ /\ /\
Click installed! :)

Beautifull :)

[Milktruck]

For some reason it doesn't seem to be working me as far as Yahoo is concerned... maybe I've done something wrong.

Guests where the status has not yet been determined:
05:15 74.6.20.156 /showthread.php?t=26617 Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
05:30 74.6.24.110 /showthread.php?t=25687 Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
05:46 74.6.7.80 /showthread.php?t=26365&page=2 Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
06:06 74.6.24.100 /showthread.php?t=26212&page=9 Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
06:13 74.6.23.233 /forumdisplay.php?f=327 Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
06:28 74.6.28.164 /showthread.php?t=26353 Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)

I've got a whole bunch more Yahoo entries in the list.. any idea why they are staying put in my guest list? This plugin seems to work for all of the others.

[piraterevival]

Added and installed, nice one

[Marco van Herwaarden]

*** Staff note: The author of this modification has passed away in a diving accident (http://www.englishforum.ch/announcements/20529-mourning-loss-friend-leader-innovator-genuinely-nice-guy.html). We wish his family all strength in dealing with this traggic issue. ***

[r5e]

I've just granted permission to another coder (https://vborg.vbsupport.ru/member.php?u=211414) who has agreed to take over the code from Mark's mods (https://vborg.vbsupport.ru/member.php?u=190317) (like this one) and keep them up to date and free to all, which should be good news for all of you who use his mods and allow his legacy to continue.

Robert Fyvie

[Sonyuserforum]

I've just granted permission to another coder (https://vborg.vbsupport.ru/member.php?u=211414) who has agreed to take over the code from Mark's mods (https://vborg.vbsupport.ru/member.php?u=190317) (like this one) and keep them up to date and free to all, which should be good news for all of you who use his mods and allow his legacy to continue.

Robert Fyvie
Anything new here?

[r5e]

Nothing major to report that is new, but PHPKD (https://vborg.vbsupport.ru/member.php?u=211414) has been a little slower than I expected to take it up. He made an announcement a while back saying that he had started work on this, but I can't find it now. Try contacting him directly if you want more info.

[Omranic]

Hello every body,

Sorry for any inconvenience delay.
This product has been scheduled to be released two weeks ago, but since there was some internal & external problems with PHP KingDom's teamwork we've forced to postponement the release date.
But now every thing is going well, thanks GOD.
We've finally finished working on this product & it will be released for all supported vBulletin versions by tomorrow Thursday 08-01-2009 06:00 PM GMT. Yes, tomorrow. It's the final date we think.
Thanks for your patience & sorry for that delay.

Sincerely Regards
PHP KingDom

[Omranic]

Hello every body,

Sorry for any inconvenience delay.
This product has been scheduled to be released two weeks ago, but since there was some internal & external problems with PHP KingDom's teamwork we've forced to postponement the release date.
But now every thing is going well, thanks GOD.
We've finally finished working on this product & it will be released for all supported vBulletin versions by tomorrow Thursday 08-01-2009 06:00 PM GMT. Yes, tomorrow. It's the final date we think.
Thanks for your patience & sorry for that delay.

Sincerely Regards
PHP KingDom
Follow-up.
According to users spotlights around this product & user's feedback received, we've decided to include some more features such as separating spiders in the "who is online" without neglecting them, also an option to reset the "maximum online users" any time from admin control panel. There is also other new features that's currently in progress, we preferred not to release the product before implementing these new features.
As we said before, the base product has been completed already, but will be released after implementing these new features. We will update this thread with further details & with release time (which should be very very soon).
Thank you for your patience.

[Sonyuserforum]

As we said before, the base product has been completed already, but will be released after implementing these new features. We will update this thread with further details & with release time (which should be very very soon).
Thank you for your patience.
??? Anybody here ???

[Birched]

I can't find the equivalent to this for 3.8.4. Is anyone running it in 3.8.x?

It looks like the developer who took this particular mod over disappeared.

Thanks.

[Claverhouse]

It's working on 3.8.4...

[Stormtrooper]

Is anyone working on something similar for 4.x?