Major Additions - Share users database among many forums

This is a feature requested many times, and we are still waiting, so I released a patch to be able to run many forums with the same user database.

This is an EXPERIMENTAL PATCH I'm running on my forums.

--- ../../../private/upload/includes/class_core.php     2008-07-10 11:17:32.000000000 +0200
+++ class_core.php      2008-07-11 17:44:20.000000000 +0200
@@ -405,6 +405,33 @@
                $this->connection_recent =& $link;
                $this->querycount++;

+               $shared_tables = Array("user","useractivation","userban","usertitle","userchangelog","usertextfield","userfield","usergroup","userlist","usernote","profilevisitor","reputation","reputationlevel","spamlog","strikes","pm","pmreceipt","pmtext","avatar","profilefield");
+
+               $main_prefix = "FP_";
+               foreach ($shared_tables as $shared_table)
+               {
+                       $orig = TABLE_PREFIX.$shared_table;
+                       $dst  = $main_prefix.$shared_table;
+
+                       $this->sql = str_replace($orig." ",$dst." "  ,$this->sql);
+                       $this->sql = str_replace($orig.".",$dst."."  ,$this->sql);
+                       $this->sql = str_replace($orig."\r",$dst."\r",$this->sql);
+                       $this->sql = str_replace($orig."\n",$dst."\n",$this->sql);
+                       $this->sql = str_replace($orig."\t",$dst."\t",$this->sql);
+
+               }
+

                if ($queryresult = $this->functions[$buffered ? 'query' : 'query_unbuffered']($this->sql, $link))
                {

You can add "session" to $shared_tables if you want to share your sessions among forums under the same domain (Thanks Lionel)

This is only useful for new forums joining anothers one user database.

Requirements:

• An old forum where all users are.
• Sharing the same database, using diferent prefixes for tables (TABLE_PREFIX)

For example: Imagine we have FORUM_A , FORUM_B, and FORUM_C each one with their respective licenses.

FORUM A has table prefix FA_ in database, FORUM_B has FB_ and FORUM_C has FC_

FORUM A is the main, and older forum, where we have all our users.

then we would install FORUM_B and FORUM_C normally, using the same database that FORUM_A uses, but setting up their own table prefixes.


once FB_ and FC_ are installed then we go patching class_core.php from FORUM_B and FORUM_C setting up $main_prefix = "FA_"; so they use FORUM_A for shared tables.

Possible problems:

• memcached issues?, not sure
• avatar/signature pics/etc issues when storage is set to disk.
• more to come...

Author: Optimizacion Web

[tfw2005]

So, if you have 4 forums active, with members and posts now, you would first need to merge the databases manually, making sure each have their own prefix in the new mega DB. Then, somehow import just users from the other 3 into the primary one, and merge accounts for duplicate users. Then install the script, and use primary site user tables from there forward. That sound right?

Second, how is performance affected? I have several 1 million+ post forums, probably 100,000 members between them. After combining it all, the DB would be 4-6 gigs. Those are some large table scans. When server load is heavy now, I get issues with a 1.5 gig DB being accessed repeatedly (getting new server, but still).

So, yes, I know you need a solid server just to handle a large DB situation in general, but, how much overhead does this script use, and could it "crap out" if dealing with large amounts of data like this. Several 1000+ users online sites accessing the same DB concurrently.

Disk based avatars and profile pics need to be addressed, as anyone with a lot of members most likely moved to that. Xcache, memcached, eaccelerator issues need to be tested for sure. I couldn't use that until it was in some way.

Question - I take it this does not "log you in" to another site, on another domain, it just makes your user account available to log in again if you go there, correct? So you are logged in and viewing site A, then jump to site B, you would need to log into site B then again. Site A and B are different domains. Correct?

RE: Mysql view, I looked into that, and many people online say that it is not designed for high volume usage, and could actually slow things down. Tho, I just quickly scanned, and don't have deep knowledge of it in general. I'd like to hear how the view technique would work in the above situation too.

And, for the record, I would love VB to have something built into the backend for this officially, more integrated and optimized. Would allow for some really cool stuff.

[mangel.ajo]

Originally Posted by tfw2005

So, if you have 4 forums active, with members and posts now, you would first need to merge the databases manually, making sure each have their own prefix in the new mega DB. Then, somehow import just users from the other 3 into the primary one, and merge accounts for duplicate users. Then install the script, and use primary site user tables from there forward. That sound right?

Hmmm, that's the most complex situation, joining forums that are already running.

First of all, you would have to merge all the users in the Primary forum database, and then -somehow- manage to change all your userid's from pms, posts, threads, and so one to the new ID of the joined database. I'm not sure but that could require specific scripts.
[/quote]

Second, how is performance affected? I have several 1 million+ post forums, probably 100,000 members between them. After combining it all, the DB would be 4-6 gigs. Those are some large table scans. When server load is heavy now, I get issues with a 1.5 gig DB being accessed repeatedly (getting new server, but still).

It should add len(shared_tables)*4 str_replaces to every SQL query your system is doing. Probably is not too much, but it's load, of course. (We could do some tests to discover how much is that.)

We should also think if mysql would also penalyze us by having all the tables in one database with same prefix.

So, yes, I know you need a solid server just to handle a large DB situation in general, but, how much overhead does this script use, and could it "crap out" if dealing with large amounts of data like this. Several 1000+ users online sites accessing the same DB concurrently.

We will have to analyze that. May be running some test over real vbulletin querys and measuring time.

Disk based avatars and profile pics need to be addressed, as anyone with a lot of members most likely moved to that. Xcache, memcached, eaccelerator issues need to be tested for sure. I couldn't use that until it was in some way.

I've been thinking more deeply about memcached / xcache an so on, and probaby it won't be a problem, but we must test.

Question - I take it this does not "log you in" to another site, on another domain, it just makes your user account available to log in again if you go there, correct? So you are logged in and viewing site A, then jump to site B, you would need to log into site B then again. Site A and B are different domains. Correct?

Yes, exactly, that's it. Don't know if it would be possible to develop some kind of mechanism to move around forums using the different session tables to move around forums.

RE: Mysql view, I looked into that, and many people online say that it is not designed for high volume usage, and could actually slow things down. Tho, I just quickly scanned, and don't have deep knowledge of it in general. I'd like to hear how the view technique would work in the above situation too.

I've never tested it, and for the simplest idea (having the mirror of a diferent table it should'nt be so slow. But analyzing VIEW in depth you can see it let's you make some complex views selecting some rows in tables, and that its a complex situation, and being prepared for that could be the reason of slowliness.

And, for the record, I would love VB to have something built into the backend for this officially, more integrated and optimized. Would allow for some really cool stuff.

Absolutely: me too.

[Datenpapst]

Hi,
is it possible that maybe some forums like off topic or so can also be used by more than 1 forum?

[Super Jinni]

does that work if the new forum in another server and have completely another domain?

[Tekmon]

I just want to make sure... This is for Seperate URL's/Licenses to share the same User tables. Correct?

When I download the file I get a diff file and I do not know what to do with it.

Thanks

[Tekmon]

Could you please tell me where in class_core.php I put the code?
There seems to be many sections that this could go in.

I am guessing I put in the code with the "+"'s on the front of the line and
to remove those "+"'s as well?

Thanks

[takerman]

hello,

or should he put the php codes I put in class_core but I have this error

Parse error: syntax error, unexpected $end in /home/www/fd0ea7be7c752c55ff235159345d0e2a/web/tnawrestling/includes/class_core.php on line 3326

sorry for me english

[TraumTeam]

Is this the project to run several different Forums (search, new Thread, Categories...) but with the same userdatabase and login?

I requestet this about 5 years ago in vbulletin support forum and i got the answer, that this wont work.

It is very interesting for sites with very different content categories. This is a very nice try and i hope you get it done.

best regards
rob

[lovevn]

What should I do if my main forum used no prefix?Shoud I use this:
$main_prefix = " ";

[dazed12]

i installed my main forums without a prefix. im having probs getting it to work. also do you leave the + on the code?