Maximum Forums?

What's the maximum number of forums supported?

I've added 1100+ forums and am now getting a MySQL error when attempting to add any more (MySQL Error : Got a packet bigger than 'max_allowed_packet' bytes Error Number : 1153). It gets this error when trying to update the serialized data in the "datastore" table.

The particular setup we need requires a minimum of 10,000 forums, so running out at 1000 is a bit disconcerting. And yes, we really need that many forums - it's one for each "product" and each product will need it's own threads.

Is there a way (plugin/etc) to remove the cacheing for the forums or at least optimize it better? We thought of running multiple databases, but a couple would need 3k forums, so that's not helpful either (and user signon implications would be bothersome too). Also, increasing max_allowed_packet would just delay the problem.

Thanks!
-steve

[Yours]

That is rather large I would have to say. That many forums would be a bit much for me; but back to the subject.

The only way I can think of is by increasing your max_allowed_packet yourself, and ask your webhost to do the same. The host you have is just setup to have much smaller MySQL packet sizes.

Now about a mod I am not possitive that there is an existing mod that can do what you are asking for.

[RevBooyah]

It's a dedicated beefy machine just for the forums. The original plan was to develop forum software all our own, but vB gets such great reviews we thought we'd give it a try.

Changing the MySQL options is possible, but it's not a long term solution, since it's set pretty high already. It seems like the datastore table is holding ALL of the existing forums as a serialized object. That's cacheing a lot of unnecessary stuff IMHO .

I guess I can rewrite the cacheing - it'll be easier than writing all new forum software.

Some other forum software could handle the # of forums, but got extremely slow... So, we'll see.

Thanks!
-steve

[Yours]

As far as I am concerned it isn't a forum software issue its just the MySQL issue.

As for the MySQL edit being a short-term fix I honestly think it would be a fairly long term fix if you set it to a super high extreme.

Another option would be to just not have as many forum categories or having whole seperate subdomains for this (costing you extra for more licenses of course)

[Eikinskjaldi]

When creating that many forums and sub-forums, you'll likely have horrendous load times and high memory usage not just from the querying, but also for what's loaded into the forumcache. Are you certain that your needs of categorization/compartmentalization cannot be handled by using something like the Thread Prefixes hack?

[Marco van Herwaarden]

Even though vBulletin supports "unlimited" forums, in real life more then 1.500-2.000 forums will put a too high pressure on the server (performance and memory). The real maximum depends on the server and the server configuration. As vBulletin 3.x will cache each forum in the datastore (and will load this information into memory on each page load), you would quickly run out of memory.

Some boards that use this many (sub)forums are still using vBulletin 2.x only because that version does not do forum caching and will preserve memory in cases like this. One of the things that will be focussed on when developing vBulletin 4 (there is no indication on a date for that) is the reduction of memory usage for large boards.

what would be good here is to explain the goal of having 10 000+ forums on your site... maybe a better structure and ghost forums would do the trick... i had a lot of clients requesting millions of forums, and after some discussions, the target was more human in nature ... sometimes the goals are not reached by the number but by the structure and a good administration protocol...

[macwiz]

I'm also contemplating creating several thousand forums. Would it make a difference if members only have permission to view a very small number of these? For example - there are 6000 forums, but any given member only sees 10-20 based on permissions. Would this save the server in terms of memory / performance? Thanks.

[Dean C]

It wouldn't make any difference at all no. Do you know how difficult it'd be to manage that many forums anyway!

when you have a US-states-based forum, with a sub-forum for each state on each of your hundred forums, you can easily split your count to 50 times less forum by using thread-prefixes instead of sub-forums... (each state is a prefix)

using prefixes as identifiant for topics will cut down the need to have forums as identifiants... you have the best example here on vb.org... they use prefixes to identify each type of hack release here, so instead of having 15 sub-forums for each vb version, there is 15 prefixes for each version... and you can choose to display only prefix X or Y, so it helps making more seperation on topics...

the worst way to manage big forums is to split the content by forums...