This is a great idea and an interesting implementation method. There are lots of potential for uses, and would really reduce server load if it is possible to complete remove the need to connect to the database for normal thread perusing by guests, spiders or users. It would be most useful for close and old threads, which do not need dynamic updates of post information.

I look forward to updates on this concept. :)

Well said Erwin my thought exactly. I have said many times that vbulletin needs a true archive which would take closed threads or older threads and put them in flatfiles to rerad from there. This seems to be taking it one step further. To have all threads read from static files would reduce the server load imensly. The script itself after the initial run would be resource intense, but, once it only has to do a few threads say every 15 minutes it wouldn't be bad at all.

I hope this becomes a reality. Someone started something like this for vb2 and never finished it.

so be it... i'm happy my questions are answered well... ;) - i'm a bit devil's advocate so i ask strange things!

... this hack answer my need of an archive way to do things... hope to see the ACP soon!

feature request: archive a complete thread in zip or tar when the topic is closed or deleted... so we can make a followup.. ;)

hmm, i think i know what i'll do in my Holidays ^^

i'll port my vb Archive to vb3 ;)

Actually to get back to this hack, i'm with erwin, that seems to be a great idea.

also, once we're in... would be good to have a specific archive browser for this cache... so if we want to keep these cached threads in a different pattern, we can have a different template to display them... i like that idea, not used for everyone though!

I actually hadnt thought of that erwin, my general idea was for it to be seamlessly integrated into the forum itself, but now that you mention it. Really is a great idea, considering not having to use "any" connection.

Also, if mysql went down the potential would be there for people to still read thread in some cases, if I can put together a script that uses Vbulletins code to reproduce a queryless forum.

Ps. anyone got any ideas on reducing memory usage, or about the increase in file size sequentially. I know its not that big of a deal as we can adjust the sizes for memory limit... but I want this hack to be available to everyone, not just those that can manually tweak their settings.

Basically, you would be creating a new way of storing threads and posts in parsed html in files, rather than in the mysql database. :) Very interesting concept! vB3 can already store parsed HTML in the database to save on queries and load, but imagine if you could do this in real-time in files, and really only update the threads daily if a new post is posted... you would cut down server load by a lot, especially for large sites like mine, by eliminating database queries altogether for normal thread reading! :)

Some of you are stating that this would reduce server load, but I am confused by this. Programs like UBB Classic originally used a flat-file type system for the entire board. They then upgraded it to "Threads" that uses MYSQL. Threads can handle many more simultaneous users and posts. Itsn't that what also makes VBulletin able to handle many more users & posts? I previously used UBB which did use flat files and it was much more server intensive.

-Victor

PS: Perhaps this is only true if users do searches/etc? I'm almost certain MYSQL will use less resources on a search than flat files.

You are correct vissa in the search aspect, although I have not done realtime benchmarks im almost certain that a fulltext search in mysql would be faster than an ereg or array manipulation THEN search inside the result of a flatfile.

But that is not what this script is doing, you will still be using mysql to search, the post will still be stored in mysql. This script will only make a change to the part of vbulletin that actually shows the post in which case will be drawn from a flatfile WHICH will be faster and less CPU intensive than a query of result rows from MySQL, I should also mention using less Memory.

In your regardst to threads, i doubt that is wh at UBB is referring to. They probably just coined that name in reference to "threads" as in topics. "Real" "Threads" are kernel dependent things. Depends on if your BSD/*Nix/etc build supports "Threading" which extended support can be compiled from within MySQL if you build it yourself.

To support optimized MultiThreading you have to build a kernel with modified File Handles, a new Glibc and latest and greatest LinuxThreads module then build your MySQL using other-libc its a pretty complicated process but lets you optimize the range of MySQL of about 400% or more. In tests ive done as well as various books, its shown that doing a MultiThread modfication can take a fully optimized MySQL from being able to handle several hundred active connections to well over 4000, limited only by your file handle count.

Only problem is, a lot of people dont know that trick and instead rely on getting multiple DB servers, when they can solve their problem with a semi-complex recompile.

Back to the subject, flatfile will always be faster for fetching results, and since these are done in read mode, they can support multiple file reads at once, there is no locking when done with reading.

Interesting,

but will discspace grow huge?

For instance: I have almost 900.000 posts and 60.000 threads.
If one cached thread takes up 30kb, it will cost me around 1800mb discspace.

That's almost twice my database size?