Go Back   vb.org Archive > vBulletin Modifications > Archive > vB.org Archives > General > Big Board Discussions
FAQ Community Calendar Today's Posts Search

Reply
 
Thread Tools
How did you solve searching in large forums? Details »»
How did you solve searching in large forums?
Version: , by 99Percent 99Percent is offline
Developer Last Online: Jul 2013 Show Printable Version Email this Page

Version: Unknown Rating:
Released: 06-21-2006 Last Update: Never Installs: 0
 
No support by the author.

We are experiencing server performance when a user searches through our fairly large forum. The fulltext search aparently locks thread and post tables causing Apache processes to wait until the search is done. This in turn causes a surge of new Apache connections to open. If the search takes too long everyone experiences a downtime of the forum while the search is done.

What is the solution to this? How did you solve it?

The only thing I can think of right now is to install another instance of mysql on this same server and configure it as slave with replication. This instance would have its own database files and a different port.

Then configure vBulletin to use the slaveserver for the search script.

However I notice that vBulletin (in 3.5.3, function db->connect) checks to see if the master and slave have different names. This will not work if both of them are set to localhost even if they are using different ports. So at least a hack will have to be done here.

Show Your Support

  • This modification may not be copied, reproduced or published elsewhere without author's permission.

Comments
  #22  
Old 08-16-2006, 10:30 PM
Troy Roberts Troy Roberts is offline
 
Join Date: Mar 2003
Posts: 21
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Unfortunately I paid someone else to set it up so I don't have all the detailed tech info. It was a feature that I had back when I used zeroforum and wanted in vb. It really should be built in to vbulletin.

You can see it in action at http://forums.corvetteforum.com
Reply With Quote
  #23  
Old 08-17-2006, 12:38 AM
Joe Joe is offline
 
Join Date: Oct 2001
Posts: 75
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Thanks Troy.
Reply With Quote
  #24  
Old 08-29-2006, 06:02 AM
Erwin's Avatar
Erwin Erwin is offline
 
Join Date: Jan 2002
Posts: 7,604
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

The way I solve this is to offload search to a slave database that replicates the relevant tables. 26 million posts.
Reply With Quote
  #25  
Old 08-29-2006, 01:54 PM
Troy Roberts Troy Roberts is offline
 
Join Date: Mar 2003
Posts: 21
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

We used to use a dedicated search machine. That can work very well also.

One benefit we found with the archived data method is that if we run into a problem on the live data it's a lot easier to run utilities to fix it. A smaller database is easier to fix when things go wrong.

The archived data is not as likely to have problems since it is read only. Of course you do have other drawbacks like not being able to edit/delete a thread through normal methods once it is archived.
Reply With Quote
  #26  
Old 09-01-2006, 01:53 AM
lazytown lazytown is offline
 
Join Date: Feb 2004
Posts: 503
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Quote:
Originally Posted by Erwin
The way I solve this is to offload search to a slave database that replicates the relevant tables. 26 million posts.
Does this require a separate server or is there still a benefit to doing this on the same server that houses the forum? I would imagine it would free up table locks, etc even if it was on the same server.

-vissa
Reply With Quote
  #27  
Old 09-03-2006, 10:08 PM
Erwin's Avatar
Erwin Erwin is offline
 
Join Date: Jan 2002
Posts: 7,604
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Different server. You could put it on the same server to free up table locks, but make sure you have enough CPUs/RAM to handle it. I prefer to offload it to a different box.
Reply With Quote
  #28  
Old 09-04-2006, 07:29 PM
Dontom Dontom is offline
 
Join Date: Oct 2001
Location: Germany
Posts: 115
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Hi Erwin,
thanks for sharing. we already have a slave db server and would like / need to offload search there due to the locking issues (3M posts)

How did you tellvb to use the slave for searching? did you modify config.php or search.php? Any help is appreciated,
Thomas
Reply With Quote
  #29  
Old 09-05-2006, 03:12 AM
Erwin's Avatar
Erwin Erwin is offline
 
Join Date: Jan 2002
Posts: 7,604
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Different ways to do it - for me, I modified search.php quite extensively (I'm running a vB-based Wiki off that slave too).
Reply With Quote
  #30  
Old 09-06-2006, 04:32 AM
kerplunknet kerplunknet is offline
 
Join Date: Jul 2002
Posts: 50
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Erwin, where could I learn more about MySQL replication? If we wanted to offload search to a separate database server, we would need to use MySQL replication, correct? The search database server would be a slave server. I am unsure how to set this up. Any advice? Tips? Pointers? URLs? Thanks.
Reply With Quote
  #31  
Old 09-06-2006, 04:37 AM
Dontom Dontom is offline
 
Join Date: Oct 2001
Location: Germany
Posts: 115
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

<a href="http://dev.mysql.com/doc/refman/5.0/en/replication.html" target="_blank">http://dev.mysql.com/doc/refman/5.0/en/replication.html</a>
Reply With Quote
Reply


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT. The time now is 05:01 PM.


Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.
X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.07446 seconds
  • Memory Usage 2,314KB
  • Queries Executed 26 (?)
More Information
Template Usage:
  • (1)SHOWTHREAD
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)ad_showthread_beforeqr
  • (1)bbcode_quote
  • (1)footer
  • (1)forumjump
  • (1)forumrules
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (1)modsystem_post
  • (1)navbar
  • (6)navbar_link
  • (120)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (11)post_thanks_box
  • (11)post_thanks_button
  • (1)post_thanks_javascript
  • (1)post_thanks_navbar_search
  • (11)post_thanks_postbit_info
  • (10)postbit
  • (11)postbit_onlinestatus
  • (11)postbit_wrapper
  • (1)spacer_close
  • (1)spacer_open
  • (1)tagbit_wrapper 

Phrase Groups Available:
  • global
  • inlinemod
  • postbit
  • posting
  • reputationlevel
  • showthread
Included Files:
  • ./showthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/functions_bigthree.php
  • ./includes/class_postbit.php
  • ./includes/class_bbcode.php
  • ./includes/functions_reputation.php
  • ./includes/functions_post_thanks.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • showthread_start
  • showthread_getinfo
  • forumjump
  • showthread_post_start
  • showthread_query_postids
  • showthread_query
  • bbcode_fetch_tags
  • bbcode_create
  • showthread_postbit_create
  • postbit_factory
  • postbit_display_start
  • post_thanks_function_post_thanks_off_start
  • post_thanks_function_post_thanks_off_end
  • post_thanks_function_fetch_thanks_start
  • post_thanks_function_fetch_thanks_end
  • post_thanks_function_thanked_already_start
  • post_thanks_function_thanked_already_end
  • fetch_musername
  • postbit_imicons
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • postbit_display_complete
  • post_thanks_function_can_thank_this_post_start
  • pagenav_page
  • pagenav_complete
  • tag_fetchbit_complete
  • forumrules
  • navbits
  • navbits_complete
  • showthread_complete