vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 3.0 Full Releases (https://vborg.vbsupport.ru/forumdisplay.php?f=33)
-   -   Fulltext boolean search v.2.2 for vB (https://vborg.vbsupport.ru/showthread.php?t=62282)

Erwin 05-08-2004 12:16 PM

I'm waiting for kmike's version - why reinvent the wheel? :)

kmike, did you just use the vB2 code? Coz if you did, I can try to modify it to fit too.

ImportPassion 05-08-2004 08:39 PM

Here is some excellent info that everyone should look at.

http://dev.mysql.com/doc/mysql/en/Fu...ne-tuning.html

Erwin 05-08-2004 08:42 PM

My database is fine-tuned already. No matter how much you fine-tune, if your database is big, a LEFT JOIN is a bad query to have.

ImportPassion 05-08-2004 09:46 PM

wasn't meant to be about that. About changing the minimum and maximum word length and other stuff like the stop word list.

Erwin 05-09-2004 05:09 AM

Ahhh... thanks. I see now. :) Interesting... do you have any particular settings that work for you?

kmike, do post your code once you have them ready. :)

JohnWoo 05-09-2004 08:17 AM

If you want to remove left joins together with revelance why not to say just
Code:

$isboolean="'";
if ($vboptions['allowwildcards'] ==1 && preg_match("~[\"\-\*+]~", $query)) {
        $isboolean="' IN BOOLEAN MODE";
}
$fulltext_sql = "SELECT postid FROM ".TABLE_PREFIX."post WHERE MATCH (".$what_field.") AGAINST ('$query".$isboolean.") LIMIT 0, ".$vboptions['maxresults'];

? :)
or in one string
Code:

$fulltext_sql = "SELECT postid FROM
        ".TABLE_PREFIX."post WHERE MATCH
        (".$what_field.") AGAINST
        ('$query'". iif($vboptions['allowwildcards'] ==1 && preg_match("~[\"\-\*+]~", $query), ' IN BOOLEAN MODE', '').")
        LIMIT 0, ".$vboptions['maxresults'];


ScottW23 05-09-2004 08:19 AM

Just wanted to say that we installed this hack on our 950,000 post VB3 board with good results so far. Searches were slow with relevance enabled so I disabled it and searches seem snappy.

ScottW23 05-09-2004 08:22 AM

Quote:

If you want to remove left joins together with revelance why not to say just
Yes why not?! <Slaps JohnWoo> :) Works great, even faster now.

JohnWoo 05-09-2004 09:00 AM

Still want to say few words for left joins :)
Lets say that you have large board with many forums, usergroups and complex access level. After it lets suppose that somebody with common access level (just registered) will search for some common phrase (for example "+javascript +flash").
On one board, where i tested it, such search with no left joins return nothing because fulltext query find first 200 matches in forums invisible to beginner members and stop on it. Showresult page (looking on current member access level) hide all that 200 matches from search result and with great enthusiasm say that nothing found :) On same forum search with left joins return complete set of 200 matches (from other forums visible to beginner members) for same search phrase and may return more if "maxresult" will be set to larger number.
But if your forum access level structure is not too complex, you may remove left join part without visible consequence :)

Erwin 05-09-2004 12:13 PM

Awesome, John Woo. Your new query has fixed my slow_query/ mysql locking/ apache crashing problem (as far as I can tell with testing).

I'm using the first code example, just because the original code is like that. :) Thanks!

Erwin 05-12-2004 02:47 AM

Even without the LEFT JOIN, certain phrases do make the query go into my slow_query log, but it's only around 40 seconds now. :) Much better.

Another question for those with this hack - when I use the Advanced Search page, and look for all posts by username, it works.

But when I look for all threads started by username (from the dropdown menu under the box where you put the username) only really old threads come up.

Does this happen to you too? Any ideas?

Erwin 05-12-2004 09:03 PM

Sorry to ask another question - does Search Thread work for you guys? My members are saying it searches the whole site instead of just the thread.

msimplay 05-12-2004 09:14 PM

Quote:

Originally Posted by Erwin
Sorry to ask another question - does Search Thread work for you guys? My members are saying it searches the whole site instead of just the thread.

i can confirm this because i did post about it earlier
but nothing has been done as of yet

cerebro 06-05-2004 09:51 PM

Kmike, can you post that templates?

and JohnWoo, maybe make a new release whit Template MOD too, and left join together.

Please this project dont have to die, is really nice...come on :D

Erwin 06-05-2004 10:41 PM

Quote:

Originally Posted by cerebro
Kmike, can you post that templates?

and JohnWoo, maybe make a new release whit Template MOD too, and left join together.

Please this project dont have to die, is really nice...come on :D

This is the best thing that's come out for large vB sites. :)

cerebro 06-05-2004 11:23 PM

yeah...i know!
i have a 3 Gigabyte Database...insall this hack and reduce to 800 Mg

But something i have to close search for a few day, becauso all forum goind down. I think that the hack need a little more work, but i dont know where. :P

Erwin 06-06-2004 04:01 AM

As some may know, Search Thread does NOT work with this hack installed.

Here is my fix:

Open search.php:

[Removed code - I've made a new piece of code to replace this one that also makes Search Individual Forums work too - refer to post below.]

Erwin 06-06-2004 04:04 AM

Quote:

Originally Posted by 7thgenCivic.Com
u deleted ur post! argh!

The post is back! I made it better! ;) I decided to share my optimization!

ImportPassion 06-06-2004 04:15 AM

problem somewhere

Database error in vBulletin 3.0.1:

Invalid SQL: SELECT postid FROM post WHERE MATCH (post.pagetext) AGAINST ('hood) LIMIT 0, 500 mysql error: You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near ''hood) LIMIT 0, 500' at line 1

mysql error number: 1064

Erwin 06-06-2004 05:23 AM

Quote:

Originally Posted by 7thgenCivic.Com
problem somewhere

Database error in vBulletin 3.0.1:

Invalid SQL: SELECT postid FROM post WHERE MATCH (post.pagetext) AGAINST ('hood) LIMIT 0, 500 mysql error: You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near ''hood) LIMIT 0, 500' at line 1

mysql error number: 1064

Are you sure you copied that line as it is? I just cut and pasted that line EXACTLY AS IT IS POSTED THERE just to test it, and it works on my site.

Morgalis 06-06-2004 07:21 AM

i have +8million lines and 165 megs committed to postindex with +500k posts

i've felt searching drags my server so much, would this be a better alternative?

kmike 06-06-2004 07:24 AM

Quote:

Originally Posted by Erwin
Are you sure you copied that line as it is? I just cut and pasted that line EXACTLY AS IT IS POSTED THERE just to test it, and it works on my site.

Erwin: I believe there's an error in your pasted query here:
AGAINST ('$query".$isboolean.")
unmatched single quota. It should be:
AGAINST ('$query' ".$isboolean.")

ImportPassion 06-06-2004 10:57 AM

Erwin,

Yes copied exactly.

I thik there is a single quote missing here
PHP Code:

AGAINST ('$query".$isboolean.") 


Erwin 06-06-2004 11:55 AM

Quote:

Originally Posted by kmike
Erwin: I believe there's an error in your pasted query here:
AGAINST ('$query".$isboolean.")
unmatched single quota. It should be:
AGAINST ('$query' ".$isboolean.")

Nope, that doesn't work on my site.

The code that I've posted works as it is - don't ask me why. You might have to modify it for your site.

ImportPassion 06-06-2004 01:10 PM

yup, i added the single quote and it works no prob, and fast

Showing results 1 to 25 of 342
Search took 0.06 seconds.

it's extremely weird that it works on your site. it's invalid SQL.

kmike 06-06-2004 02:33 PM

It just means there's matching single quote added either at the end of $query variable or at the beginning of $isboolean somewhere.

cerebro 06-06-2004 02:52 PM

Thanks for update!

[high]* cerebro install :D[/high]

cerebro 06-06-2004 04:43 PM

Nice mod...really nice, help a lot. Thanks Erwin for Sharing.

What d u think about stats cache...for optimize forum home.

https://vborg.vbsupport.ru/showthread.php?t=61581

some one install this hack and bollean search?

Erwin 06-10-2004 01:56 PM

Okay, I've made the hack work for "Search Forum" now as well as "Search Thread" - before, "Search individual forums" did not work.

I've also cleaned up the code a lot more. Plus optimized the search query by removing the JOIN function when it is not required (unlike before when you had a JOIN query for every search for no real reason). I also removed relevance and other bits that slowed the search down and increased server load. Straight searching should be faster with this.

Remove my previous codes.

Then, open search.php and replace the line beginning with:

$fulltext_sql =

with this:

PHP Code:

// Erwin's Add-On
 
$searchlimit " WHERE";
$searchforumadd "";
 
if (
$searchthread) {
$searchlimit " WHERE post.threadid=$searchthreadid AND";
}
 
if (
$forumchoice) {
$searchforumadd ", forumid";
$searchlimit " LEFT JOIN thread AS thread ON post.threadid=thread.threadid WHERE forumid IN($forumchoice) AND";
}
 
$fulltext_sql "SELECT postid$searchforumadd FROM post$searchlimit MATCH (".$what_field.") AGAINST ('$query".$isboolean.") LIMIT 0, ".$vboptions['maxresults'];
 
// Erwin's Add-On 

Some people have had problems with this:

('$query".$isboolean.")

It works for me, but some others have had to modify this bit by removing or adding quotation marks - not sure why.

Hope it works for you! It's working well for me - Search Threads and Search Forums are now working well finally! Plus search is optimized even more!

Erwin 06-10-2004 02:07 PM

Note, when I said removed my previous code above that applies only to people who have applied my prevuous Search Thread bug fix.

When I say the query is optimized even more, it's optimized compared to the original fulltext_sql query - it's basically similar to my previous fix, but written slightly different.

Just to clarify. :)

nexialys 06-10-2004 02:12 PM

so, erwin... when will we see a final version of your own ?!.... it looks promissing, but we don't have the entire package... ;)

msimplay 06-10-2004 02:55 PM

hmm weird i'm sure search relevancy was fixed for 2.2

i can search within forums and threads etc without those modifications

Erwin 06-10-2004 08:35 PM

Quote:

Originally Posted by nexialys
so, erwin... when will we see a final version of your own ?!.... it looks promissing, but we don't have the entire package... ;)

I've only modified the code for my site slightly to remove show results as threads, so all my results are shown as posts, as that's less server intensive than to have to join the threads in.

ImportPassion 06-10-2004 08:53 PM

ya, i don't get it cause it's working fine for me too.

Erwin 06-10-2004 10:04 PM

Quote:

Originally Posted by 7thgenCivic.Com
ya, i don't get it cause it's working fine for me too.

Weird, because it didn't work for me until I made that mod. I am talking about the search form at the top of a forum. The search forum on the Advance Search page does work. It's just the search form at the top of forums that didn't work for me.

msimplay 06-11-2004 06:12 AM

Quote:

Originally Posted by Erwin
Weird, because it didn't work for me until I made that mod. I am talking about the search form at the top of a forum. The search forum on the Advance Search page does work. It's just the search form at the top of forums that didn't work for me.

in 2.2 they all seem to be working for me now then again i uninstalled the hack

and when i saw the advanced you made i reinstalled the hack but i don't get it everything seems to be running in order now without modifying extra

do you have similar threads on by any chance because i don't :P

Erwin 06-11-2004 01:32 PM

Quote:

Originally Posted by msimplay
in 2.2 they all seem to be working for me now then again i uninstalled the hack

and when i saw the advanced you made i reinstalled the hack but i don't get it everything seems to be running in order now without modifying extra

do you have similar threads on by any chance because i don't :P

No, I never had similar threads on.

cerebro 06-30-2004 03:41 PM

Some one have some upgrate or news about new version? jejeje
My board increise a lot...have to change something..CPU is on TOP! :P

Erwin 07-02-2004 03:03 AM

Quote:

Originally Posted by cerebro
Some one have some upgrate or news about new version? jejeje
My board increise a lot...have to change something..CPU is on TOP! :P

The next update of vB3 may have this - not sure.

ShiningArcanine 07-02-2004 04:26 PM

Quote:

Originally Posted by Erwin
The next update of vB3 may have this - not sure.

It is in 3.02 but not supported so the devs won't say how to enable it in 3.0.2. :(


All times are GMT. The time now is 11:11 AM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.01990 seconds
  • Memory Usage 1,838KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (2)bbcode_code_printable
  • (2)bbcode_php_printable
  • (13)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (40)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete