vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 3.0 Full Releases (https://vborg.vbsupport.ru/forumdisplay.php?f=33)
-   -   Fulltext boolean search v.2.2 for vB (https://vborg.vbsupport.ru/showthread.php?t=62282)

kmike 04-15-2004 08:50 AM

Thinking about two post tables idea, it could go really well along with mysql replication.
Idea: set up second mysql server (even on the same physical machine), set it up to replicate post table from main forums db. All updates to that second mysql server will be handled by only one slave thread, and it could be locked freely without any effect on first mysql server perfomance.
Since JohnWoo's hack uses two tables in the main query, it appears that thread table need to be replicated along with post table.
Changes to current hack code would be really small: initialize another db connection and fire query to a second db instead of main one.

I don't plan to implement this any time soon, though accidentally we have workng replication of our forum db to another server. I'm polishing a vb2-like hack of this (JohnWoo's) hack :) . It eliminates LEFT JOIN and always searches IN BOOLEAN MODE. Had to drop search by relevance though, I highly doubt someone used it even once though.

cerebro 04-16-2004 08:23 AM

i will try..and see what happen. Allway can come back .

cerebro 04-16-2004 09:43 AM

i dont have to wait to much...

and work Fine whit 3.0.1

mysql> ALTER TABLE post ADD FULLTEXT (title);
Query OK, 1762278 rows affected (4 min 2.67 sec)
Records: 1762278 Duplicates: 0 Warnings: 0

mysql> ALTER TABLE post ADD FULLTEXT (pagetext);
Query OK, 1762278 rows affected (14 min 59.70 sec)
Records: 1762278 Duplicates: 0 Warnings: 0

mysql> OPTIMIZE TABLE post;
+----------+----------+----------+----------+
| Table | Op | Msg_type | Msg_text |
+----------+----------+----------+----------+
| vb3.post | optimize | status | OK |
+----------+----------+----------+----------+
1 row in set (1 min 21.15 sec)

cerebro 04-18-2004 09:51 PM

mmm same problem whit server load...
Treads: 112,110, Post: 1,745,876, Members: 29,111

First day work fine whit 400 users online, second day server go to 8...11.11!!!!!!

i have to uninstall...some fix?

cuerty 04-22-2004 03:06 AM

It's great, mysql load average goes as down as it can ;-) Thanks.

Erwin 05-01-2004 06:27 AM

I've installed this!

It's working well... I'll look over the server loads over the next few days and report on it.

Overall, took me only 3 hours... most of the time it was trying to find out why I couldn't add a fulltext index to post, to find out that my secured tmp directory was too small, so I had to modify my.cnf to change the tmpdir temporarily. :)

JohnWoo 05-01-2004 10:00 AM

Sorry for disappearing :)
Returning to discussion about SQL requests with or without LEFT JOIN :)
I recently had a chance to do a lot of tests on one large programming forum with about 900.000 posts. And yes - it is true that excluding some forums from searches do not make search run faster... But including left join don't make it perceptible slower :)
But after removing revelance it runs up to 10 times faster!
Here are results as requests and time in seconds below. Each request was executed 10 times with clearing DB cache after each. Fastest and slowest time below. Forums were not closed at that moment and there were about 120 online users. Think that numbers too different because of it :)
Code:

============
SELECT postid
FROM post AS post
LEFT JOIN thread AS thread ON post.threadid = thread.threadid
WHERE MATCH (
pagetext
)
AGAINST (
'+user +acces'
IN BOOLEAN
MODE
)
LIMIT 0 , 200

0.0277 -  0.3782 s

============

excluding security forums

============
SELECT postid
FROM post AS post
LEFT JOIN thread AS thread ON post.threadid = thread.threadid
WHERE MATCH (
pagetext
)
AGAINST (
'+user +acces'
IN BOOLEAN
MODE
) AND thread.forumid NOT IN (57, 64)
LIMIT 0 , 200

0.0197 - 0.2272 s
============

============
SELECT postid
FROM post AS post
WHERE MATCH (
pagetext
)
AGAINST (
'+user +acces*'
IN BOOLEAN
MODE
)
LIMIT 0 , 200

0.0144 - 0.1043 s
============

============
SELECT postid,
MATCH (
pagetext
)
AGAINST (
'user acces'
) AS relevance
FROM post AS post
LEFT JOIN thread AS thread ON post.threadid = thread.threadid
WHERE MATCH (
pagetext
)
AGAINST (
'+user +acces*'
IN BOOLEAN
MODE
) AND thread.forumid NOT
IN ( 57, 64 )
LIMIT 0 , 200

0.6938 - 1.3414 s
============

============
SELECT postid,
MATCH (
pagetext
)
AGAINST (
'user acces'
) AS relevance
FROM post
WHERE MATCH (
pagetext
)
AGAINST (
'+user +acces*'
IN BOOLEAN
MODE
)
LIMIT 0 , 200

0.5509 - 1.4414 s
============


Erwin 05-01-2004 10:56 AM

Well, my current server loads seem a bit lower than before I installed the hack. But it's not peak time yet. Search appears to be noticeably faster.

John, just tell me which code to use that is the fastest and puts the least amount of stress on the server, and I'll use it. :) Do I just remove the RELEVANCE part in search.php?

JohnWoo 05-01-2004 11:40 AM

yes :) to remove revelance replace

Code:

//fulltext search query
        $fulltext_sql = "SELECT postid,MATCH (".$what_field.") AGAINST ('".$norm_query."') as relevance FROM ".TABLE_PREFIX."post AS post LEFT JOIN ".TABLE_PREFIX."thread AS thread ON post.threadid=thread.threadid WHERE MATCH (".$what_field.") AGAINST ('$query'".iif($vboptions['allowwildcards'] ==1 && preg_match("~[\"\-\*+]~i", $query), ' IN BOOLEAN MODE', '').")".$limit_conditions." LIMIT 0, ".$vboptions['maxresults'];

with

Code:

//fulltext search query
        $isboolean="'";
        if ($vboptions['allowwildcards'] ==1 && preg_match("~[\"\-\*+]~i", $query)) {
                $isboolean="' IN BOOLEAN MODE";
        }
        $fulltext_sql = "SELECT postid FROM ".TABLE_PREFIX."post AS post LEFT JOIN ".TABLE_PREFIX."thread AS thread ON post.threadid=thread.threadid WHERE MATCH (".$what_field.") AGAINST ('".$query.$isboolean.") ".$limit_conditions." LIMIT 0, ".$vboptions['maxresults'];

and remove string
Code:

$postscores[$thispost["postid"]] = $thispost["relevance"] * $vboptions['multimatchscore'];
:)

cerebro 05-01-2004 04:22 PM

Thanks again JohnWoo...just modify that in my file....in a few days will see :D


All times are GMT. The time now is 01:45 AM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.02605 seconds
  • Memory Usage 1,742KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (4)bbcode_code_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (1)pagenav_pagelinkrel
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (10)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete