Sphinx Search

[mute]

Yay! That fixed it for me too!

So, I'm thinking if you do plan on cleaning things up and releasing it at some point as a hack, that it would be best to gather up the "settings" into one file or at the top of the sphinx include. For example I have a multi server setup, so I specify the searchd server's ip rather than localhost, and I've renamed my indexes. To the average joe they might not notice or know how to make those changes to get things working. I'm going to do some more testing later on but things are looking very good

[orban]

Yeah...to be honest I intend to do that when 0.9.7 comes out where more than one group is supported natively and things should be a lot cleaner (and prolly faster too).

I also hope I can figure out the show as posts and show as thread until then (tho I believe best would be to use subscriptions for that [a member told me he was searching for his posts + show as threads to track threads he posted in: SUBSCRIPTIONS]).

[kmike]

Originally Posted by orban

Search posts and display as threads:

Let's say somebody searches for "book" and returns 150.000 posts. Those 150.000 posts are in 40.000 threads. If you find any way to fetch all 150.000 threadids, sort them and make a unique list of them, then let me know, but I really have no idea how to do that. I also think that this is a major problem of the vB search...(there are queries with several tens of thousands threadids in them).

I assume you're storing threadid as a group attribute, to support searching within threads. So you'll get it back in the search results for every post found.
Just collect all threadid's in an array, throw out the duplicates using array_unique, and voila, you have your results as threads.

Originally Posted by orban

Search threads and display as posts

There's no such option, looks like you mean "search titles only". But posts have titles too, you know?

[orban]

Originally Posted by kmike

I assume you're storing threadid as a group attribute, to support searching within threads. So you'll get it back in the search results for every post found.
Just collect all threadid's in an array, throw out the duplicates using array_unique, and voila, you have your results as threads.

Yeah, just when there's 120.000 posts found......you'd have to increase the limit in sphinx.conf to 200.000 or so, and loop through ALL of them, then throw out the uniques, and then sort by lastpost....!?

Originally Posted by kmike

There's no such option, looks like you mean "search titles only". But posts have titles too, you know?

Yes there is...you can select "Search Titles Only" and then "Show as Posts"....

I think it returns the first post of all threads found....

[ubuntu-geek]

Orban just curious, how often do you re-index the big index?

[ubuntu-geek]

Right on, I'll give it a go. I do have a weird one though. If I do a search for just a username and leave everything else default it will pull only older threads nothing new. hrm...

[orban]

If you don't enter any search terms the default vB search should be used....

[kmike]

Originally Posted by orban

Yeah, just when there's 120.000 posts found......you'd have to increase the limit in sphinx.conf to 200.000 or so, and loop through ALL of them, then throw out the uniques, and then sort by lastpost....!?

What is your search results limit, is it really that high (120000)? I highly doubt it because your current search implementation would choke on that number, too, since the part of the script responsible for the search results display already goes through all the returned results.

So I guess you have more reasonable limit to the number of returned search results (around 1000?). At which point going through all of them suddenly doesn't look so bad.

Originally Posted by orban

Yes there is...you can select "Search Titles Only" and then "Show as Posts"....

"Search titles only" combined with "show as posts" should search within the titles of the posts. They happen to be the same as the titles of the threads in the case of a first post in a thread (well, at least in most cases).
Now, the original vB search implementation (non-fulltext one) is following this logic. But vB fulltext implementation is throwing this concept away and searches within the titles of the threads, displaying only first posts in the threads found. I'll let you judge if this is correct or not.

Personally, I, too, think it's too confusing, but it's the legacy of the decision to allow each post to have its own title. Most of the members don't bother to type anything in a post title field when replying, and even if they do, it's completely inconspicuous in the default vB layout (and in the most vB layouts I've seen, for that matter).
But it's there, and it's there for good, so we should bear with it.

*edit*: cool, 100 posts! I'll let it sit there for some time ;-)

[orban]

Originally Posted by kmike

What is your search results limit, is it really that high (120000)? I highly doubt it because your current search implementation would choke on that number, too, since the part of the script responsible for the search results display already goes through all the returned results.

Well...

Let's assume you have

thread1 - 100 times "word"
thread2 - 50 times "word"
thread3 - 10 times "word"
thread4-50 5 times "word"

A search for "word" will return us 2500 posts. BUT there are only 50 different threads.

If your limit is 1000 (like mine) this will only return like 30 threads. So you're missing out 20......I'm actually seeing this on very common words (when searching post and "show as threads").

----------

1. Search Titles Only - Show as Threads = full text index on thread titles
2. Search Titles Only - Show as Posts = full text index on post titles
3. Search Entire Posts - Show as Threads = full text index on posts but grab threadids and display them, basically grouped by thread
4. Search Entire Posts - Show as Posts = full text index on posts

1., 3., 4. is working already. 2. is not (yet). I'll need to fix this then. (At the moment it's searching thread titles only and displaying the first post).

Also it's not weighting post titles/bodies yet (I think).

[TECK]

Guys, when you compiled Sphinx, did you specified the mysql directory or you simple used --with-mysql?
Thanks.