Sphinx Search

[orban]

full text so vB doesn't populate its search tables (at least me, I just noticed that this is actually not mentioned in the guide)

[ubuntu-geek]

great thanks..

Orban have you updated to sphinx rc2 yet?

[amcd]

orban, i think you should clearly mention this in the tutorial. if vb is not switched to fulltext, there will be hardly any benefit from sphinx. also, the fulltext indices should be dropped, otherwise mysql will keep updating them and waste time.

[mute]

So, is there any hope to having sphinx handle searching for all of a users posts? I forget what we determined earlier in the thread. Despite sphinx being fast, I'm still seeing slowdowns related to doing the "find all posts by user" searches

[orban]

I don't think Shodan (from sphinx) has implement keyword-less queries yet but he plans to afaik.

[kmike]

You can emulate the search by user in sphinx by adding a fake unique keyword per each member in the mix (e.g. "_userid_12345"). Searching by this keyword will return all posts by the member with userid 12345.

[ubuntu-geek]

I've noticed a few people talking about the sorting of searches being off. I am having the same issue, has anyone found a fix for it yet?

[mute]

Originally Posted by ubuntu-geek

I've noticed a few people talking about the sorting of searches being off. I am having the same issue, has anyone found a fix for it yet?

Not I. I can't say i've looked into it, but a magical fix would be awesome.

[DigitalCrowd]

I think I figured out the problem, but not sure I know how to fix it at this very moment.

The sphinx.conf file that is being distributed in this thread builds the date_column as "dateline" for post index, and "lastpost" for thread index. I noticed the output from Sphinx is in proper order by dateline, but since the two indexes are not being given the same date source, then your threads will be out of order on search results when, I assume, that it groups the posts into the threads and the output displays the last post date of the thread and not the "post date" of the post that your search matched.

While you could fetch the lastpost date of a thread that is associated with the post and this way you use one date column unique across the indexes, my assumption is that unless you rebuild the full index (not deltas) constantly, that your searches will still be messed up.

We already know that when we say we want 1000 results and 7000 documents match, that we may only have 853 results, once all posts are grouped under specific threads.

I will have to think about this one.

Wait, hold on, things are coming to quickly...

Example:

You do a search on "Trees", and it finds 10 posts with the word "Trees" in it. For the sake of this discussion, "Trees" is only in one post in each thread. The results come back and order in DESC order, all the post's dateline.

Well, this is great, except, additional posts to those threads may have happened, and as such the search results are all out of order, because the search returns the dateline order, not the lastpost order of the associated thread.

This ALSO explains why when each of us first installed this and indexed our boards, that everything worked perfectly, because it was a brand new index. But, once you start building onto that index that is when things go astray.

I believe this is what is causing it all, but I might be missing something.

[amcd]

DigitalCrowd, I think you have hit the nail right on the head. I rebuild my full index everyday, and my results are just slightly out of order. And the thread which are our of order are the ones which have been updated today, after the last rebuild.

So, how do we fix this problem? Once the search results have been received from sphinx, we then re-sort them by lastpost if the user has requested 'show results as threads'?