Sphinx Search

[Rayn21]

I'm finding that my sphinx search is only returning results to vbulletin from the last month or so, regardless of the advanced search settings. When I run commands directly to search on the command line, I get many, many more results ... but when interacting with vbulletin something goes wrong.

Any immediate ideas?

[telescopi]

Works here - I just followed the instructions on obans post on page 26.

Could your vbulletin max results be limiting what sphinx is returning?

What happens if you sort ascending?

[Rayn21]

I set the vbulletin max results to 6000 just to see what happens and I wound up with more posts, but still I am missing some. I am wondering what I am missing. It had been working for me as far as I know, it's just that the popular search terms seem to have a hidden time constraint on them ... maybe because they return so many results.

If I search for a popular search term, I'll get back maybe 300 results all from a recent time frame (about a month). If I search for a less popular search term, I'll get results going all the way back to 2000 like I should. I will have to go through and re-examine all the scripts again, but something is funny going on.

What could limit my results between the search command line and vbulletin itself ... because there are many more results returns on the command line.

If I search for a popular term like 'Kurayami' a popular poster who is mentioned often ... with 6000 max results in vB and 1200 max results in sphinx.conf ... I'll max out at 1000 results all very recent. Weird but understandable. If I search 'Rayn' ... a few results but many less than 1000 ... all within a relatively recent time frame. There should be more, the command line returns thousands of posts.

I cannot locate the source of this discrepancy.

http://www.tribalwar.com/forums/search.php

[mlx]

I think the problem with the instructions posted here is that the search result isn't grouped by threadid if you are searching for threads only.

i.e. Sphinx' API only returns 500 posts but not threads containing the search query (or whatever you are using as the vb search limit).

I think we have fixed this at our forum, I can post our improved sphinx.php once I'm back home.

[Rayn21]

What you are describing is consistent with what I am observing right now. I'd love to see what solution you found to the issue if you remember to post it later.

[mlx]

Originally Posted by orban

I'm leaving this place if somebody wants to take over this thread and keep the guide up to date feel free to do so it's on page 26 I think. Bye.

So I guess it's OK to post our improved sphinx.php here. Thanks to orban and everyone else contributing to this thread for their work!

I have added some code to group the search results by threadid and to actually sort the results just like vBulletin's default search engine does.

I think the code should work fine with the config posted by orban on page 26. Let me know if you have any problems though.

[Rayn21]

The code works wonderfully for most of my users, but for some users I get an empty searchd result back if - and only if - I am returning results as threads. (When searching for posts). When I use the search command line with the threadid as the groupby and the same index, I get results. I looked through the code, but I am not sure what would cause this particular issue. Have any ideas? I love the improvements.

[Jon]

Sphinx works fine on our forums when displaying results as threads. Displaying results as posts however returns an incomplete set of results, but only with terms that occur often (which is not hard with > 8M posts ). On top I get results with a match on the title only (why..?), and those are followed by older results with a match on posts. Recent posts seem to be missing in the result set.

Any hints on how to get the search to behave properly?

[amcd]

maybe your post delta is not being updated properly.

[Jon]

No, the indexer works fine. There could be up to hundreds of posts not included. Say I'm searching for a term, with a match on posts, displayed as posts, I first get 3 pages of matches on title, ranging from 2007 to 2005. After that I get matches on post, ranging from 2005 to 2003. Recent posts are nowhere to be found.