search and archives - say it isnt so

29 posts in this topic

Posted

have tested but am not 100% sure yet
but does seems at first look that the archives are not included in the default forum search (sphinx)

you have to go to advanced search and then tick archive option

if the above is right then can i ask why has it been done this way

if there are separate searches for forum and archive then for my own uses this makes the whole archive feature unusable

please someone tell me that I am wong

Share this post


Link to post
Share on other sites

Posted

mmmm seems above is right

sphinx can search multiple indexes and so in theory could have a complete all in one forum/archive search, yes?
(and so in theory could have a complete site wide search)
but would guess there's no plans for this, or at least in the near future

so it looks like one of the major improvements is going to be unusable for anyone who needs to offer a decent search service that can search every past forum post with no fuss


all a bit disappointing really
do have to wonder yet again about invision logic when it comes to features like this

I mean why archive posts if the only way members can then find them is by then using the advanced search ?
how many casual members are going to be aware of that requirement when using the top search or indeed if even aware are then going to bother searching and viewing two sets of results ?

still have slight hope that have got this wrong and am missing something

Share this post


Link to post
Share on other sites

Posted

I would assume it's because the archived posts are moved to a different table (kind of the point) and that searching in multiple tables at once is a no no.

Share this post


Link to post
Share on other sites

Posted (edited)

[quote name='Rimi' timestamp='1332690544' post='2245462']
I would assume it's because the archived posts are moved to a different table (kind of the point) and that searching in multiple tables at once is a no no.


sphinx apparently can search multiple indexes/tables

http://sphinxsearch.com/forum/view.html?id=6575



would think that chances are the ones who may have a lot of posts to archive may use sphinx

Edited by mikesound

Share this post


Link to post
Share on other sites

Posted

The point of the archive system is to take stress of your database by moving posts to a table that will be rarely queried. If every search was auto-queried to the archive table that sort of defeats the purpose :)

Steven@wt and Michael like this

Share this post


Link to post
Share on other sites

Posted

[quote name='Charles' timestamp='1332711482' post='2245614']
The point of the archive system is to take stress of your database by moving posts to a table that will be rarely queried. If every search was auto-queried to the archive table that sort of defeats the purpose :smile:


removing large amounts of posts from the default search should not be the answer

do you seriously think people are going to do one search, browse the results and then find the advanced link and do another search and browse results all off their own back

would that work on this board ? are you going to archive posts on here then?
try it and see how it goes am sure having to double search in such an akward manner will go down a treat

the disapointing thing is that the basic sphinx setup to give the option to search two or one different tables is fairly easy to implement
in fact it could be said it is easier as there would be less duplication of code and you could still offer search 1 or 2 option

would it have hurt to think a bit about this before giving such an dismissive answer to my feedback

invision it seems has either missed or dropped the whole sphinx search more than one index aspect and by doing so have imo made a useful feature useless to anyone who runs a site that gets searched a lot for info/research etc

Share this post


Link to post
Share on other sites

Posted

Please keep in mind that sphinx is a secondary search method, so sometimes it can do more than we take advantage of.

This is something we can keep in mind for future versions in any event. :)

Share this post


Link to post
Share on other sites

Posted

[quote name='bfarber' timestamp='1332779925' post='2246033']
Please keep in mind that sphinx is a secondary search method, so sometimes it can do more than we take advantage of.

This is something we can keep in mind for future versions in any event. :smile:


thanks for reply
fair enough, I think at times my requirements from the software must be a bit different than the out of the box solution
hence the frustration creeping in when that difference comes up
guess way to look at archive for myself is for content that doesn't require being made available for searching

Share this post


Link to post
Share on other sites

Posted

If you want full functionality of topics then you should not archive them.

The archive system is designed as a sort of "deep freeze" for topics so normal day-to-day operations like search, reply, moderation actions, etc. are not available.

Share this post


Link to post
Share on other sites

Posted

Being a forum admin of a 10,000,000 post site. The archive in theory is a great idea.

However, I have to agree with mikesound on this one.

The entire reason big sites run things like sphinx is to keep searching of a large database fast.

There is really no reason I can see to not take advantage of the power of sphinx and the archive feature together.

Sphinx can eaisly handle a single result across both the live and archive tables. I can't speak to using mysql full text search (since I stopped using that 9 million posts ago). But for a sphinx search setup, it should absolutly search the archive and live posts at the same time. There's just no downside to doing it that way.

AlexJ and TSP like this

Share this post


Link to post
Share on other sites

Posted

[quote name='Charles' timestamp='1332798626' post='2246234']
If you want full functionality of topics then you should not archive them.

The archive system is designed as a sort of "deep freeze" for topics so normal day-to-day operations like search, reply, moderation actions, etc. are not available.

But Sphinx is not using a connection to the DB when searching, is it? I thought it made it's own indexes and copies of the post?

We're not using sphinx, but a custom made search that are using the Solr-engine. We export posts from the database to Solr for indexing and when searching it's only looked up to the information Solr have gathered. No DB-queries are used in the actual search. I would expect it to be similar for your built-in sphinx option?

As mentioned in the post above I can't see any harm in including the archive as part of the search results when using Sphinx. If you're having so many posts that you need to archive them, I wouldn't expect them to still be using the mysql full text search in the first place.


[quote name='h2ojunkie' timestamp='1346128970' post='2301944']
There is really no reason I can see to not take advantage of the power of sphinx and the archive feature together.

Indeed.

Share this post


Link to post
Share on other sites

Posted

What about a system setting along with a front end option? System setting would toggle the front end feature to be available or not (and if on, minimum access required to use it.. guest, registered member, moderator, global moderator, admin) and that front end feature (when enabled) would let a member know that the search results don't include the archived content, with a link to do the same search but in the archived posts.

Would let the powers-that-be have the ability to do searches themselves while not letting regular members use it (save resources) or might just not let guests use it, or whatever else.

Share this post


Link to post
Share on other sites

Posted

Part of the problem is that there is almost no way to do this with built in MySQL search. While we could do it with Sphinx (because Sphinx can support multiple index searches), there is then a feature-set discrepancy between using sphinx and not using sphinx, which we try to minimize.

We will be taking a very hard look at the search system as a whole as we move to the Community Suite, and will consider specific improvements in closer detail then.

Share this post


Link to post
Share on other sites

Posted

[quote name='bfarber' timestamp='1346161341' post='2302069']
Part of the problem is that there is almost no way to do this with built in MySQL search. While we could do it with Sphinx (because Sphinx can support multiple index searches), there is then a feature-set discrepancy between using sphinx and not using sphinx, which we try to minimize.

We will be taking a very hard look at the search system as a whole as we move to the Community Suite, and will consider specific improvements in closer detail then.

think said it before

my suggestion is to drop sphinx from the invision core

change it to a module that can be installed by those who wish to use it

back when sphinx was a stand alone module it was easy to modify/adapt
i had full site search with drupal intergrated before ipcontent

a seperate application would allow easier development and better features (all in one site search etc)
with sphinx

AlexJ likes this

Share this post


Link to post
Share on other sites

Posted

[quote name='mikesound' timestamp='1332679810' post='2245373']
you have to go to advanced search and then tick archive option

Share this post


Link to post
Share on other sites

Posted

[quote name='mikesound' timestamp='1346167766' post='2302127']
think said it before

my suggestion is to drop sphinx from the invision core

change it to a module that can be installed by those who wish to use it

back when sphinx was a stand alone module it was easy to modify/adapt
i had full site search with drupal intergrated before ipcontent

a separate application would allow easier development and better features (all in one site search etc)
with sphinx

A separate application would mean less compatibility across not only the Suite, but any third-party application's supporting sphinx, sorry, cannot agree.

Share this post


Link to post
Share on other sites

Posted

one reason I went with IP was due to advertised compatibility with sphinx, needed the 2 letter and wildcard searches.

TSP likes this

Share this post


Link to post
Share on other sites

Posted

[quote name='Marcher Technologies' timestamp='1346169177' post='2302140']
A separate application would mean less compatibility across not only the Suite, but any third-party application's supporting sphinx, sorry, cannot agree.


sphinx has been slowly phased out - its just been dropped from the gallery
new content needs to be mysql if you wish instant listings
downloads next ?

soon could end up with it just being used for the forum search

if was an application third parties could just plug in to it

i would say with arguably less coding/dev than takes now

Share this post


Link to post
Share on other sites

Posted

just noticed it does not work on gallery now.
dammit.
maybe time to dump gallery.

Share this post


Link to post
Share on other sites

Posted

[quote name='mikesound' timestamp='1346169701' post='2302146']
sphinx has been slowly phased out - its just been dropped from the gallery
new content needs to be mysql if you wish instant listings
downloads next ?

soon could end up with it just being used for the forum search

if was an application third parties could just plug in to it

i would say with arguably less coding/dev than takes now

It was dropped over a year ago(Gallery 4), and that is nothing new, and it *has* to be abstracted by application in the core, a 'global' site search is infeasible, what good is searching on downloads custom fields through forums indexes? None, it is an utterly wasteful act and you know it.

Share this post


Link to post
Share on other sites

Posted

[quote name='Marcher Technologies' timestamp='1346170202' post='2302150']
It was dropped over a year ago(Gallery 4), and that is nothing new, and it *has* to be abstracted by application in the core, a 'global' site search is infeasible, what good is searching on downloads custom fields through forums indexes? None, it is an utterly wasteful act and you know it.



you want to tone down your words, if I thought it was a wasteful act i wouldn't suggest it


sphinx was integrated in gallery 4 - it has just been dropped from gallery version 5 this month- not a year ago
it didnt work out of the box unless you yourself fixed an error - which highlights imo the failing of the current integration

an application offering site search across all the current applications should be fairly easy to do
all you need to do with sphinx is give it a query and which index or indexes to search and it returns results

its what you do with the results that seems to be the hard part for some reason

I would have thought the benefits of what an 'across the whole site search' would bring are obvious to any forum owner and fits in with the whole community direction

Share this post


Link to post
Share on other sites

Posted


It still has to format the results by application.

and..... no, don't state things without facts...
/gallery40002/admin/applications_addon/ips/gallery/extensions/search/engines/sphinx.php

Straight from the client area, this is not new for this application, it has been proxying to sql for a while now.

its what you do with the results that seems to be the hard part for some reason

sphinx was integrated in gallery 4 - it has just been dropped from gallery version 5 this month- not a year ago


<?php

/**

* <pre>

* Invision Power Services

* IP.Board v4.0.2

* Sphinx Gallery Search

* Last Updated: $Date: 2011-03-11 16:16:55 -0500 (Fri, 11 Mar 2011) $

* </pre>

*

* @author $Author: ips_terabyte $

* @copyright © 2001 - 2009 Invision Power Services, Inc.

* @license http://www.invisionpower.com/community/board/license.html

* @package IP.Board

* @subpackage Forums

* @link http://www.invisionpower.com

* @version $Rev: 8047 $

*/

if ( ! defined( 'IN_IPB' ) )

{

print "<h1>Incorrect access</h1>You cannot access this file directly. If you have recently upgraded, make sure you upgraded all the relevant files.";

exit();

}

include( IPS_ROOT_PATH . '/applications_addon/ips/gallery/extensions/search/engines/sql.php' );/*noLibHook*/

AlexJ likes this

Share this post


Link to post
Share on other sites

Posted

[quote name='Marcher Technologies' timestamp='1346172319' post='2302175']
It still has to format the results by application.

and..... no, don't state things without facts...
/gallery40002/admin/applications_addon/ips/gallery/extensions/search/engines/sphinx.php
ent area, this is not new for this application, it has been proxying to sql for a while now.





Share this post


Link to post
Share on other sites

Posted

[quote name='bfarber' timestamp='1346161341' post='2302069']Part of the problem is that there is almost no way to do this with built in MySQL search. While we could do it with Sphinx (because Sphinx can support multiple index searches), there is then a feature-set discrepancy between using sphinx and not using sphinx, which we try to minimize.


[quote name='mikesound' timestamp='1346167766' post='2302127']my suggestion is to drop sphinx from the invision core

change it to a module that can be installed by those who wish to use it

Not sure how feasible it would be, although it could open the door for 3rd party developers to come up with their own modules for site searches. Imagine someone making a module that is optimized for low traffic sites but isn't efficient on high volume sites (and vice-versa). Admin adds modules (and thus options) and then chooses which module to actively use (and with multiple installed, could simply change which to use at the drop of a hat, without having to download/install one since it'd already be there). Would make it easier to make 'plugin' modules that are optimized for sphinx or other server setups, etc.


[quote name='mikesound' timestamp='1346171951' post='2302172']you want to tone down your words, if I thought it was a wasteful act i wouldn't suggest it
<snip>
its what you do with the results that seems to be the hard part for some reason


[quote name='Charles' timestamp='1346176030' post='2302217']We're going to rethink searching with Suite 4.0. We agree it needs work.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Who's Browsing   0 members

    No registered users viewing this page.