Jump to content


Photo
- - - - -

search and archives - say it isnt so


  • Please log in to reply
28 replies to this topic

#1 mikesound

mikesound

    Advanced Member

  • +Clients
  • 426 posts

Posted 25 March 2012 - 07:50 AM

have tested but am not 100% sure yet
but does seems at first look that the archives are not included in the default forum search (sphinx)

you have to go to advanced search and then tick archive option

if the above is right then can i ask why has it been done this way

if there are separate searches for forum and archive then for my own uses this makes the whole archive feature unusable

please someone tell me that I am wong

#2 mikesound

mikesound

    Advanced Member

  • +Clients
  • 426 posts

Posted 25 March 2012 - 10:48 AM

mmmm seems above is right

sphinx can search multiple indexes and so in theory could have a complete all in one forum/archive search, yes?
(and so in theory could have a complete site wide search)
but would guess there's no plans for this, or at least in the near future

so it looks like one of the major improvements is going to be unusable for anyone who needs to offer a decent search service that can search every past forum post with no fuss


all a bit disappointing really
do have to wonder yet again about invision logic when it comes to features like this

I mean why archive posts if the only way members can then find them is by then using the advanced search ?
how many casual members are going to be aware of that requirement when using the top search or indeed if even aware are then going to bother searching and viewing two sets of results ?

still have slight hope that have got this wrong and am missing something

#3 Rimi

Rimi

    Strip Me

  • +Clients
  • 6,121 posts

Posted 25 March 2012 - 10:49 AM

I would assume it's because the archived posts are moved to a different table (kind of the point) and that searching in multiple tables at once is a no no.

#4 mikesound

mikesound

    Advanced Member

  • +Clients
  • 426 posts

Posted 25 March 2012 - 10:53 AM

I would assume it's because the archived posts are moved to a different table (kind of the point) and that searching in multiple tables at once is a no no.


sphinx apparently can search multiple indexes/tables

http://sphinxsearch....ew.html?id=6575



would think that chances are the ones who may have a lot of posts to archive may use sphinx

Edited by mikesound, 25 March 2012 - 11:18 AM.


#5 Charles

Charles

    Needs Life

  • IPS Management
  • 9,018 posts

Posted 25 March 2012 - 04:38 PM

The point of the archive system is to take stress of your database by moving posts to a table that will be rarely queried. If every search was auto-queried to the archive table that sort of defeats the purpose :)
  • Michael and Steven@wt like this

Charles Warner
Invision Power Services, Inc. - President
charles.warner@invisionpower.com
IPS Twitter news feed: @invisionps

 

Please do not PM me but feel free to contact me by email.

 

IPS 4 preview site ... Feel free to register your account now. We are approving several new accounts each day so, once you register, keep an eye out for activation email.


#6 mikesound

mikesound

    Advanced Member

  • +Clients
  • 426 posts

Posted 25 March 2012 - 05:36 PM

The point of the archive system is to take stress of your database by moving posts to a table that will be rarely queried. If every search was auto-queried to the archive table that sort of defeats the purpose :smile:


removing large amounts of posts from the default search should not be the answer

do you seriously think people are going to do one search, browse the results and then find the advanced link and do another search and browse results all off their own back

would that work on this board ? are you going to archive posts on here then?
try it and see how it goes am sure having to double search in such an akward manner will go down a treat

the disapointing thing is that the basic sphinx setup to give the option to search two or one different tables is fairly easy to implement
in fact it could be said it is easier as there would be less duplication of code and you could still offer search 1 or 2 option

would it have hurt to think a bit about this before giving such an dismissive answer to my feedback

invision it seems has either missed or dropped the whole sphinx search more than one index aspect and by doing so have imo made a useful feature useless to anyone who runs a site that gets searched a lot for info/research etc

#7 bfarber

bfarber

    RBT-KS

  • IPS Management
  • 28,587 posts

Posted 26 March 2012 - 11:38 AM

Please keep in mind that sphinx is a secondary search method, so sometimes it can do more than we take advantage of.

This is something we can keep in mind for future versions in any event. :)

Brandon Farber
Development Manager / Senior Support

If it sounds like fun, it's not allowed on the bus!

php5_zce_logo_new.gif     

Invision Power Services, Inc.


#8 mikesound

mikesound

    Advanced Member

  • +Clients
  • 426 posts

Posted 26 March 2012 - 12:56 PM

Please keep in mind that sphinx is a secondary search method, so sometimes it can do more than we take advantage of.

This is something we can keep in mind for future versions in any event. :smile:


thanks for reply
fair enough, I think at times my requirements from the software must be a bit different than the out of the box solution
hence the frustration creeping in when that difference comes up
guess way to look at archive for myself is for content that doesn't require being made available for searching

#9 Charles

Charles

    Needs Life

  • IPS Management
  • 9,018 posts

Posted 26 March 2012 - 04:50 PM

If you want full functionality of topics then you should not archive them.

The archive system is designed as a sort of "deep freeze" for topics so normal day-to-day operations like search, reply, moderation actions, etc. are not available.

Charles Warner
Invision Power Services, Inc. - President
charles.warner@invisionpower.com
IPS Twitter news feed: @invisionps

 

Please do not PM me but feel free to contact me by email.

 

IPS 4 preview site ... Feel free to register your account now. We are approving several new accounts each day so, once you register, keep an eye out for activation email.


#10 h2ojunkie

h2ojunkie

    IPB Member

  • +Clients
  • 54 posts

Posted 27 August 2012 - 11:42 PM

Being a forum admin of a 10,000,000 post site. The archive in theory is a great idea.

However, I have to agree with mikesound on this one.

The entire reason big sites run things like sphinx is to keep searching of a large database fast.

There is really no reason I can see to not take advantage of the power of sphinx and the archive feature together.

Sphinx can eaisly handle a single result across both the live and archive tables. I can't speak to using mysql full text search (since I stopped using that 9 million posts ago). But for a sphinx search setup, it should absolutly search the archive and live posts at the same time. There's just no downside to doing it that way.
  • TSP and AlexJ like this

#11 TSP

TSP

    Needs Serious Help

  • +Clients
  • 1,037 posts

Posted 28 August 2012 - 04:51 AM

If you want full functionality of topics then you should not archive them.

The archive system is designed as a sort of "deep freeze" for topics so normal day-to-day operations like search, reply, moderation actions, etc. are not available.

But Sphinx is not using a connection to the DB when searching, is it? I thought it made it's own indexes and copies of the post?

We're not using sphinx, but a custom made search that are using the Solr-engine. We export posts from the database to Solr for indexing and when searching it's only looked up to the information Solr have gathered. No DB-queries are used in the actual search. I would expect it to be similar for your built-in sphinx option?

As mentioned in the post above I can't see any harm in including the archive as part of the search results when using Sphinx. If you're having so many posts that you need to archive them, I wouldn't expect them to still be using the mysql full text search in the first place.


There is really no reason I can see to not take advantage of the power of sphinx and the archive feature together.

Indeed.

I am administrator TSP on the norwegian IPB 3.4.5-board Diskusjon.no.


#12 Wolfie

Wolfie

    Don't get mad, get N*Raged!

  • +Clients
  • 12,500 posts

Posted 28 August 2012 - 04:55 AM

What about a system setting along with a front end option? System setting would toggle the front end feature to be available or not (and if on, minimum access required to use it.. guest, registered member, moderator, global moderator, admin) and that front end feature (when enabled) would let a member know that the search results don't include the archived content, with a link to do the same search but in the archived posts.

Would let the powers-that-be have the ability to do searches themselves while not letting regular members use it (save resources) or might just not let guests use it, or whatever else.

٩(͡๏̯͡๏)۶ Click here to browse or purchase IPS software. ٩(͡๏̯͡๏)۶
n-raged.com
- Dacity.Com

♪ Me and you ♪
♪ a two-man crew ♪
♪ side by side we're unified ♪
♪ and we will never be divided ♪

Spoiler

CLICK HERE AND VOTE IP.BOARD AS BEST FORUM SOFTWARE FOR 2013!!!


#13 bfarber

bfarber

    RBT-KS

  • IPS Management
  • 28,587 posts

Posted 28 August 2012 - 08:42 AM

Part of the problem is that there is almost no way to do this with built in MySQL search. While we could do it with Sphinx (because Sphinx can support multiple index searches), there is then a feature-set discrepancy between using sphinx and not using sphinx, which we try to minimize.

We will be taking a very hard look at the search system as a whole as we move to the Community Suite, and will consider specific improvements in closer detail then.

Brandon Farber
Development Manager / Senior Support

If it sounds like fun, it's not allowed on the bus!

php5_zce_logo_new.gif     

Invision Power Services, Inc.


#14 mikesound

mikesound

    Advanced Member

  • +Clients
  • 426 posts

Posted 28 August 2012 - 10:29 AM

Part of the problem is that there is almost no way to do this with built in MySQL search. While we could do it with Sphinx (because Sphinx can support multiple index searches), there is then a feature-set discrepancy between using sphinx and not using sphinx, which we try to minimize.

We will be taking a very hard look at the search system as a whole as we move to the Community Suite, and will consider specific improvements in closer detail then.

think said it before

my suggestion is to drop sphinx from the invision core

change it to a module that can be installed by those who wish to use it

back when sphinx was a stand alone module it was easy to modify/adapt
i had full site search with drupal intergrated before ipcontent

a seperate application would allow easier development and better features (all in one site search etc)
with sphinx
  • AlexJ likes this

#15 Rimi

Rimi

    Strip Me

  • +Clients
  • 6,121 posts

Posted 28 August 2012 - 10:43 AM

you have to go to advanced search and then tick archive option

I don't see it anymore. >_>;

#16 Marcher Technologies

Marcher Technologies

    $life=FALSE;$code=TRUE;$time--;

  • +Clients
  • 11,717 posts

Posted 28 August 2012 - 10:52 AM

think said it before

my suggestion is to drop sphinx from the invision core

change it to a module that can be installed by those who wish to use it

back when sphinx was a stand alone module it was easy to modify/adapt
i had full site search with drupal intergrated before ipcontent

a separate application would allow easier development and better features (all in one site search etc)
with sphinx

A separate application would mean less compatibility across not only the Suite, but any third-party application's supporting sphinx, sorry, cannot agree.

#17 Dmacleo

Dmacleo

    Needs Life

  • +Clients
  • 9,749 posts

Posted 28 August 2012 - 10:57 AM

one reason I went with IP was due to advertised compatibility with sphinx, needed the 2 letter and wildcard searches.
  • TSP likes this

Dave M
I'm a man.
I can change.
If I have to.
I guess.....


#18 mikesound

mikesound

    Advanced Member

  • +Clients
  • 426 posts

Posted 28 August 2012 - 11:01 AM

A separate application would mean less compatibility across not only the Suite, but any third-party application's supporting sphinx, sorry, cannot agree.


sphinx has been slowly phased out - its just been dropped from the gallery
new content needs to be mysql if you wish instant listings
downloads next ?

soon could end up with it just being used for the forum search

if was an application third parties could just plug in to it

i would say with arguably less coding/dev than takes now

#19 Dmacleo

Dmacleo

    Needs Life

  • +Clients
  • 9,749 posts

Posted 28 August 2012 - 11:08 AM

just noticed it does not work on gallery now.
dammit.
maybe time to dump gallery.

Dave M
I'm a man.
I can change.
If I have to.
I guess.....


#20 Marcher Technologies

Marcher Technologies

    $life=FALSE;$code=TRUE;$time--;

  • +Clients
  • 11,717 posts

Posted 28 August 2012 - 11:10 AM

sphinx has been slowly phased out - its just been dropped from the gallery
new content needs to be mysql if you wish instant listings
downloads next ?

soon could end up with it just being used for the forum search

if was an application third parties could just plug in to it

i would say with arguably less coding/dev than takes now

It was dropped over a year ago(Gallery 4), and that is nothing new, and it *has* to be abstracted by application in the core, a 'global' site search is infeasible, what good is searching on downloads custom fields through forums indexes? None, it is an utterly wasteful act and you know it.




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users