Sign in to follow this  
Followers 0

Suggestion - Cache Query but not Block

14 posts in this topic

Posted

I think there should be an option to cache the query used in feed blocks, not just the whole block.

For example, lets say you have some latest content feed blocks like latest posts, downloads, articles, comments, etc. You want these cached to reduce server load (my page loads start to lag having say 4 of these types of blocks on a page not cached).

The problem is, the block needs to be processed on a per member basis to set the correct timezone for the user.

So I think it would be great to have an option to cache the query used for the block, without having to cache the html output. That way you could set the cache query to say 5 mins, to reduce server load, and then just process the html template on demand using the cached query.

Share this post


Link to post
Share on other sites

Posted

[quote name='alexp999' timestamp='1334824137' post='2256956']
I think there should be an option to cache the query used in feed blocks, not just the whole block.

For example, lets say you have some latest content feed blocks like latest posts, downloads, articles, comments, etc. You want these cached to reduce server load (my page loads start to lag having say 4 of these types of blocks on a page not cached).

The problem is, the block needs to be processed on a per member basis to set the correct timezone for the user.

So I think it would be great to have an option to cache the query used for the block, without having to cache the html output. That way you could set the cache query to say 5 mins, to reduce server load, and then just process the html template on demand using the cached query.

I like the idea... just wanted to note... there is still a query regardless of caching the result set query seperately... the HTML template is stored/pulled in/from the database all the same, and no cache file is generated for it.... though again, this still would be helpful.... and me, I would be similarly pleased with having those same html templates compiled and cached... I have 0 need for a query to be run for custom html blocks I have not gone and put in myself.... and caching the compiled template code to a file would similarly reduce extraneous queries, without actually having to go and cache the resulting html.... which honestly i've never been fond of... caching the html out direct to the database kills language abstraction amongst other bits.

Share this post


Link to post
Share on other sites

Posted

[quote name='Marcher Technologies' timestamp='1343006196' post='2289967']
I like the idea... just wanted to note... there is still a query regardless of caching the result set query seperately... the HTML template is stored/pulled in/from the database all the same, and no cache file is generated for it.... though again, this still would be helpful.... and me, I would be similarly pleased with having those same html templates compiled and cached... I have 0 need for a query to be run for custom html blocks I have not gone and put in myself.... and caching the compiled template code to a file would similarly reduce extraneous queries, without actually having to go and cache the resulting html.... which honestly i've never been fond of... caching the html out direct to the database kills language abstraction amongst other bits.


In our profiling, we've often found that database skins (requiring "an extra query") outperform the regular cached template files in many cases, because disk i/o is slow. Caching these templates to some sort of flat file isn't necessarily faster than querying the templates at run time.

Share this post


Link to post
Share on other sites

Posted

Ive since written my own plugin using the IP.Content blocks, basically: [*]Cached Feed Block, grabs all the latest content, etc (resource intensive query), and the output is a serialized $records array. [*]My custom plugin then grabs the feed block output from above (non intensive query) and unserializes it. [*]The template for my plugin is the generic feed template (modified slightly for my site), and shows the cached content, but generated on every page load to have the correct time for the user (and time difference, i.e. 5 mins ago). It works as the original (cached) query only outputs timestamps, so my block later on which isnt cached can always display the correct time and zone for the user. Thats why I started this thread, I think IP.Content should have a caching system for the query. It basically means I dont have to do resource intensive queries every page load, its dramatically sped up my site.








Share this post


Link to post
Share on other sites

Posted

[quote name='bfarber' timestamp='1343057131' post='2290108']
In our profiling, we've often found that database skins (requiring "an extra query") outperform the regular cached template files in many cases, because disk i/o is slow. Caching these templates to some sort of flat file isn't necessarily faster than querying the templates at run time.

But a hard file has not the low limits on storage SQL has.... there is an upper on that 'full' cache. I have hit it before and had to adjust the server.... and recache all blocks/templates to fix the data truncation when it hits field limits/max insert.

Share this post


Link to post
Share on other sites

Posted

Everything has limits (including hard drives). That doesn't change my statement, however. :)

Share this post


Link to post
Share on other sites

Posted

Just a side FYI to the OP, the data result set stored would have to be cleaned, and I mean heavily, to be reasonably stored. Even serialized, the results array from a single feed block pulling say, 10 rows is HUGE.

Share this post


Link to post
Share on other sites

Posted

[quote name='Marcher Technologies' timestamp='1344459299' post='2295193']
Just a side FYI to the OP, the data result set stored would have to be cleaned, and I mean heavily, to be reasonably stored. Even serialized, the results array from a single feed block pulling say, 10 rows is HUGE.


I've got 4 blocks with 10 rows each stored serialized and it is still considerably quicker than no caching at all.

Share this post


Link to post
Share on other sites

Posted

[quote name='alexp999' timestamp='1344460385' post='2295201']
I've got 4 blocks with 10 rows each stored serialized and it is still considerably quicker than no caching at all.

I never said there would be no great benefit, it just needs implementing, and a further abstraction layer of feed blocks to separate the method executeBlock into two methods, one that gathers the data and returns the rows, and the other returning the template fed either cached or new rows.
It has IMMENSE benefits, I razed a statistics applications weight down to board index no hooks vanilla weight doing this.

Share this post


Link to post
Share on other sites

Posted

I misuderstood what you meant by heavy cleaning then :/

Share this post


Link to post
Share on other sites

Posted

[quote name='alexp999' timestamp='1344462033' post='2295211']
I misuderstood what you meant by heavy cleaning then :/

I assume you are not storing the member data loaded for each result in the feed then, basically, only store what is actually used or such a cache goes into the 1mb+ for 1 feed block scale, taking into consideration the ability to add custom fields, as many as you wish on x database, and a user-defined limit, and the array to be serialized can get unwieldy.

Share this post


Link to post
Share on other sites

Posted

I havent stripped anything from the array, its serialized the entire $records array.

I might look into stripping what I dont need. I imagine its only going to affect how quickly the server reaches max capacity as I cant see how the page can load any faster than it already does.

Share this post


Link to post
Share on other sites

Posted


In our profiling, we've often found that database skins (requiring "an extra query") outperform the regular cached template files in many cases, because disk i/o is slow. Caching these templates to some sort of flat file isn't necessarily faster than querying the templates at run time.


I disagree with this specifically how HDD speeds have been improved lately. We use enterprise level SSD's on our live servers and no database query will read data faster than filesystem. Especially if you have a pretty busy board and big database.


Reducing query number is very critical for big boards. So even a query that inserts html output is a BAD query.

Alfa1 likes this

Share this post


Link to post
Share on other sites

Posted

It is simply a statement from experience.  We run a hosting company and host many many large sites, after all.

 

For the record, I was also not stating we would or would not be doing anything specific.  I was only making a comment relevant to the current discussion. :)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0

  • Who's Browsing   0 members

    No registered users viewing this page.