I'm getting ready to deploy a site that is using a query string search algorithm for our catalog search functionality. The query string concept is coming from Jeffrey's lesson on a eloquent model. My issue that I'm having is the page load time before I perform the caching.
On a very query intensive page I could have in excess of 150 queries, with a page load time of ~7secs, generating everything I need. I cache these queries during the initial page is load. When that exact page is reloaded, leveraging cache, the page loads in about 800ms. Now here is the million dollar question...
How can I create a script/process/console command that can iterate over every possible link (query string) of the e-commerce site, building the page, thus building the cache. My goal is to have a site that is extremely fast when we launch it for the consumer.
@jandk4014 Spidering your site for every possible query string permutation to “pre-cache” things sounds like an intensive and wholly impractical solution. Instead, I’d be more worried about this statement:
On a very query intensive page I could have in excess of 150 queries
Work out why a single page is performing 150 database queries. No page should be performing that many queries. So see if there are some relationships that can be eager-loaded. Or if you have a dynamic menu coming from the database, cache that (as it’s not going to change from page to page unless an administrator modifies the menu in the admin panel).
@martinbean - While I appreciate your insight in the matter, I wonder how sites can perform dynamic prefetched queries over millions of possible combinations for products. I know that I have a deeper understanding about my situation than others.
I'd like to draw your attention to the filters on the left hand side of each site. Each filter has multiple filter items. Of each of those filter items, there are associated "products available". I have the same concept on our site, and it works as desired. Even with eager loading, on the links above, there would be overhead for each filter item and products available. When caching is leveraged those sites would perform dramatically faster.
Is my theory right on caching for sites of this magnitude? How do they get the page to load so quickly?
Big e-commerce sites like this do not use mysql (or any other relational database) for that. Things like NoSQL and/or elastic search are way better at this job. It is possible to store all your product data in a structured way in your relational database (mysql) but than 'sync' it to a nosql/elastic search every night or based on some events.
