tylernathanreed's avatar

Never Execute a Duplicate Query Again

I bet you have Barryvdh's Debug Bar or some equivalent enabled, and you're seeing tons of duplicate queries, and you simply don't have the time to track them all down.

Boy do I have the solution for you!

Have you considered caching every single query per request? Check this out:

Every SELECT query in Laravel, whether it's using Eloquent, or the DB Facade, all runs through a single function. We're going to override that function:

~/app/Support/Database/Builder.php

<?php

namespace App\Support\Database;

use Cache;
use Illuminate\Database\Query\Builder as QueryBuilder;

class Builder extends QueryBuilder
{
    /**
     * Run the query as a "select" statement against the connection.
     *
     * @return array
     */
    protected function runSelect()
    {
        return Cache::store('request')->remember($this->getCacheKey(), 1, function() {
            return parent::runSelect();
        });
    }

    /**
     * Returns a Unique String that can identify this Query.
     *
     * @return string
     */
    protected function getCacheKey()
    {
        return json_encode([
            $this->toSql() => $this->getBindings()
        ]);
    }
}

So, we're caching every query by it's SQL and it's Bindings. Neat, right? But what's this all about?

Cache::store('request')->...

That's right, we need to set up a custom Cache Driver. We're doing this so that we don't mess with any existing Cache logic you might already be using. Here's how I've defined this:

~/config/cache.php

'stores' => [
    'request' => [
        'driver' => 'array'
    ]
]

In case you're wondering, the array driver is an in-memory Cache store that just stores everything inside an array. This is typically wiped at the end of each request, as once the application gives a response, this is naturally freed from memory. This is actually exactly what we want, as we only care about queries being run within the same request.

Just in case you decide to set things up differently, you can flush the Cache at the end of every request by adding this:

~/public/index.php

try
{
    // Flush the Request Cache
    $app->make('cache')->store('request')->flush();
}
catch(ReflectionException $ex)
{
    // Do nothing
}

After the $kernel->terminate($request, $response) statement.

Now, don't go trying this yet, as it's not going to work. Remember how we "created" a custom Query Builder? Well, we have to tell our Models how to use it. This can be done with the following trait:

~/app/Support/Database/CacheQueryBuilder.php

<?php

namespace App\Support\Database;

trait CacheQueryBuilder
{
    /**
     * Get a new query builder instance for the connection.
     *
     * @return \Illuminate\Database\Query\Builder
     */
    protected function newBaseQueryBuilder()
    {
        $conn = $this->getConnection();

        $grammar = $conn->getQueryGrammar();

        return new Builder($conn, $grammar, $conn->getPostProcessor());
    }
}

You'll want to slap this on any Model that you want to use this new feature:

~/.../SomeModel.php

<?php

namespace App;

use App\Support\Database\CacheQueryBuilder;
use Illuminate\Database\Eloquent\Model;

class SomeModel extends BaseModel
{
    /**
     * For Caching all Queries.
     */
    use CacheQueryBuilder;
}

And viola! Give it a try! See what happens!

0 likes
13 replies
therons's avatar

This is sweet, any use cases where one would not want to do this?

1 like
martinbean's avatar

any use cases where one would not want to do this?

@therons Where data is manipulated in a single request:

$notifications = $user->notifications()->paginate(15);

$notifications->each->markAsRead();

$notifications = $user->notifications()->paginate(15);

Even those an UPDATE has occurred, the second paginate() will just return the results of the first one as there’s nothing to differentiate it from the first time the query was ran (and then cached).

I imagine the same will be true when trying to load “fresh” model instances, i.e. $order->fresh(). If you’ve already loaded the model, fresh() is just going to retrieve it from the cache, rather than query it from the database again.

In my opinion, caching doesn’t belong at the database abstraction layer like this for the reasons above, and other use cases. If you’re application is making lots of queries, fix that. Don’t just put a “catch-all” solution somewhere else and hope it fixes your problems. If you have an expensive query, cache it where it’s executed:

class Report extends Model
{
    public function generate()
    {
        // Pretend, super-expensive query, so cache it
        return Cache::remember('report', 60, function () {
            return Report::all();
        });
    }
}
// Will be cached, because we explicitly want this method cached
$report = Report::generate();
5 likes
Cronix's avatar

I bet you have Barryvdh's Debug Bar or some equivalent enabled, and you're seeing tons of duplicate queries, and you simply don't have the time to track them all down.

If you enable the backtrace option in the db section of the debugbar config file (disabled by default), it shows the file and line number that initiated each query, so it's really not hard to "track them all down"

7 likes
jimmck's avatar

PDO bindings require an active connection. Once the connection is closed the bindings are stale. Really want to reduce duplicate queries, use Join.

bhushan's avatar

is this working .. cause i could not see difference in number of queries in Telescope

bhushan's avatar

It says Cache is Set and Missed but not Hit.

I believe its not using cache

andrewc's avatar

This worked first time on my local dev environment but when deploying on production, it died with Cache store [request] is not defined.

FIXED - dont forget to clear the config cache! php artisan config:clear

sdj's avatar

The update/create problem can be solved by clearing the cache on model updated/created events. Are there any other reasons one should not use this? It might be lazy/bad practice but it would save a lot of time hunting duplicate queries.

andypl100's avatar

We use something similar to this in our projects and it works very well.

You can hook Laravel's 'Illuminate\Database\Events\QueryExecuted'' event and clear the cache when any non "select" statement is run. This solves the write/update issue in a central place. Super easy to implement.

Please or to participate in this conversation.