Be part of JetBrains PHPverse 2026 on June 9 – a free online event bringing PHP devs worldwide together.

sergionader's avatar

Laravel 5.5/Scout: how to extend Scout to use Eloquent joins?

Hello. This is my first post here and have to say I am glad to join the Laracasts community! Let's go, then!

I am using Laravel/Scout/Elasticsearch via https://packagist.org/packages/babenkoivan/scout-elasticsearch-driver and I have this situation:

  1. To search, I must use a model like this:
$visits = Visit::search("canada")->with(['origin', 'profile', 'user'])->paginate(10);

It works very well.`

  1. As you can see, this model is related to three other tables: origins (that are countries), profile and users. The way everything is set, I can search for "canada" and get all the visits that came from Canada, what is fine as well.

  2. The result is shown in a grid using paginate(), what works well, too.

  3. However, if I want to sort the grid by Origins -- for everything I read and tried so far -- I can't use the $model::search('canada')->with(...) approach. Instead, I have to use something like

$visits = DB::table('visits') 
->join('origins', 'origins.id', 'visits.origin_id')
->orderby($sort_column, $sort_az_za)
->paginate(10);

This is the thing: the search() will not work with the DB::table('visits') syntax and Visit::search("canada")->with(....)... approach does not allow me to sort by a field that is not in the Visits table. BTW,

Visit::search("$query")->join('origins', 'origins.id', 'visits.origin_id')...

will not work as well.

The question is: how can I search and sort the results by any field I want?

I thought that it should be possible to extend Scout to search DB::table('TABLE') or to try to sort directly via ES -- though I would prefer to keep all the code bound to the "Laravel way".

If extending the DB::table('TABLE') is a way, could someone point me where to start?

Other than that, any other ideas/insights?

Thanks!

0 likes
8 replies
sergionader's avatar

No ideas, guys?

I believe that more people have been faced (or facing) it as having a searchable and sortable grid is a common need. Thanks?

pierrem's avatar
Visit::where('column', 'LIKE', '%$query%')->join('origins', 'origins.id', 'visits.origin_id')
?
sergionader's avatar

Thanks, Pierre! Unfortunately, it will not work.

Visit::where('column', 'LIKE', '%$query%')
->join('origins', 'origins.id', 'visits.origin_id')
->search('canada')
->get();

will produce:

Call to undefined method Illuminate\Database\Query\Builder::search()

and

$visits = Visit::search('canada')
->where('column', 'LIKE', '%$query%')
->join('origins', 'origins.id', 'visits.origin_id')
->get();

will give us:

Method join does not exist. SQLSTATE[42S22]: Column not found: 1054 Unknown column 'origin_name' in 'order clause' (SQL: select * from visits order by origin_name asc limit 10 offset 0)

I am trying to figure out how to extend ../vendor/laravel/framework/src/Illuminate/Database/Query/Builder.php to "understand" the search() method.

Any suggestions are welcome!

CcrisS's avatar

Hi!

If you go to http://<your-elastic-host>/<your-elastic-index>/_mappings, you can see how your "visit" model are mapped.

Maybe you have this structure: visit > origin > { id, name, ... }, and you can do:

$visits = Visit::search('canada')->with(['origin', 'profile', 'user'])->orderBy('origin.name.keyword')->paginate(10);

I hope it helps you :)

sergionader's avatar

Hi, @CcrisS. Thanks for the post, but it won't work. The mapping shows

   "origin_name": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        },
                        "fielddata": true

but

$sort_column = 'origin_name';
$visits = Visit::search("canada")
            ->orderby($sort_column, $sort_az_za)
            ->with(['origin', 'profile', 'user'])
            ->paginate(10); 

will give us:

SQLSTATE[42S22]: Column not found: 1054 Unknown column 'origin_name' in 'order clause' (SQL: select * from visits order by origin_name asc limit 10 offset 0)

The whole problem is that the Visit model does not know the origin table even with the chained "with('origin') method.

Does anyone know how we can make Scout understand the following syntax?

$visits = DB::table('visits') 
->join('origins', 'origins.id', 'visits.origin_id')
->orderby($sort_column, $sort_az_za)
->paginate(10);
sergionader's avatar

Update: I give Algolia a try and:

  1. it searches very well related tables. It also has fuzzy search out of the box and a basic web interface for index management. To make it work, it's important to set up the main model. In my case, the model Visit has:
  public function toSearchableArray()
    {
        $array = $this->toArray();
        $array['products'] = $this->products->toArray();
        $array['origin'] = $this->origin->toArray();
        $array['profile'] = $this->profile->toArray();
        $array['user'] = $this->user->toArray();
        return $array;
   }

Searching is a breeze:

  $visits = Visit::search("$query")
            ->paginate(10);

Works very fast and nice but... -- and this is very annoying but -- I can't sort the resulting grid by any field I want. I dug a little and found out the following:

  1. To sort, it's necessary to create an index replica. That's easy and makes sense;
  2. only numeric fields can be sorted -- and this is very weird...
  3. for instance, to sort the date, I created a new field called dt_unix where I store strtotime($date). Then I created two indices replicas: dt_asc and dt_desc.
  4. When I want to sort I just have to indicate which index should be used. Here is how to set the index on the fly:
  $index_name = 'dt_desc';
  $visits = Visit::search("$query")
            ->within($index_name)
            ->paginate(10);
  1. However, only the fields on the main table/model can be sorted. Thus, if I want to sort Origin (by any numeric criteria I could create), there is no way to set the index to sort by a field from a related table. Maybe using an "inverted model", i.e, Origin and the "related" Visits, it can be done -- it does look very messy, kind of the patching the workaround!

If I am missing something, please let me know. The only thing I need is to have a server paginated grid that is searchable and sortable -- sure, it's a very common thing.

As always, any ideas are welcome!

EDIT: I also tried the algolia instantsearch for building the search interface. It's nice, but not what I want righ now.

CcrisS's avatar

The "toSearchableArray" function is a scout functionallity, it works for Elasticsearch too. Now that your "visit" index knows about "origins", you can try this again (with scout.driver = 'elasticsearch'):

$visits = Visit::search('canada')->with(['origin', 'profile', 'user'])->orderBy('origin.name.keyword')->paginate(10);

It's important to use the "keyword" field (you can see it in your mappings).

sergionader's avatar

Hello, Ccris! Thanks for the hint. I've changed back to Elastic Search two days ago - now using Erick's driver, that sorts related tables out of the box: https://github.com/ErickTamayo/laravel-scout-elastic. This driver relays on you to set up whatever you want via ES API and does a great job so far. I will soon post the complete code. Thanks!

Please or to participate in this conversation.