Updating results of large datasets is super slow

@Tray2 I don't think it would be a slow database query since the first query I make is 'fast' independent of the filtering I choose. It only seems to slow down or stall out when I make a second query and the data changes. I noticed this same behavior on another page of the site that had a lot of data loaded. The answer is probably to simply reduce the data size when working with livewire since it has to do 'morphing / hydrating' of the data. There may be a solution to make it work with large data but my livewire ignorance is preventing me from resolving the issue. I actually just rewrote the page to simply use ajax and it works well...slow (a few seconds for each request) because of the data size I am requesting...but it isn't stalling out like it was with livewire. For what it is worth, below is some code snippets that I was using when I was having the issue.

livewire class

<?php
namespace App\Livewire\Report;
use Livewire\Component;
class PAndL extends Component{

    public $filter = array();
    public $results = array();
    public $table = array();

    public function mount(){
        $this->filter['displayBy'] = 'client';
        $this->filter['period'] = 'yearly';        
    }

    public function queryData(){

        $viewModel = new \App\Models\ViewModels\Reports\PAndL($this->filter);
        $this->results = $viewModel->getResults();
        $this->table = $viewModel->getTableColumns();

    }

}

livewire blade component

 @if($results)
            <table class="standard sortable">
                <thead>
                    @if(count($table) > 1)
                    <tr>
                        <th rowspan="2">{{$filter['displayBy']}}</th>
                        @foreach($table as $period => $value)
                        @php $colspan = array_sum(array_column($value, 'include')); @endphp
                        <th @if($colspan> 1) colspan="{{$colspan}}" @endif>{{$period}}</th>
                        @endforeach
                    </tr>
                    @endif
                    <tr>
                        @if(count($table) == 1)
                        <th>{{$filter['displayBy']}}</th>
                        @endif

                        @foreach($table as $period)
                        @foreach($period as $key => $col)
                        @if($col['include'])
                        <th>{{$col['label']}}</th>
                        @endif
                        @endforeach
                        @endforeach
                    </tr>
                </thead>
                <tbody>
                    @php $i=0; @endphp
                    @foreach($results as $key => $result)
                    <tr wire:key="{{$filter['displayBy']}}_{{$filter['period']}}_{{$i}}">
                        <td>{{$key}}</td>
                    </tr>
                    @endforeach
                </tbody>
            </table>
            @endif

@quegs You will be able to see if the delay is on the server or on the client by simply observing the network traffic.

By your estimation, the response to the filters update should be recieved in a few seconds with the render taking a long time.

Just make sure you have a wire key, (probably related to the record ID) so that Livewire can quickly determine what has changed.

If it is dom diffing that is slowing it down, then I would revert to traditional server side rendered view (not livewire)

@quegs You could also give the tbody a wire key that changes on each change in filters. Livewire should avoid checking anything inside the tbody and swap it out in its entirety.

@Snapey Thanks. I'll give these suggestions a try. I hadn't thought of the putting a wire key on the tbody.

UPDATE: So I added a wire key to the tbody tag but that didn't help. I looked at the network traffic and this is what I see: when I submit the form an 'update' request is made and a status code of 200 is returned after a couple of seconds. The 'update' request is followed by a 'open?op=get&id=' request which is in pending state for about 5 minutes before it resolves or fails. Would that be DOM diffing that is occurring during that pending state?

@quegs So that means that something on the server side is taking time, so we are once again back to the query.

Let's make a simple example

Your first query does this.

SELECT * FROM some_table;

it just takes every record without looking at it.

Now add some conditions to it

SELECT * FROM table WHERE some_column = 'some_value' and some_other_column = 'some_other_value';

Now the it needs to look at two conditions, and without some kind of index on those columns, the database needs to look at each row and evaluate if it should be included or not. This takes time, and you can't say that the first query is fast, then the second should be, because they have completely different execution plans.

@Tray2 I guess that is where my lack of understanding comes into play with livewire. When I run the first query with some_column = 'foo' and then I run the second query with some_column = 'bar' then the second query is always super slow compared to the first query and the first query is always 'fast'. It doesn't matter if I was filtering by 'bar' in the first query or 'foo' in the first query. If I switch over to using AJAX then it simply runs what every query based on the options I send it and the queries always take approximately the same amount of time...a couple of seconds. I am guessing that the slowness results from the hydration of the livewire snapshot. If that is the case then it seems that I am either doing something wrong in how I am using livewire or livewire is not a good application for this unconventional amount of data that I am trying to display.

@quegs Do you always filter on the same column? If so you should probably create an index on that column. Big amounts of data is always a potential issue. Can you share your query, and the execution plan for it?

@quegs no. this second request is debugbar

It is this that is causing the issue. disable it and see

@Snapey I dug into things a little more and it seems the delay is at the livewire 'diff' function. My array was a multidimensional array so I thought maybe the 'dom diffing' was struggling because of that. I flattened the array but I am not seeing any improvement. I made sure the 'tr' wire key matched the index of the results but that didn't help either. I decided to simply send the data to the view via the render function...thus eliminating the livewire functionality for the data results but still keeping it for my form and that is working great...and maybe that is the best way for this particular application since all data changes anyway based on the selected filters of the form. Thanks for your spending the time to help.

@quegs So did you have debugbar installed?

When you bind a large amount of data to a public property, all the data is sent to the client, as well as being rendered in the view. Debugbar then tries to iterate over all the model data in order to display it in the debugbar view.

When you pass the data to the view via render, you are NOT sending as data to the client in addition to the server side rendered view.

Please note that when you send data via render DOM diffing still occurs. Its perfectly acceptable to change elements of the data sent via render and livewire will still swap out just the differences.

@Snapey Sorry...I should have mentioned that I did have debugbar installed. I disabled it and then I was only seeing the single update request but there wasn't any improvement in speed. The update request would give a status code of 200 after a couple of seconds but the response was blank for about 5 minutes at which point the results would finally be displayed. I looked under the devtools livewire sources debugger and all I saw it hitting during that time was the diff function so I assumed that the delay was happening in the stage. Thanks for explaining how the data is and isn't sent to the client...that is very helpful.