Be part of JetBrains PHPverse 2026 on June 9 – a free online event bringing PHP devs worldwide together.

prince69's avatar

How optimize query performance?

I have 1M+ records. The code below works, but it takes too long to load because I'm using get() and then filter based on request data. I tried using SQL groupBy() directly, but got a MySQL ''Syntax error or access violation: 1055 Expression #1 of SELECT list is not in GROUP BY clause and contains nonaggregated column'''. For this I do not want to change strict mode false in config . How can I make this faster? Any suggestion to optimize it?

   $orders = Order::with([ 
            'customer:id,email,country_id,name',
            'orderDetails:id,invoice_no, tx_id', 
            'customer.address:id,title, address', 
            'statusLog:order_id,created_at' 
        ])
        ->select('id','customer_id','requested_at','approved_at','status','amount',  'commission_amount', 'group_id', 'type',  'created_at') 
        ->whereNotNull('group_id')
          ->get();

    $grouped = $orders->groupBy('group_id')->map(function ($items) use ($csvGroupBy) { 
        $firstItem = $items->last(); // I need to get last item of group by 
        $firstItem->amount = $items->sum('amount');
        $firstItem->total_requested = $items->count();
        return $firstItem;
    })->values();

    if (isset($request['invoice_no']) && $request['invoice_no']) { 
        $grouped = $grouped->filter(function ($item) use ($request) {
            return $item->orderDetails && $item->orderDetails ->invoice_no== $request['invoice_no']; 
        });
    }


    if (isset($request['amount']) && $request['amount']) {
        $grouped = $grouped->filter(function ($item) use ($request) {
            return $item->amount == $request['amount'];
        });
    }

    $page = $request->get('page', 1);
    $perPage = $request->get('per_page', $perPage);
    $orders = new \Illuminate\Pagination\LengthAwarePaginator(
        $grouped->forPage($page, $perPage),
        $grouped->count(),
        $perPage,
        $page,
        ['path' => url()->current(), 'query' => $request->query()]
    );
   return $orders;
1 like
11 replies
jlrdw's avatar

I suggest you study GROUP BY here:

https://www.mysqltutorial.org/mysql-basics/mysql-group-by/

And at times I query a query. And as @vincent15000 mentioned and to put a different way, let the database do it's job.

Also make sure you need a GROUP BY. Many times such as an accounts receivable report is a one to many.

company a
-----list of their receivables

company b
-----list of their receivables

I usually double paginate here:

Have company name and a next company link at top

And below that a paginated list of that companies receivables.

Of course I have a search company at top also
1 like
Tray2's avatar

I agree with both @vincent15000 and @jlrdw on what they say, and don't forget to paginate, if you have more than 100-ish records, then paginate, or you will have people complaining about the app being slow.

2 likes
jdc1898's avatar

The MySQL syntax error is a simple one to solve though. Just put your group_id column in the select statement. Assuming group_id is a foreign key, it has an index, the query even with 1M records should be quick enough. Paginating the results is secondary if you actually need 1M records returned. I suspect you don’t really.

1 like
vincent15000's avatar

@jdc1898

Just put your group_id column in the select statement.

The error rather say to add group_id in the group by clause.

jdc1898's avatar

@vincent15000 perhaps I am misreading his query, but it doesn’t look to have a groupBy in it. He isn’t doing the grouping until it is a collection as the get() is returning a collection.

1 like
vincent15000's avatar

Have you checked that all foreign keys are indexed in the database ?

krisi_gjika's avatar

your query seems strange, what are you actually trying to do? Why query orders if you only care about the last order of each group? I think you would have a much better time if you start the query from your Group model. Than you can:

  1. use a relation to load the last Order of each group.
  2. use withCount and withSum to do your calculations using orders relation
  3. use a whereHas constraint on the last Order relation to load only groups that pass your filters
  4. paginate your results for a view/api endpoint or use ->each(fn (Group $group) => ...) if you want to build some report about your orders.

just an example without knowing much about your db schema

Group::query()
  ->with([
    'lastOrder.customer.address', 'lastOrder.details', 'lastOrder.statusLog'
  ])
  ->withCount('orders')
  ->withSum('orders', 'amout')
  ->when($request->filled('invoice_no'), function (Builder $query)  {
      $query->whereHas('lastOrder', function (Builder $query) {
        $query->whereHas('details',  fn (Builder $q) => $q->where('invoice_no', $request->input('invoice_no'));
      });
  })
  ->paginate();
1 like

Please or to participate in this conversation.