Be part of JetBrains PHPverse 2026 on June 9 – a free online event bringing PHP devs worldwide together.

eddy1992's avatar

Optimizing query using chunk

Hi, I have a snippet code which will take records from one table and add it to a new table. The problem with it is that it will use a lot of memory and it won't work as we have a lot of records.

The code snippet looks like this

        $timeparts = explode(" ",microtime());
    $carbon = Carbon::now()->subMonths(2);
    $newdates = bcadd(($timeparts[0]*1000), bcmul($carbon->timestamp, 1000));

    $products = DB::table('products1')->where('startdate', '>', $newdates)->get();
    
    $data = collect($products)->map(function($item){ 
        unset($item->multi_new);
        unset($item->subCategoryId);
        return (array) $item; 
    })->toArray(); 

    $add = DB::table('advertisements')->insert($data);

How can I improve this code by using chunks method? Please assist. Thank you.

0 likes
4 replies
Borisu's avatar

You can chunk the data like this:

$chunks = collect($products)->map(function($item){ 
        unset($item->multi_new);
        unset($item->subCategoryId);
        return (array) $item; 
    })->chunk(num_of_chunks_here)->toArray();

foreach ($chunks as $chunk) {
    DB::table('advirtisments')->insert($chunk);
        // maybe even use sleep(number_of_seconds);
}
// if you want to be really thorough at this point:
unset($chunk);
1 like
MikeMacDowell's avatar

@eddy1992 as an alternative you can chunk the original DB query to limit the memory usage.

DB::table('products1')->where('startdate', '>', $newdates)->chunk(100, function ($products) {
        $data = collect($products)->map(function($item){ 
            unset($item->multi_new);
            unset($item->subCategoryId);
            return (array) $item; 
        })->toArray();

    $add = DB::table('advertisements')->insert($data);
});
1 like
D9705996's avatar
D9705996
Best Answer
Level 51

Why do you not want to use chunk as its specifically designed for this sort of scenario

https://laravel.com/docs/5.7/eloquent#chunking-results

you should be able to do


$timeparts = explode(" ",microtime());
$carbon = Carbon::now()->subMonths(2);
$newdates = bcadd(($timeparts[0]*1000), bcmul($carbon->timestamp, 1000));

DB::table('products1')
    ->where('startdate', '>', $newdates)
    ->chunk(100, function ($products) {
        DB::table('advertisements')->insert(
            $products->except(['multi_new','subCategoryId'])->toArray();
        );
    });

If you cannot use chunk and need to reduce the memory usage you would need to split your two month query into smaller queries e.g. per day and execute in a loop but this hardly seems worth it as chunk is effectively doing the same.

Please or to participate in this conversation.