Laravel keep eating memory with every request

Published 8 months ago by asachanfbd

I was working to improve performance of my Laravel application and got below stats -

  • Request 1(db interactions) - 1238(Ram Used)
  • R2(db interactions) - 1250
  • R3(db intr.) - 1275
  • R4(db intr) - 1289
  • R4 - again(db intr) - 1295
  • R5(db intr) - 1313
  • R5 - again(db intr) - 1313
  • Rn - 1873(out of 2004MB total memory)

On performing tests on DB - No memory leak found.( I tried inserts in look and checked memory after every 10000th insert. memory was stable).

Extensions used -

  • "laravelcollective/html": "^5.2",
  • "gbrock/laravel-table": "^0.3.4",
  • "guzzlehttp/guzzle": "^6.1",
  • "lavary/laravel-menu": "^1.6",
  • "intervention/image": "^2.3",
  • "maatwebsite/excel": "^2.1",
  • "aws/aws-sdk-php-laravel": "^3.1",
  • "barryvdh/laravel-dompdf": "^0.7.0",
  • "yajra/laravel-datatables-oracle": "^6.0",
  • "rap2hpoutre/laravel-log-viewer": "^0.7.1",
  • "brozot/laravel-fcm": "^1.2"

Please suggest why Laravel is using all the RAM and not releasing it. It starts crashing at the end.

jlrdw
jlrdw
8 months ago (215,690 XP)

Are you running a query that returns a huge resultset rather that paginating? Also just load what's needed on a page. As an example if an edit page only uses some js for a popup calendar just load that js, not 10 million lines of other js.

asachanfbd
  1. No Queries are not returning huge result sets only required data.
  2. No JS is loaded requests are web services. JSON response only.
Cronix
Cronix
8 months ago (398,210 XP)

Laravel keeps all of the queries that have been ran in memory for debugging. I run some processes that run for hours inserting tens of thousands of records in the db. Each loop takes more memory until it crashes... until.. I discovered how to turn that off.

DB::disableQueryLog();
// run lots of queries

After doing that, laravel consumed no more than 5M memory for the entire thing.

jlrdw
jlrdw
8 months ago (215,690 XP)

@Cronix good to know, where do you put

DB::disableQueryLog();
Cronix
Cronix
8 months ago (398,210 XP)

I just have it before running a ton of queries, like when I'm pulling in data from apis and storing it in a huge loop.

asachanfbd

I had already tested with DB::disableQueryLog(); but this is not the issue.

My issue is memory is not released even after completion of request.

tdobry

I have the same issue. With every database query --> more memory is consumed and it will never released. Is there a solution?

artcore

Perhaps your queries are doing full row scans. If all rows are read into memory and you have a ton, that could be a cause. You can do an EXPLAIN to see if that's an issue.

36864
36864
1 month ago (42,370 XP)

What version of PHP are you using?

Can you check if it happens in both php 7.1 and php 7.2?

asachanfbd

My Issue was resolved but it was not because of queries memory, it was my stupidity.

Lots of queued jobs were executing at the same time and Cron Jobs were crashing after a timeout. Server was going full memory within 10-20 mins after a restart.

After handling the queued jobs issue, memory issue disappeared.

tdobry

I use PHP 7.2.2-1+ubuntu16.04.1+deb.sury.org+1.

I have only one Task. And I start the command with artisan.

$dCounter = 0;               
             
$data_array = array();
                
while ($row = fgetcsv($handle, 0, "|")) {
                                        
        if($dCounter>100)
        {      

             //Additional memory is used after the insert-function  
             Article::insert($data_array);
                        
             echo memory_get_usage() . "\n";
             unset($data_array);
                                                
             $data_array = array();
             $dCounter = 0;
         }                  

    array_push($data_array, $this->getAudioDataArray($row));

        $dCounter++;                            
        unset($row);
        \gc_collect_cycles();
}

fclose($handle);      

while(true)
{
       //The used memory is never released 
       echo memory_get_usage() . "\n";
       \gc_collect_cycles();
       sleep(1);
}

For example, I have the same problem with:


while(true)
{

     //Additional memory is used after request
     $Article = \App\Model\Article::where('ean', '=', $EAN)->first();

     unset($Article);   
}

36864
36864
1 month ago (42,370 XP)

Can you downgrade to 7.1 and see if the issue persists? I'm fairly sure there's a PDO memory leak issue in 7.2.

tdobry

@36864: You are my hero!!! You're right. There is no problem with PHP 7.1. Thanks alot!

jlrdw
jlrdw
1 month ago (215,690 XP)

I am then holding on to 7.1, thanks from me also.

Please sign in or create an account to participate in this conversation.