Laravel 5.8 memory leak

I wouldn't call this a memory leak. Your application requires more memory the longer it runs because it has to store more variables.

Case 1 requires more memory because$songs = $album->songs stores the query result in the $album model. The more iterations your loop has, the more memory it requires.

Case 2 behaves differently because it only overwrites the value of $songs with the current query result. The previous results aren't stored anywhere. This keeps the memory usage stable.

Case 3 uses the same amount of memory as case 1 but requires it all at once in the first line.

Yes this doesn't look like a "leak". Try filling up an array in your second example instead of overwriting the $songs variable.

$albums = Album::take(10000)->get();
$songs = [];
foreach ($albums as $album) {
    $songs[] = $album->songs()->get(); // why this don't cause the memory leak?
}

it would behave the same way as your first case I suppose

@staudenmeir @click I try to unset the variables every iteration, the memory still grows, is there any way to free the memory?

$albums = Album::take(10000)->get();
foreach ($albums as $album) {
    $songs = $album->songs; // cause memory leak
    unset($songs);
    unset($album); // can't free the memory, I think in this case is memory leak
}

What does your actual code look like? What are you doing the loop?

note that when you unset() anything it does not instantly free up the memory. The garbage collector needs to pass by first.

Some SO references:

• https://stackoverflow.com/questions/584960/whats-better-at-freeing-memory-with-php-unset-or-var-null
• https://stackoverflow.com/questions/31863777/php-having-some-memory-issues-inside-a-loop

btw, what you are seeing is not a real "memory leak" it is just consuming memory for all the data you collect.

And I doubt if unsetting $songs is actually saving you memory because it is only a reference to the real collection object that is stored in the $album model. So if you want to clear that you should also clear the relation itself maybe $album->setRelation('songs', null); can do the trick.

@staudenmeir @click Actual code looks like this: (also updated on GitHub) https://github.com/fukuball/Leak58

$albums = Album::take(10000)->get();

$i = 1;
foreach ($albums as $album) {
    $songs = $album->songs;
    echo "$album->id start \n";
    echo "#executions = " . $album->id . " - mem: " . memory_get_usage() . "\n";
    echo "$album->id end \n";
    $album->setRelation('songs', null);
    unset($songs);
    unset($album);
    $i++;
}

$album->setRelation('songs', null); still can do the trick, the memory goes up and never come back.

...
1261 start 
#executions = 1261 - mem: 14746184
1261 end 
1262 start 
#executions = 1262 - mem: 14746560
1262 end 
1263 start 
#executions = 1263 - mem: 14746936
1263 end 
1264 start 
#executions = 1264 - mem: 14747320
1264 end 
1265 start 
#executions = 1265 - mem: 14747696
1265 end 
1266 start 
#executions = 1266 - mem: 14748064
1266 end 
1267 start 
#executions = 1267 - mem: 14748456
1267 end 
1268 start 
#executions = 1268 - mem: 14748816
1268 end 
1269 start 
#executions = 1269 - mem: 14749200
1269 end
...

$songs = $album->songs;

is that a relationship? If so, it's performing a query (in a loop). It would be better to eager load them, then there would only be a total of 2 queries instead of N+1. Also, every query gets logged in memory. Running them in a loop like that is not good. You can disable query logging.

@cronix I know it's a N+1 query, but N+1 query should not cause the memory leak, I discuss by cases in the main article above.

I haven't looked into this but if you really think you've found an actual memory leak, you should probably open up an issue at the official github repo so that the devs can comment on it:

https://github.com/laravel/framework/issues

How about?

If you have a collection of $albums and then you foreach over those as $album, presumably $album is a reference to the member of the $albums collection?

When you load extra data into $album, actually you are expanding the member of $albums - memory which you never release.

When you execute ->get() you are perhaps creating new $song rather than expanding the content of the $albums collection?

@nash They want me ask here.

@snapey If the underlying mechanism is expanding the member of $albums, then we can explain why the memory grows, but it can't free the memory by $album->setRelation('songs', null);, I think it is a memory leak, do you think it's a memory leak?

It's not a memory leak it's bad code plain and simple. You have 10000 records that you load all songs from into memory. It will use alot of memory that later on will be released when the query is finished. If you load 10 albums or even 100 in one go instead of 10000 you will use up a more reasonable amount of memory and it will be released much faster.

Simple calculation

One album = 1Kb One album contains aprox, 10 songs of 512b each

(10000 x 1024b) + (10000 x 512b x 10) = 9.7Mb + 48.8Mb = 58.5Mb

if you need to process that many records:

1. use chunking
2. use RAW SQL (do the joins yourself to get relationships).

did you disable query logging?

If the underlying mechanism is expanding the member of $albums, then we can explain why the memory grows, but it can't free the memory by $album->setRelation('songs', null);, I think it is a memory leak, do you think it's a memory leak?

@fukuball PHP's memory is not instantly cleared when you set a variable to null or use unset(). This is simply how PHP works. It is cleared when the garbage collector is passing by. You can try to call it manually: gc_collect_cycles()

• https://stackoverflow.com/questions/3110235/php-garbage-collection-while-script-running
• https://www.php.net/manual/en/function.gc-collect-cycles.php
• https://php.net/manual/en/features.gc.collecting-cycles.php

Another approach is to execute your "memory heavy code" into a separate function as explained in an earlier link I've posted but also in a comment on the php doc site. This should automatically clean up the memory each time the function is finished. This only works if there are no other references to the variables.

// After testing, breaking up memory intensive code into a separate function allows the garbage collection to work.

For example the original code was like:-
while(true){
   //do memory intensive code
}

can be turned into something like:-
function intensive($parameters){
   //do memory intensive code
}

while(true){
   intensive($parameters);
}

// source: https://www.php.net/manual/en/features.gc.collecting-cycles.php#122544

@fukuball I've created my own quick test and it does seem to work well here.

Output without unsetting the variables

Memory Used for 832 Items:         2.41 mb
    0       510.63 kb
   25         10.4 mb
   50         5.78 mb
   75          2.1 mb
  100         1.93 mb
  125         4.41 mb
  150          4.3 mb
  175         1.49 mb
  200         4.93 mb
  225         4.88 mb
  ....
  775         3.61 mb
  800         6.31 mb
  825         6.47 mb
Memory Used:        230.8 mb

When I clear the relation it gives me:

Memory Used for 832 Items:         2.41 mb
    0        142.5 kb
   25        32.38 kb
   50         32.7 kb
   75        36.95 kb
  100        29.38 kb
  125               0
  150               0
  175               0
  200               0
  225               0
...
  775               0
  800               0
  825               0
Memory Used:         2.64 mb

See the difference at the end: 2.6mb vs 230mb

Just a quick piece of code, you can run it in your artisan console as long as you set your model and you children. It shows the memory difference between the last 25 items.

function formatmem($m) {
    if ($m) {
        $unit = array('b', 'kb', 'mb', 'gb', 'tb', 'pb');
        $m = @round($m / pow(1024, ($i = floor(log($m, 1024)))), 2) . ' ' . $unit[$i];
    }
    return str_pad($m, 15, ' ', STR_PAD_LEFT);
}

$memoryStart = memory_get_usage();
$items = SomeModel::take(10000)->get();
$memoryAfterItems = memory_get_usage();
echo 'Memory Used for '. $items->count() .' items: '. formatmem($memoryAfterItems - $memoryStart) . PHP_EOL;

$prevMemory = $memoryAfterItems;
foreach ($items as $i => $item) {
    $records = $item->children;
    $records = null;
    unset($records);

    // $item->setRelations([]); // this one performs better as the one below for me
    // $item->setRelation('children', null);

    if ($i % 25 === 0) {
        $currentMemory = memory_get_usage();
        echo str_pad($i, 5, ' ', STR_PAD_LEFT) . ' ' . formatmem($currentMemory - $prevMemory) . PHP_EOL;
        $prevMemory = $currentMemory;
    }
}

$memoryEnd = memory_get_usage();
echo 'Memory Used: '. formatmem($memoryEnd - $memoryStart) . PHP_EOL;

@cronix Yes, I disabled query logging.

@click I think $item->setRelations([]); solves my problem, thanks!

Hi @fukuball, I had a similar problem and in my case laravel/telescope and ramsey/uuid were the culprits. Later this week I'll try to go deeper in this problem to give more details.

@spelcaster Do you mind following up with some details here? We're trying to track a memory leak and use both of those packages.

@hotmeteor I did not use those specific packages used by @spelcaster but my issue was Sentry (Error logger), so probably similar behavior as Telescope.

My issue was:

This consumed memory

\DB::table('users')->get();

This did not:

$stmt = \DB::getPdo()->prepare('SELECT * FROM users');
$stmt->execute(); 
$stmt->fetchAll(); 

After a while I figured out my service provider for Sentry (pretty old version) was the culprit. After disabling it the leak was gone. What I did was instead of automatically including the service provider in config/app.php I only load it when necessary:

# AppServiceProvider::register() 
if ($conditionToLoadThis) {
    $this->app->register(SentryLaravelServiceProvider::class);
}

I don't know if this helps you out but try disabling all (custom) service providers you have and see if it has any effect.

If you have auto-discovery turned on (you probably have) than you can try disabling it via the dont-discover property in your composer.json: https://medium.com/@taylorotwell/package-auto-discovery-in-laravel-5-5-ea9e3ab20518