Laravel horizon workers failed due to memory limit in production environnement.
I tried to increase memory limit in config file horizon.php but still the sames errors.
And also tried to debug the job(Images process) by getting the memory usage and it doesn't go beyong 400M.
My php memory_limit = 768M
Here is my supervisor configuration
'supervisor-1' => [
'connection' => 'redis',
'queue' => ['default', 'uploads', 'amounts'],
'balance' => 'simple',
'processes' => 3,
'memory'=> 768,
'timeout'=> 360,
'tries' => 3,
]
PS: The same configuration works fine in local where php memory_limit = 128M
Your assumption that this works locally and not on production is incorrect I guess. Because your production might have more data and because of that the memory usage is more as well.
Anyway, if you change the memory of supervisor you have to restart it as well!
What does your php.ini say about memory?
In the production horizon is launched with the command
php -dmemory_limit=768M artisan horizon
so the memory is set to 768M
I know this is an old thread, it seemed better than starting a new one.
Does anyone know how the memory_limit options work here? It's not in the documentation.
if I run php -d memory_limit=5G artisan horizon
Will each job have 5G of memory? Or does Horizon execute a new PHP process which won't have this memory limit set?
I have looked in the code and found the MonitorSupervisorMemory which just checks how much memory is being used and then terminates, this is of course different to memory set in php.ini. So if you set the limit to 500MB and php.ini only let you have 5MB it would still fail.
This makes me think setting memory in horizon.php won't make a difference.
I can't find in the code where a process is run or whether the memory usage is passed to it
It will still have the limits set in your php.ini file.
Ah, so there's no way to set higher memory limit for some queues? Or even to set a custom .ini file?
That's really annoying.
Do you happen to know where the PHP process is created? Maybe horizon can be overwritten or extended in some way.
Hey @robjbrain , did you manage to find a way to fix it?
From my reading, the memory limit set in Horizons config is a very soft limit https://github.com/illuminate/queue/blob/master/Worker.php#L302 after processing a job it checks if it is currently exceeding the memory limit set in Horizon config, and if so, terminates itself It doesn’t stop memory exceeding that limit during execution of a job
Please or to participate in this conversation.