Level 73
It is impossible to answer without access to the server, but if I were you I would start running top or htop to see what is using the resources. I'm guessing here that you are running linux and not Widows.
Dec 9, 2023
4
Level 1
Server idle memory and peak memory too high with laravel horizon
Hello everyone. To give some background, our software is deployed on a multi-tenant architecture. The application is built in laravel 9. For every customer a separate application is deployed, each with its separate database and queue. All of the applications are present on a single VM and are delivered by separate subdomains for every client. Previously we were using a database queue but now we have recently migrated to laravel horizon.
Our app uses periodically executed jobs to sync data from third party channels. For every client, these jobs run every 15 minutes. Our VM is 8 core, 32 GB. When idle, the memory usage is around 24 GB. Every 15 minutes when the jobs are executed for every client our memory reaches around 30 GB. This causes problems as some jobs fail to execute when not able to allocate enough memory.
With the old system of database queues, for every client we had 5 workers which processed different kinds of jobs. At peak time we reached 23 GB of RAM usage. Now with horizon we only have 2 supervisor groups. One group has max 1 worker while the other can scale up to 5 workers on significant workload. We switched to horizon as we were getting lock wait timeout errors when enqueueing jobs.
My question is why is the peak RAM usage so high when using horizon (24 GB to 30 GB on horizon) when compared to the database queues (19 GB to 23 GB). And how can we reduce this?
Level 1
@Tray2 We are indeed using linux. Ubuntu 18.04. Upon running htop and sorting by memory usage I see that there are over 10 mysqld processes each taking about 1 GB each and they are followed by 4 redis-server processes taking 500 MB each.
Level 73
@concurrent_Manifest What happens if you disable the workers that does the things in the DB, does it still have 10 processes of about 1Gb each?
Level 1
@Tray2 I tried this in the dev environment, the mysqld processes are still present.
Please or to participate in this conversation.