Be part of JetBrains PHPverse 2026 on June 9 – a free online event bringing PHP devs worldwide together.

TheBlueDragon's avatar

Using Laravel Queue for other PHP scripts

hello, I have a project which cant be to scale up with cron, the current project is having around 3k users with use of crontab to run every mint and execute functions, and to handle the number of resources need a dedicated server with more than 32GB just to run the cron as for each function, there is a cron file and this cron will execute this function for all users for example posts scheduling need to check every mint and publish these on the giving time so imagine we have 3k users who just try to post on the same day not to mention how the original structure of the custom framework used in this project make things to be improved difficult without doing a rewrite stuff

so I come with 2 solution

1 - is separate the functions to be run on different servers on a private network and connect to DB with remote connection 2 - is using Queue like what we have here in Laravel and use Redis or other custom drivers like RabbitMQ

now for solution 1 it's considered as a temp solution after some time it will reach to limits also and will have issues when the users become more and more than resources will be an issue I have around 7 to 10 functions need to be separate that a 10 servers to configure!!

now for option 2 is more logical also feel better but I face an issue, how to implement it.

at the moment

in my crontab I have to check every mint and run the cron.php file this class is doing a connection to DB get the table for the action required

processFunction {

    $this->schedulePosts();
    $this->emailSending();
    $this->notifications(); 

}

so these functions will be run one by one till they finish.

when I have around 100 user its ok a small VPS with 4gb can run them no issue but when its reach 3k the project need to have 32GB ram and still get delays.

so what I should do for solving this using Laravel Queue package

if the queue package can be migrated to the project, I just need to send the function? for example, $this->schedulePosts(); to queue and let the package do the magic, or I need to separate all functions to be on their own classes and pass them to the queue, or what!!

hope someone has the knowledge to guide me to solve this issue.

Thanks.

0 likes
5 replies
Snapey's avatar

3000 users is definitely not a lot, and I would not expect you to need such a huge amount of memory.

Sounds like you need to think about the architecture. There is no need to run the same jobs every minute if the data does not change.

When you say 3000 users, how many concurrent requests are you dealing with or is all your work from the cron jobs?

When those jobs run, how much actual work is done?

When those jobs run, do they finish before the next minute?

TheBlueDragon's avatar

@SNAPEY - mmmm can't explain in details about what type of actions but if its 3k users and each of them do 10 per mint so this is 30k on the same time

all things work with cron jobs this type of requests and actions need to connect to some api and do some requests and get result etc... not to mention some of them need to connect to proxy also

the cron now take 40 to 50 sec to finish 1 action from the 10 actions per user

so i make each action to have its own cron file

well some of them finish some of them no

Snapey's avatar

how can it help if you do more than one minute's work every minute?

You either need to do the work quicker or do less work.

How you organise queues will not help if you do not have the computing capacity or are being unrealistic in the number if things to do each minute

Please or to participate in this conversation.