Queue vs. Schedule performance

I understand the functionality of Queue vs. Scheduled task. My question is not about the function, but the performance differences of each. Namely: when I use $schedule->command('command:name')->whateverSchedule() is the scheduler running any commands that match the schedule timing as synchronous processes? Or is it going to start a new background process of each task it finds, more similarly to how a queue would run?

burlresearch

7 years ago

Best Answer

Level 41

When you're running the $schedule->command('command:name') version - you will have a CRON job running on the server to wake your Laravel app (every minute, or whatever you set) and see if there are any scheduled Jobs that should be run, since the last time it ran, if so, it runs them, serially. This is akin to 'Polling' and rely's on the CRON daemon invoking Laravel over and over again to see if there is something to do. So, like most polling, 99.99% of the calls are a small bit of overhead.

Queues, on the other hand, are more akin to 'Work Pile' job processing. A bunch of jobs are [serialized] and queued to run, as soon as possible, in order, subject to the resources you have allocated to the system. These rely on some sort of Queue Processor, either hosted on your server (like Redis, or Beanstalk), or from a 3rd party service (like SQS).

Laravel has a nice piece of kit in Horizon that makes setting up Asynchronous queues on your server a snap, along with some pretty nice monitoring for it all. This asynchrony does introduce some complexity to your system though, so beware, if your 'workpile jobs' compete or share resources, you can run into concurrency issues. Some of which have been documented here on Laracasts.

I'm not sure if that answers your 'performance' question - but it should help with understanding the differences between the two. I've often found it useful to use them together:

At 08:00 wake up and download a bunch of stuff; then foreach item downloaded, Queue a job to process it; then Queue a job to cleanup the downloads and do analytics counts for everything that just happened; then sleep until tomorrow...

1 like

kylearch

7 years ago

Level 1

Thanks! I think the answer I was looking for is in there:

...if so, it runs them, serially.

That was my concern/question, was if, behind the scenes, each of the scheduled tasks get offloaded to be processed, or if like you said they get processed by the cron task itself.

mnengwa

4 years ago

Level 1

@kylearch Does this change things? Running scheduled tasks in the background

$schedule->command('analytics:report')
         ->daily()
         ->runInBackground();

Please or to participate in this conversation.