Task Scheduling in a big environment with two or more application servers

Published 8 months ago by araeuchle

Hello guys. My first post here! We have started creating a new web application using Laravel 5.5. We are expecting to have several thousands of users so we will host this application on AWS using a Load Balancer and two application servers for the start.

To my question: I have given the following task to create a Job that runs daily at 00:00. Check if a user have birthday, if he has birthday, sent him an email. Also check for his age and depending on his new age he gets a new user role.

I realised that Laravel's build Task Scheduler should be perfect for it, but it wants me to a cronjob every one minute. So i am looking for some solution were i can build something that is really reliable to call long running tasks.

Any help is really appreciated!

martinbean

@araeuchle I could have sworn Laravel added a method to run the scheduler on a single machine in a cluster environment, but struggling to find what the method’s called.

I’ll update this reply when I find it.

martinbean

@araeuchle Ah, so it seems using withoutOverlapping() will store a lock in the cache, as opposed to that single machine’s filesystem. So if another server attempts to run the same command, it’ll check the cache, find the lock, and not run:

https://github.com/laravel/framework/pull/16196

araeuchle

@martinbean: Thanks for your answer.

So lets assume i have two application servers running and are connected through Load Balancer. I am build a job and add the job. when i call withoutOverlapping() it should prevent to be Double Triggered. is this right? :)

araeuchle

My understanding problem atm is the following:

The command is called at the same time. so Laravel will have two instances trying to execute the jobs. How is decided which app is executing the job AND is the job really blocked when hit by the other instance first? :)

martinbean

@araeuchle Well as per my answer and the PR linked, it stores a lock in a cache both application servers use. So it’ll be a case of first come, first served. The first server to attempt the job will create a lock and then process the job, and when the second server attempts the job it’ll check if there’s a lock, see it, and not process it.

araeuchle

@martinbean: ok awesome! that what i was looking for! thanks for your help!

Please sign in or create an account to participate in this conversation.