Understanding queues better Hello,
I'm approaching for the first time queues. Please let me know if a I got it right.
I have the following environment:
1 x www server with laravel "installed"
1 x redis server
1 x encoding server laravel project synced from www server
The job is created on www server, then on the encoding server queue:work is launched and magically the job is performed on the encoding server.
I have reduced it to max simplicity.
Can you confirm that I have understood it rightly?
Thanks,
Simon
have you proved that it needs a second server?
Why do you need a specific server for the jobs ?
@thesimons And why do you need a specific server for video encoding ? Can you explain ?
@vincent15000 Because on the encoding server I have 96 cores available just for encoding videos. Encoding on the www node would mean having an almost fixed LA of about 50.
@thesimons Ok now I understand.
The job is created on www server, then on the encoding server queue:work is launched and magically the job is performed on the encoding server.
When you create a queued job, it's saved on Redis. Then the queue worker consumes the jobs registered on Redis.
Nothing magic ;).
You probably store the videos on a specific storage ?
@vincent15000 Video sources are saved into S3 bucket, then the encoding server take it from the S3 bucket, work on it locally, and then save it into S3 bucket once again.
@thesimons And what don't you have understood correctly about queues ?
@vincent15000 I have found the solution to my problems
public $queue = 'emails';
ProcessEmails::dispatch()->onQueue('emails');
php artisan queue:work --queue=emails
Just some random pieces of code but now I got how to handle different queues and have different workers each one handling a specific queue.
Thanks,
Simon
@thesimons You didn't speak about any problem, you just asked if you had the right approach.
I would also look at horizon. This will give you a better picture of workload across your queues and allow you to balance workload
Please sign in or create an account to participate in this conversation.