this is completely unnecessary please ignore, #fail

Intro


Today is started using queues and ran into a problem with my current setup. 
I managed to fix it and thought i share my solution with the community. 
 In case someone else runs into this and to see if there might be a better solution. 
 First of all please excuse my english it is not my native language.



Setup

Forge

• 4 servers, 1 loadbalancer, 3 webservers

• 3 website, dev.site.com, test.site.com, site.com

• 12 sites in forge, 1 per website per server


AWS


• Aurora RDS

• Elasticcache redis server

• Alot of other services


Envoyer


• 3 projects, one per site


• The projects are completly seperated, diffrent redisDB, Mysql DB etc

The problem


So i have this job that needs to be dispachted on the queue


dispatch(new UpdateFriends($user));

So i created a redis worker with the default queue on the first webserver but then i ran into
the following problem:


ReflectionException: Method __PHP_Incomplete_Class::handle() does not exist in /home/forge/dev.site.com/releases/20161219140033/vendor/laravel/framework/src/Illuminate/Container/Container.php:553


Because the job was dispachted randomly on webserver-3 and the worker was on webserver-1 
the `releases/201612191400 folder didn't exist.



So i needed a worker on every server that only processes jobs that originated from that server.



Solution


To fix this problem i did the following:



i added this line to config/app.php:

'machine' => gethostname(),



So now the application knows on wich server it is.



Then in config/queue.php i did this: 


'redis' => [
            'driver' => 'redis',
            'connection' => 'default',
            'queue' => config('app.machine'),
            'retry_after' => 90,
        ],

So now when you dispatch a job it will dispatch it to a queue with the name of the machine i originated from.



Then in forge i created 9 workers. 1 for every website for every server with the following settings:


Connection: redis

Queue: webserver-1/2/3 (name of server this site is on)

Maximum Seconds Per Job: 60

Rest Seconds When Empty: 10

Maximum Tries: 3

Environment: production/staging/development (value of app_env this site is on)
Run Worker As Daemon: yes (This is a MUST!)  



You need to run the worker as a deamon or else the next step wont work.

Because workers are long running processes you need te restart them after a code change or they won't pick it up.

In envoyer add the following Development hook on all the servers on all sites, i did it after activate release but it should be fine after the git pull.

Name: Restart workers
Run As: forge
Script: cd {{release}}
        php artisan queue:restart
On Servers: All

With this setup a worker on a webserver will only process the jobs that originated from that server and also receive code changes

You can now have working queues for multiple sites behind a loadbalancer with envoyer + forge

Final thoughts


I think this solution is relatively easy to implement.

Also when i need to add a server i only have to add 3 queues in forge en change 3 hooks in envoyer, that can be
done within a minute.


But if anyone has a better solution i would love to hear it. 
 Please keep in mind that i learned about queues only a few hours ago so this might all be completely unnessary :)