cboxdk's avatar

Laravel Horizon Queue Worker Configuration

Does anyone know if i can start different Queue Worker Configuration for servers? E.g. i have a supervisor config for my frontend and one for my worker servers.

It seems i can only run the artisan horizon command, but i see no parameterer here to tell which config to use.

0 likes
5 replies
jab1000's avatar

Yes, you can change the /config/hoirzon.php setup and add more "supervisors" to listen on different queues. For instance, I have 3 different "supervisors" setup:

'production' => [
            'supervisor-1' => [
                'connection' => 'redis',
                'queue' => ['default','property'],
                'balance' => 'auto',
                'processes' => 14,
                'tries' => 3,
            ],
            'supervisor-2' => [
                'connection' => 'redis',
                'queue' => ['photos'],
                'balance' => 'simple',
                'processes' => 10,
                'tries' => 3,
            ],
            'supervisor-3' => [
                'connection' => 'redis',
                'queue' => ['photo-delete'],
                'balance' => 'simple',
                'processes' => 3,
                'tries' => 2,
            ],
        ],

In my "supervisor" file it is purely running Laravel Horizon.... different than my Beanstalkd (different site/project) where I have multiple queues setup in the supervisor conf files.

BIG thing I have learned in last couple of days though is Horizon seems to CACHE EVERYTHING! If your "job code" needs updating, you have to terminate Horizon and start it back up again -- or it won't pick up on your updated "job" code. Multiple times I have terminated Horizon and then even rebooted server (unfortunately) to clear the "internal Horizon cache".

1 like
cboxdk's avatar

I know i can add multiple configurations, but i want specify per server which of the configurations to run. E.g. on server1 i want to run the supervisor-1 and on server2 i want to run supervisor-2 and supervisor-3 config.

I guess for now the only way is to change the environment from production to something like frontend and worker or similar, but not really what i want.

1 like
gregpeden's avatar

Just to add to this, as far as I know there is nothing wrong with including logic code in the config files, so long as the returned result is an associative array which conforms to the expectations of the application, and so long as the returned result is deterministic for a given server/deployment (for result caching reasons). So, create a custom variable your .env file, change the value on each of your deployments, and in the config file run a lookup and return the appropriate configuration.

bgarrison25's avatar

the environment keys match your APP_ENV in your .env file. So if you want to run different supervisors add in another key like so:

'uat' => [
        'supervisor-1' => [
            'connection' => 'redis',
            'queue' => ['default','property'],
            'balance' => 'auto',
            'processes' => 14,
            'tries' => 3,
        ],
        'supervisor-2' => [
            'connection' => 'redis',
            'queue' => ['photos'],
            'balance' => 'simple',
            'processes' => 10,
            'tries' => 3,
        ],
        'supervisor-3' => [
            'connection' => 'redis',
            'queue' => ['photo-delete'],
            'balance' => 'simple',
            'processes' => 3,
            'tries' => 2,
        ],
    ],

and change your .env APP_ENV to match it

cboxdk's avatar

Thank you @bgarrison25. The solution is already the one in mention in my previous post.

I guess for now the only way is to change the environment from production to something like frontend and worker or similar, but not really what i want.

What I would have liked was the ability to add an option to select the config instead as different servers may have different purposes. Changing the Environment have other side effects not wanted.

I solved it by moving my heavy jobs away from Horizon and into regular queues in this case. I found horizon was great for e.g. mails and events, but not heavy long running tasks anyway

Please or to participate in this conversation.