Create a single queue worker processing multiple queues

Couldn't you just specify the queues as a comma separated list in Forge, the same way you'd do it as a regular artisan command?

@bastiaan89 I've tried that but it does not work. Forge requires that you enter a value for the Connection and Queue fields. Based on my config/queues.php those values are:

Connection: high,medium,low
Queue: high,medium,low

If I enter them in Forge, it spawns a process with this command:

php artisan queue:listen high,medium,low --timeout=60 --sleep=10 --quiet --queue=high,medium,low

This command fails to work through the queue. I can ssh into the droplet and run the following command:

php artisan queue:listen --queue=high,medium,low

and it will run as expected.

I'm not sure that Forge will let me do what I want (via the web UI at least).

Correction: Forge does not require you to enter a value for "Queue" but if you leave it blank, it assumes the value is "default":

php artisan queue:work high,medium,low --queue=default --delay=0 --memory=128 --sleep=10 --tries=0 --env=staging

Once again, this will not work as each of my connection/queues in config/queues.php has a different queue name. If I was to set them all to "default", then it would defeat the purpose of attempting to prioritise queued jobs.

Well in that case, I'd advise you to manually edit the supervisor file, located at /etc/supervisor/conf.d/worker-something. In here, the artisan command is stored. After that, run sudo service supervisord restart, or sudo supervisorctl and then reread.

At times, this queue worker will be doing some very memory intensive tasks, which is why I'd like a single worker instead of multiple, concurrent workers (which is how I assume Forge would run each of the queue workers).

Would it make sense in your use case to instead have one worker that works on the queue for larger jobs (assuming larger jobs are all going into a single queue), and another worker that works on the multiple smaller queues?

(I'm guessing you thought of that and decided you want them all to be worked on 1 at a time instead of with any concurrency at all, but just throwing that out there in case).

Another option is to spin up a server that's use purely for workers, which might free up some concern on CPU/Memory usage.

Well in that case, I'd advise you to manually edit the supervisor file, located at /etc/supervisor/conf.d/worker-something. In here, the artisan command is stored. After that, run sudo service supervisord restart, or sudo supervisorctl and then reread.

@bastiaan89 I was hoping I could keep everything within the Forge GUI but if it's not supported, I'll have to do this. Thanks

Would it make sense in your use case to instead have one worker that works on the queue for larger jobs (assuming larger jobs are all going into a single queue), and another worker that works on the multiple smaller queues?

@fideloper The current plan is to have 2-4 servers (Digital Ocean droplets) as dedicated workers. One of these will be a faster/bigger droplet than the others, which will focus on handling the larger, memory intensive tasks (generating ZIP files). However, when there are no ZIP file jobs waiting, it would be ideal for it to process the smaller (image processing) jobs while waiting. I was hoping (assuming) I could set up worker/queue priorities using Forge GUI but it doesn't seem to be supported. I'll manually set up the supervisord to work as required.

Loved your book too, it was very helpful! :)

It turns out that the GUI does support what I'm after. Take this config/queues.php setup:

'connections' => [

    'high' => [
        'driver' => 'database',
        'table' => 'jobs',
        'queue' => 'high',
        'expire' => 60,
    ],

    'medium' => [
        'driver' => 'database',
        'table' => 'jobs',
        'queue' => 'medium',
        'expire' => 60,
    ],

    'low' => [
        'driver' => 'database',
        'table' => 'jobs',
        'queue' => 'low',
        'expire' => 60,
    ],

],

The array key is mostly irrelevant in this setup, so Forge should be set up with the following:

Connection: high         // It can be "high", "medium" OR "low" because they all share the same "driver" details
Queue: high,medium,low   // Set the queues in the priority they should be processed.

It's pretty obvious and I'm not sure how I didn't get that working earlier.

Thanks for everyone's help.