I created:
2x workers watching high prirority queue 1 worker watching mid priority queue 1x worker watching high and mid priority queue 2x workers watching mid and low priority queue
Be part of JetBrains PHPverse 2026 on June 9 – a free online event bringing PHP devs worldwide together.
I have an app which user uploads a list of phone numbers, and api request is made, then updates a value tied to those phone numbers. Its a spark app with many users. If user 1 uploads a list, and process begins to check phone numbers (potentially 100k phone numbers so it takes some time).
I have setup 3 queues for high, mid, and low priority, and 3 workers to run 3 jobs concurrently MAX. Each job and the upload_id, not individual phone numbers.
If I have a job on the high priority queue, and a job in the medium priority queue, job 1 (high priority) begins to process as expected, but the other workers do not pick up jobs from the mid priority queue until the high priority queue job is completed.
If I have 3 workers, and a job in high, medium, and low priority queue, how can i make the workers process each, rather than wait for the high priority job to complete?
I created:
2x workers watching high prirority queue 1 worker watching mid priority queue 1x worker watching high and mid priority queue 2x workers watching mid and low priority queue
Please or to participate in this conversation.