What's the purpose of the additional :notify queue in redis-based queues?
Hello,
I am doing a proof of concept on pushing jobs from a laravel app onto a redis queue, then have those jobs processed by nodejs workers and returning results back to the laravel app. That went well and jobs popped out by the nodejs workers are removed from the redis queue.
However, for each job pushed onto a given queue, Laravel also pushes an item in a queue:notify queue it creates as well. All those items have a single value of 1.
I am also removing those queue:notify values from the nodejs workers but I wonder what they are for and what I am breaking by removing them.
I have searched the code - this queue is basically managed only in the LuaScripts - and could not find any hint as to what they are for.
Does anyone know or can you point me at some documentation?
After spending a bit more time, I understand now how this is used and why. Here is a what I understood for reference:
Popping a job out of a queue is done with Redis lpop() function which is not blocking, ie it returns immediately either with a job or returning false if no job is found
One can use instead the blpop() redis function for that: the difference is that if no job is present, the connection is blocked until another client pushes a new job onto the queue
Laravel wants to use blpop() so that we wait efficiently (without wasting CPU) until a job is present. But as it uses only one connection, if it was listening with blpop() to the main queue, it could never add a new job as the connection would be blocked waiting
so instead it waits with blpop() on separate queue (queue:notify): when a job is added to the main queue, a value is also pushed to the queue:notify. This "unblocks" the pending blpop() and we now a job is available on the queue: it can be then read with lpop.
Hi @weeblr i don't understand a thing.
Which is the difference between blocking with the blpop the :notify queue instead of the main one?
I see that the push pushes on both queues.
Thanks
