Queued Jobs "ShouldBeUnique" Should Actually Be Unique But At Times Isn't
Hi all, I was wondering if anyone has encountered this. We're using queued jobs pretty heavily within our application and I've been making use of the "ShouldBeUnique" interface to prevent the same exact job from running concurrently. While investigating some things that should have happened but didn't (always fun), I noticed that in certain cases some jobs that should indeed be considered unique (same job class but with different parameters) were not actually being pushed to the queue. I'm assuming this is due to an atomic lock not being acquired but a bit stumped as to why exactly.
Re: the environment, we're running Laravel 10.48.4 and using Horizon with Redis as the queue driver. I've put together a very simple example of a job that is not being dispatched even though it should be unique.
<?php
namespace App\Jobs;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\{ShouldBeUnique, ShouldQueue};
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Log;
class TestJob implements ShouldQueue, ShouldBeUnique
{
use Dispatchable, InteractsWithQueue, Queueable;
public function __construct(public int $id)
{
Log::info('__construct(): ' . $this->uniqueId());
}
public function handle(): void
{
Log::info('handle(): ' . $this->uniqueId());
}
public function uniqueId(): string
{
return (string) $this->id;
}
}
From within a controller class I'm then dispatching 3 should-be-unique instances of TestJob and watching the output of both Horizon and the application log.
$this->dispatch(new TestJob(1));
$this->dispatch(new TestJob(2));
$this->dispatch(new TestJob(3));
This is essentially how it's done in the docs: laravel.com/docs/10.x/queues#unique-jobs
/**
* Get the unique ID for the job.
*/
public function uniqueId(): string
{
return $this->product->id;
}
However when I look at horizon and the logs, the log output from the handle() method is never appearing and the job isn't being executed / actually pushed to the queue:
┌ 22:44:59 INFO | __construct(): 1 │
┌ 22:44:59 INFO │ __construct(): 2 │
┌ 22:44:59 INFO │ __construct(): 3
If I then modify my uniqueId method ever so slightly like so (should be no less unique than previously) ...
public function uniqueId(): string
{
return (string) $this->id . '_test';
}
The jobs get dispatched and handled properly as you can see in the Horizon output:
2024-04-05 22:51:27 App\Jobs\TestJob ............................... RUNNING
2024-04-05 22:51:27 App\Jobs\TestJob .......................... 27.23ms DONE
2024-04-05 22:51:27 App\Jobs\TestJob ............................... RUNNING
2024-04-05 22:51:27 App\Jobs\TestJob ........................... 3.35ms DONE
2024-04-05 22:51:27 App\Jobs\TestJob ............................... RUNNING
2024-04-05 22:51:27 App\Jobs\TestJob ........................... 2.41ms DONE
as well as the log output
┌ 22:51:24 INFO | __construct(): 1_test
┌ 22:51:24 INFO │ __construct(): 2_test
┌ 22:51:24 INFO │ __construct(): 3_test
┌ 22:51:27 INFO │ handle(): 1_test
┌ 22:51:27 INFO │ handle(): 2_test
┌ 22:51:27 INFO │ handle(): 3_test
Any insights as to what you think might be going on here would be much appreciated, thanks in advance!!!
Please or to participate in this conversation.