I have a few jobs in my app which are dispatched onto a queue (we use Horizon to run our queues), and can take anywhere up to 10 minutes to complete their work. For reasons I cannot pin down, seemingly when no errors happen, I get the jobs rerunning and re-dispatching onto the queue even when one is in progress already. The only logs in my log file are saying "MaxAttemptsExceededException".
Here is my job:
<?php
namespace App\Jobs\People;
use App\Actions\People\ImportEmployeeCourseProgressFromCsvFile;
use Illuminate\Contracts\Queue\ShouldBeUnique;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
class ImportEmployeeCourseProgressJob implements ShouldBeUnique, ShouldQueue
{
use Queueable;
public $timeout = 600; // 10 minutes
public $uniqueFor = 600; // 10 minutes
/**
* Create a new job instance.
*/
public function __construct(
public string $filePath,
public ?int $userId = null
) {}
/**
* Execute the job.
*/
public function handle(ImportEmployeeCourseProgressFromCsvFile $action): void
{
$action->handle($this->filePath, $this->userId);
}
public function retryUntil()
{
return now()->addMinutes(10);
}
public function uniqueId(): string
{
return $this->filePath . '-' . $this->userId;
}
}
Here is my horizon config:
<?php
use Illuminate\Support\Str;
return [
/*
|--------------------------------------------------------------------------
| Horizon Domain
|--------------------------------------------------------------------------
|
| This is the subdomain where Horizon will be accessible from. If this
| setting is null, Horizon will reside under the same domain as the
| application. Otherwise, this value will serve as the subdomain.
|
*/
'domain' => env('HORIZON_DOMAIN'),
/*
|--------------------------------------------------------------------------
| Horizon Path
|--------------------------------------------------------------------------
|
| This is the URI path where Horizon will be accessible from. Feel free
| to change this path to anything you like. Note that the URI will not
| affect the paths of its internal API that aren't exposed to users.
|
*/
'path' => env('HORIZON_PATH', 'horizon'),
/*
|--------------------------------------------------------------------------
| Horizon Redis Connection
|--------------------------------------------------------------------------
|
| This is the name of the Redis connection where Horizon will store the
| meta information required for it to function. It includes the list
| of supervisors, failed jobs, job metrics, and other information.
|
*/
'use' => 'default',
/*
|--------------------------------------------------------------------------
| Horizon Redis Prefix
|--------------------------------------------------------------------------
|
| This prefix will be used when storing all Horizon data in Redis. You
| may modify the prefix when you are running multiple installations
| of Horizon on the same server so that they don't have problems.
|
*/
'prefix' => env(
'HORIZON_PREFIX',
Str::slug(env('APP_NAME', 'laravel'), '_') . '_horizon:'
),
/*
|--------------------------------------------------------------------------
| Horizon Route Middleware
|--------------------------------------------------------------------------
|
| These middleware will get attached onto each Horizon route, giving you
| the chance to add your own middleware to this list or change any of
| the existing middleware. Or, you can simply stick with this list.
|
*/
'middleware' => ['web'],
/*
|--------------------------------------------------------------------------
| Queue Wait Time Thresholds
|--------------------------------------------------------------------------
|
| This option allows you to configure when the LongWaitDetected event
| will be fired. Every connection / queue combination may have its
| own, unique threshold (in seconds) before this event is fired.
|
*/
'waits' => [
'redis:default' => 60,
],
/*
|--------------------------------------------------------------------------
| Job Trimming Times
|--------------------------------------------------------------------------
|
| Here you can configure for how long (in minutes) you desire Horizon to
| persist the recent and failed jobs. Typically, recent jobs are kept
| for one hour while all failed jobs are stored for an entire week.
|
*/
'trim' => [
'recent' => 60,
'pending' => 60,
'completed' => 60,
'recent_failed' => 10080,
'failed' => 10080,
'monitored' => 10080,
],
/*
|--------------------------------------------------------------------------
| Silenced Jobs
|--------------------------------------------------------------------------
|
| Silencing a job will instruct Horizon to not place the job in the list
| of completed jobs within the Horizon dashboard. This setting may be
| used to fully remove any noisy jobs from the completed jobs list.
|
*/
'silenced' => [
// App\Jobs\ExampleJob::class,
],
/*
|--------------------------------------------------------------------------
| Metrics
|--------------------------------------------------------------------------
|
| Here you can configure how many snapshots should be kept to display in
| the metrics graph. This will get used in combination with Horizon's
| `horizon:snapshot` schedule to define how long to retain metrics.
|
*/
'metrics' => [
'trim_snapshots' => [
'job' => 24,
'queue' => 24,
],
],
/*
|--------------------------------------------------------------------------
| Fast Termination
|--------------------------------------------------------------------------
|
| When this option is enabled, Horizon's "terminate" command will not
| wait on all of the workers to terminate unless the --wait option
| is provided. Fast termination can shorten deployment delay by
| allowing a new instance of Horizon to start while the last
| instance will continue to terminate each of its workers.
|
*/
'fast_termination' => false,
/*
|--------------------------------------------------------------------------
| Memory Limit (MB)
|--------------------------------------------------------------------------
|
| This value describes the maximum amount of memory the Horizon master
| supervisor may consume before it is terminated and restarted. For
| configuring these limits on your workers, see the next section.
|
*/
'memory_limit' => 256,
/*
|--------------------------------------------------------------------------
| Queue Worker Configuration
|--------------------------------------------------------------------------
|
| Here you may define the queue worker settings used by your application
| in all environments. These supervisors and settings handle all your
| queued jobs and will be provisioned by Horizon during deployment.
|
*/
'defaults' => [
'supervisor-default' => [
'connection' => 'redis',
'queue' => ['default'],
'balance' => 'auto',
'autoScalingStrategy' => 'time',
'maxProcesses' => 1,
'maxTime' => 0,
'maxJobs' => 0,
'memory' => 128,
'tries' => 1,
'timeout' => 360,
'nice' => 0,
],
],
'environments' => [
'production' => [
'supervisor-default' => [
'maxProcesses' => 10,
'balanceMaxShift' => 1,
'balanceCooldown' => 3,
'force' => true,
],
],
'staging' => [
'supervisor-default' => [
'maxProcesses' => 5,
'balanceMaxShift' => 1,
'balanceCooldown' => 3,
],
],
'local' => [
'supervisor-default' => [
'maxProcesses' => 3,
],
],
],
];
Any pointers hugely appreciated!
