Batching jobs - local vs deployed on environment
Hi everyone! I've been dealing with such a head scratching issue, and I've tried just about everything I can think of.
Here's a distilled version what I'm working with:
$jobs = collect([]);
Model::query()
->select([
...
])
->where(...)
->with(...)
->chunk(2000, function ($chunk) use ($jobs) {
if ($chunk->isNotEmpty()) {
$jobs->push(new JobForBatch(
arg1,
arg2,
));
}
});
$batch = Bus::batch($jobs->toArray())
->name(...)
->onQueue('default_long')
->then(function (Batch $batch) {
...
})->catch(function (Batch $batch, Throwable $e) {
...
})->finally(function (Batch $batch) {
...
})->dispatch();
Nothing crazy here; I'm querying a large set of data, chunking it, creating a new Job for each chunk, pushing that Job to a collection, and then ultimately, pushing those jobs to be worked on in a Batch. In my local environment, this works absolutely fine.
The issue I'm having is as soon as I deploy this to my development/staging environments and run this job, it fails. Here is the error I consistently get:
Call to a member function add() on null {"exception":"[object] (Error(code: 0): Call to a member function add() on null at /var/app/current/vendor/laravel/framework/src/Illuminate/Bus/PendingBatch.php:257)
If I drill down into that PendingBatch.php file (this is a Laravel vendor file) at the method + line it's calling out -
public function dispatch()
{
$repository = $this->container->make(BatchRepository::class);
try {
$batch = $repository->store($this);
$batch = $batch->add($this->jobs);
} catch (Throwable $e) {
if (isset($batch)) {
$repository->delete($batch->id);
}
throw $e;
}
$this->container->make(EventDispatcher::class)->dispatch(
new BatchDispatched($batch)
);
return $batch;
}
...referencing that error, it's not difficult to see where it's failing, but I have no idea what would be causing the $batch object to be null. Even more so, I have no idea why this would run locally but NOT when deployed to an non-local environment.
The thread I was pulling was the fact that we use an Aurora RDS instance on our deployed environments, but just regular old MySQL 8 for local... however, from various logging that I've introduced, as well as where this job actually bombs out, I don't believe that discrepancy is the issue. The PHP versions are the same, Redis and RDS instances are sufficiently beefy; I'm at a total loss. I've also logged & confirmed that the $jobs collection that gets built up for the batch does contain the proper amount of jobs.
Does anyone have any other threads I could try pulling here? Any guidance would be appreciated!
Note: I do not believe the underlying tasks being performed in the jobs being batched is relevant here, as it never even gets to the point where they would be worked on.
Please or to participate in this conversation.