Level 22
can you ignore the job when model are missing with public $deleteWhenMissingModels = true;? don't know other option.
Oct 13, 2022
10
Level 1
Laravel Queue Job - No query results for model (After Commit = true)
Hey community,
recently, sometimes jobs (SerializedModels) failed with the Exception:
Illuminate\Database\Eloquent\ModelNotFoundException: No query results for model [..]
Controller:
$model = new Model();
$model->save();
// Log::warning("Before dispatching", $model->toArray()); // Model displayed correctly, (id set)
Job::dispatch($model); // Connection: Redis - Queue: redis
// Log::warning("After dispatching", $model->toArray()); // Model displayed correctly, (id set)
Job:
class Job implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
private Model $model;
public function __construct(Model $model)
{
$this->model = $model;
}
public function handle()
{
// Exception thrown before code running here
}
Setup:
- Queue is running by horizon with after_commit =true;
- Galera Cluster (Three Nodes: For Testing: Controller + Queue uses the same node)
- Queue Driver / Connection: REDIS
- Multiple App Servers
- Dedicated Queue Server
- Dedicated Redis Server
- In this case, there is no transaction or dispatched job inside the transaction
- The model was successfully saved before dispatching the job
if job retries are enabled, (2) the job ran successfully after a second attempt (Race Condition confirmed)
For sure, an option could be to remove the Serialized Model Trait, but this could not be the solution.
Any ideas why the queue job worker can't fetch the model, but a second attempt works as expected ? (Model Save runs before, After Commit enabled)
PHP Artisan About:
php artisan about
Environment ................................
Application Name ................... laravel
Laravel Version ..................... 9.35.1
PHP Version ......................... 8.1.11
Composer Version .................... 2.1.12
Environment ..................... production
Debug Mode ............................. OFF
Maintenance Mode ....................... OFF
Cache ....
Config .............................. CACHED
Events .............................. CACHED
Routes .............................. CACHED
Views ........................... NOT CACHED
Drivers ..
Broadcasting ........................ pusher
Cache ................................ redis
Database ............................. mysql
Logs .................. stack / daily, slack
Mail .................................. smtp
Queue ................................ redis
Session .............................. redis
Level 1
Hey, thanks for your reply.
For sure, it could be ignored, but then the job is discarded immediately instead of a second attempt.
I really want to fix the root cause of this issue.
Level 22
@dash222 I also want to know the cause :). Really no idea.
Level 102
@dash222 Are you sure that the model isn't simply deleted since the job was dispatched? That is normally the problem. Personally I dont send models to jobs for this cause. I send the needed data from the model instead.
Level 1
@Sinnbeck Yes, im am :)
Level 1
Problem solved.
This is not caused due to laravel.
The error is generated by MariaDB (Galera Cluster)
If model is for example written on node A , the queue worker selects the row from node B but the Write was not reflected on node B yet.
Easy Solution:
Enable wsrep_sync_wait to 1 (READ wait until node is synced)
Source: https://mariadb.com/kb/en/galera-cluster-system-variables/#wsrep_sync_wait
With this setting, the Queue Worker waits until its node is synced before trying to read the recently created model.
Thanks everyone :)
1 like
Level 3
I get these errors with queuable mailables too, but my server is standard Forge MySQL (not mariadb).
I don't think it's a problem with db transactions and persistence, because this is how my code works:
$items = Item::where('updated_at', '<=', Carbon::now()->subMinutes(15)->toDateTimeString())->get();
foreach($items as $item) {
Mail::to($item->email)->send(new TestNotification($item));
No query results for model [App\Models\Item]
It's been 15 minutes since the record was created and it obviously exists in the database. Weird.
I'll come back if I find anything...
Level 2
@stuartcusack Did you find anything regarding this. We faces the same issue and we are sure record exist in database. And strange thing is that not every job/ listener is failed. Only some jobs/listeners in the default queue gets failed and the reason for that is ' Model not found'
Level 2
We are facing similar problems.
Our queued listeners (triggered by an event) randomly start failing, when it starts to fail, we have a lot of these failed queued listeners.
Unhandled exception within Laravel Queue job: 'Illuminate\Database\Eloquent\ModelNotFoundException'
Notice the message: "Unhandled exception within Laravel Queue job". I haven't see internet posts talking about that message. When they talk about the ModelNotFoundException they talk about the message "No query results for model [App\SomeModel]'". Also notice that the exception is 'unhandled', which could explain why the failed job doesn't reach the failed_jobs table. (we keep failed_jobs in MariaDB)
In the worker.log we see that the jobs failed. But they also don't appear in the failed_jobs table. Which is odd, since we didn't set the deleteWhenMissingModels flag. If I read the documentation, the default behaviour should be that jobs with missing models should be in the failed jobs table. But although we are still researching, it looks like they are not reaching the failed jobs table or are automatically being purged/deleted. But like I said the unhandled exception could be a cause of that. In what kind of circumstance would Laravel not be able to catch/handle the exception thrown by or within a job?
But the main thing is; why are queued events failing? Normally they work fine, but when this issue is happening, 1% of the listeners start to fail. It's not 1 particular listener or 1 particular event. We have set the after_commit flag (https://laravel.com/docs/10.x/queues#jobs-and-database-transactions) which should guarantee that a listener is executed when there aren't any open transactions.
Our config looks like this:
'redis' => [
'driver' => 'redis',
'connection' => 'queue',
'queue' => 'default',
'retry_after' => 9000,
'after_commit' => true
],
We do have multiple queue's. Not sure if that has to be specified, but then it's still strange that the jobs do land in Redis, but don't honor the after_commit flag.
Would appreciate any help of this topic.
Level 1
I have the same issue. In my case I'm using a MariaDB cluster with 2 Read Replicas in AWS RDS. The error appeared when I created the second replica and I configured in .env file as DB Read node.
Also I experimented another error. When a model's object is updated and is saved in some cases I dispatch a Job to perform some operations that depends of the a attribute value of the model. I'm noticed that this job is not dispatched because the attributed is not updated in the model's object, but in DB is updated.
Please or to participate in this conversation.