Where is the best place to put queued job dispatching?

I have an application that processes data in a fairly complex way, with data coming in from a number of sources, and being updated from various triggers.

I have a bunch of jobs that should be despatched on certain triggers, such as creation of a model instance, or changing of a status in a model instance.

What I'm not sure, is the best place to put the code to handle this dispatching. It could go into the eloquent models, and be scattered around the application that way, This would make it more difficult to see where all the despatching is. Edit: thinking about it, this is absolutely the wrong place to handle events that call up other packages that the model may not be aware of. It binds these things together far too tightly.

It could go in some central place, where it monitors all events for all eloquent models, and makes decisions on what jobs need to be dispatched. This feels a little messy.

This feels to me like it is a workflow thing. Is this the kind of problem that a Laravel workflow package would handle? Are there any recommendations I could look into? I like the idea of being able to monitor all the triggers and what gets dispatched in one place, so I can see what is happening, and defining these rules through data feels like a good way to do it.

Any other tips or anything I should be aware of, or that I should explore?

Update: further down the line I will need notifications to various roles and people on certain things happening - status changes for example - so it does kind of feel like workflow.

jjudge

8 years ago

Level 9

Here is how I am handling this for now.

In my application I have a core package (my application is written almost entirely as a bunch of packages). The core package contains all the dependencies on other packages in the application, so it knows all the models it needs to observe are present.

That core package has an Observers directory with an observer class for each model I want to observe. The observer classes are all registered in the boot() method of my core package provider.

use Vendor\Core\Observers\ModelAObserver;
use Vendor\Core\Observers\ModelBObserver;

use Vendor\NonCorePackageA\ModelA;
use Vendor\NonCorePackageB\ModelB;

class CoreServiceProvider extends ServiceProvider
{
    public function boot()
    {
        ModelA::observe(ModelAObserver::class);
        ModelB::observe(ModelBObserver::class);
        // etc.
    }
}

These observers fire off jobs to the queues when they see changes they are interested in. The observer classes are all in one place, and NOT in the framework (they are in a package, which should make upgrades much easier later on), so easy to find and manage. I still wonder whether I need an "observer of the observers" to monitor each time they are triggered by an event, but I can work on that later.

Update: there turns out to be a major flaw with this approach. By using observers, it is possible for jobs to be added to the queue before the models are committed to the database. If creating a bunch of model records in a transaction, the jobs could be on the queue, running, and failing to find the models they are supposed to be running against, because it could take some time before the transaction gets its commit(). So as far as I can see, you cannot dispatch queued jobs from an observer trigger. A delay could be added to the jobs as a hacky works-most-of-the-time approach, but it is a little hacky. Possibly the model instantiation could be modified so that if the model data is not found in the database, it throws itself back onto the queue to try again in a couple of seconds (with a count so it does not try forever in the even of an original transaction that was rolled back). I've presented this problem here to see if there are any suggestions: https://stackoverflow.com/questions/48869653/adding-laravel-queued-jobs-in-a-transaction-using-an-observer

Please or to participate in this conversation.