Be part of JetBrains PHPverse 2026 on June 9 – a free online event bringing PHP devs worldwide together.

globiweb's avatar

Looking for elegant PHP process runner

I've been watching Laracasts a lot and envy Jeffrey's clean and elegant code. I find though, that in the real world, code always ends up messy.

I can get front-end code looking pretty nice - that's straight forward and easy. But it's the back-end stuff that is causing headaches.

I have numerous apps that do many different things asynchronously, and many are extremely complex working with 3rd-party API's.

For example:

  • get email from imap
  • parse email
  • get data from API
  • massage data
  • push data to API and get return
  • more massaging
  • push to another API
  • update database etc

The issue is that any one of these steps can fail for any number of reasons, some of which we want to retry for, and some we don't. Eg: network error = retry, Rate limit error = retry later, 404 error = fail entire process, missing data error = try plan b, etc.

So all these beautiful examples of pushing a job to a queue and having a worker deal with it don't work here.

Firstly, I need many workers. The above example is just one "job" and has 8 steps, so realistically I'm looking at 40-80 different worker types per app.

Also, just having regular logs is useless, especially when there's a fatal error (like memory exhaustion). Logs should be per master process so that everything is grouped together (in the above example, per incoming email). In a production system where the logs just fill up with info warn and error lines from non-related processes, the logs are useless.

Also, the individual steps become so tightly coupled because they all need data from previous steps. Not only are they too tightly coupled, but the data they gather becomes unwieldly and huge, causing it's own issues.

Then there's the whole issue of everything being asynchronous and no visibility. It would be nice to be able to review what happened on a per-process basis what's going on (not just per simple job).

Has anyone come across a nice library or at least design pattern to deal with very complex tasks asynchronously in an elegant way?

I've been searching github for hours to no avail.

0 likes
4 replies
Drfraker's avatar

I wonder if leveling up your OOP skills could help you sort out the issues you're having. This was a great course: https://laracasts.com/series/object-oriented-bootcamp-in-php. I remember when I was feeling overwhelmed like this, it was due to lack of understanding of how to make objects that worked and write tests.

I would highly recommend watching the OOP series, getting Adam Wathan's course "Test Driven Laravel" and writing tests for your code (if you aren't already).

Those 3 things will help you clean up the mess. Warning: It may not be quick! :)

Good luck.

1 like
globiweb's avatar

Thanks for the reply.

My OOP skills are pretty solid (12+ years PHP OOP, 20+ years PHP, 35+ years coding in general).

My issue is that these apps are insanely complex. The examples in laracasts are simple and pretty, but that doesn't translate into the real world.

BTW: I do not use Laravel and due to app size and complexity, changing frameworks is a non-starter (>1MM LOC).

I understand all the traditional OOP structures, but feel like I'm missing a design pattern to handle complexity asynchronously in general.

Even something like sending an email (which sounds simple), is not simple in the least in my setup. For example, I have to choose the sending server based on a list of criteria, adjust sending envelope based on another set of criteria, inject custom headers based on more criteria, etc. And then I need the results of an API call to inject further headers before sending. So (1) it's not simple, (2) there are multiple possible failure points even in this "simple" task, and (3) there is tight coupling EVERYWHERE.

Snapey's avatar

Have you considered state machines for these tasks?

artcore's avatar

I'm surely not on your level but what you describe with a per process setup is what I deal with mostly. Having many cogs async or not and depending on eachother or not. I find lucid architecture very helpful.

Controller or command serves a Feature, and this has a couple of Laravel jobs which can be queued or not. Jobs that often are used together can be combined in an Operation, also from the Feature class

https://github.com/lucid-architecture/laravel

Please or to participate in this conversation.