Damian89's avatar

Incomplete objects in large system

Hey, so I am using a Laravel based system of the following config:

  • Main server: Frontend, MySQL DB, Redis for Cache/Queue
  • 40 smaller servers as workers to handle jobs (we have 30-50 Million jobs per day)

Each server basically has their own laravel app installed, just .env is different - a very common setup I guess. Hardware is always the same, Software also.

Deployment happens using currently using Jenkins.

So far everything is solid but I have a specific problem:

From time to time I get the following bugsnaq notifications (just three selected, be have hundret different ones):

During class fetch: Uncaught Error: The script tried to call a method on an incomplete object. Please ensure that the...
Class "Doctrine\Inflector\Rules\English\InflectorFactory" not found
Class "Illuminate\Database\Query\Grammars\MySqlGrammar" not found

Obviously the first though is: One of those apps is not "complete", maybe some deployment failed or whatever.

BUT: This error happens basically on random servers from time to time. Also we checked integrity of each app aka all files INCLUDING vendor folder against the original. All applications are intact and those classes are not missing. Also composer dump autoload is running correctly. Running code via e.g. CLI works flawlesly (also the same code which producded those generic errors).

So my current conclusion is: It has to do something with queue using redis.

Do you have maybe some kind of input? any ideas? Specific things to test? Settings of redis to test?

Help appreciated! Thank you all!

Damian

0 likes
2 replies
Damian89's avatar

Can I ask what you are working on specifically?

Please or to participate in this conversation.