@wanderingitguy Are you forking Laravel Web request handlers? If yes you are going to kill your web server by eating threads.
Packets out of order error
I'm getting the following error randomly when my method runs. Background, I have a function called "check" in my "Node" class. The method simply checks to see if the node is online or not. I have this method being called once a minute, to check nodes that need to be checked. This error, posted below, comes up randomly. What I mean by that is, the error does not occur every time the method runs. It does not occur in any sense of time either.
[2016-07-24 03:51:09] local.ERROR: ErrorException: Packets out of order. Expected 2 received 1. Packet size=12 in /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Database/Connection.php:462
Stack trace:
#0 [internal function]: Illuminate\Foundation\Bootstrap\HandleExceptions->handleError(2, 'Packets out of ...', '/var/www/vhosts...', 462, Array)
#1 /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Database/Connection.php(462): PDO->prepare('update `core_no...')
#2 /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Database/Connection.php(706): Illuminate\Database\Connection->Illuminate\Database{closure}(Object(Illuminate\Database\MySqlConnection), 'update `core_no...', Array)
#3 /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Database/Connection.php(669): Illuminate\Database\Connection->runQueryCallback('update `core_no...', Array, Object(Closure))
#4 /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Database/Connection.php(467): Illuminate\Database\Connection->run('update `core_no...', Array, Object(Closure))
#5 /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Database/Connection.php(410): Illuminate\Database\Connection->affectingStatement('update `core_no...', Array)
#6 /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Database/Query/Builder.php(2071): Illuminate\Database\Connection->update('update `core_no...', Array)
#7 /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Builder.php(531): Illuminate\Database\Query\Builder->update(Array)
#8 /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Model.php(1562): Illuminate\Database\Eloquent\Builder->update(Array)
#9 /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Model.php(1481): Illuminate\Database\Eloquent\Model->performUpdate(Object(Illuminate\Database\Eloquent\Builder), Array)
#10 /var/www/vhosts/cluster.io/clusterapp/app/Node.php(169): Illuminate\Database\Eloquent\Model->save()
#11 /var/www/vhosts/cluster.io/clusterapp/app/Console/Commands/NodeCheck.php(69): App\Node->check()
#12 [internal function]: App\Console\Commands\NodeCheck->handle()
#13 /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Container/Container.php(507): call_user_func_array(Array, Array)
#14 /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Console/Command.php(169): Illuminate\Container\Container->call(Array)
#15 /var/www/vhosts/cluster.io/clusterapp/vendor/symfony/console/Command/Command.php(256): Illuminate\Console\Command->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#16 /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Console/Command.php(155): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#17 /var/www/vhosts/cluster.io/clusterapp/vendor/symfony/console/Application.php(791): Illuminate\Console\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#18 /var/www/vhosts/cluster.io/clusterapp/vendor/symfony/console/Application.php(186): Symfony\Component\Console\Application->doRunCommand(Object(App\Console\Commands\NodeCheck), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#19 /var/www/vhosts/cluster.io/clusterapp/vendor/symfony/console/Application.php(117): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#20 /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(107): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#21 /var/www/vhosts/cluster.io/clusterapp/artisan(35): Illuminate\Foundation\Console\Kernel->handle(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#22 {main}
Next Illuminate\Database\QueryException: Packets out of order. Expected 2 received 1. Packet size=12 (SQL: update `core_nodes` set `updated_at` = 2016-07-24 03:51:07, `last_check` = 2016-07-24 03:51:02 where `id` = 7) in /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Database/Connection.php:713
Stack trace:
#0 /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Database/Connection.php(669): Illuminate\Database\Connection->runQueryCallback('update `core_no...', Array, Object(Closure))
#1 /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Database/Connection.php(467): Illuminate\Database\Connection->run('update `core_no...', Array, Object(Closure))
#2 /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Database/Connection.php(410): Illuminate\Database\Connection->affectingStatement('update `core_no...', Array)
#3 /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Database/Query/Builder.php(2071): Illuminate\Database\Connection->update('update `core_no...', Array)
#4 /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Builder.php(531): Illuminate\Database\Query\Builder->update(Array)
#5 /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Model.php(1562): Illuminate\Database\Eloquent\Builder->update(Array)
#6 /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Model.php(1481): Illuminate\Database\Eloquent\Model->performUpdate(Object(Illuminate\Database\Eloquent\Builder), Array)
#7 /var/www/vhosts/cluster.io/clusterapp/app/Node.php(169): Illuminate\Database\Eloquent\Model->save()
#8 /var/www/vhosts/cluster.io/clusterapp/app/Console/Commands/NodeCheck.php(69): App\Node->check()
#9 [internal function]: App\Console\Commands\NodeCheck->handle()
#10 /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Container/Container.php(507): call_user_func_array(Array, Array)
#11 /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Console/Command.php(169): Illuminate\Container\Container->call(Array)
#12 /var/www/vhosts/cluster.io/clusterapp/vendor/symfony/console/Command/Command.php(256): Illuminate\Console\Command->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#13 /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Console/Command.php(155): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#14 /var/www/vhosts/cluster.io/clusterapp/vendor/symfony/console/Application.php(791): Illuminate\Console\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#15 /var/www/vhosts/cluster.io/clusterapp/vendor/symfony/console/Application.php(186): Symfony\Component\Console\Application->doRunCommand(Object(App\Console\Commands\NodeCheck), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#16 /var/www/vhosts/cluster.io/clusterapp/vendor/symfony/console/Application.php(117): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#17 /var/www/vhosts/cluster.io/clusterapp/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(107): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#18 /var/www/vhosts/cluster.io/clusterapp/artisan(35): Illuminate\Foundation\Console\Kernel->handle(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#19 {main}
Here is the method in the Node Class
public function check()
{
$this->last_check = Carbon::now();
$this->save();
if (current(Salt::CLI($this->dns_name, 'test.ping')))
{
if ($this->status == 0)
{
$this->setOnline();
}
}
else
{
if ($this->status > 0)
{
$this->setOffline();
}
}
return $this;
}
And here is the task that runs each minute.
public function handle()
{
if ($this->option('force'))
{
$interval = 0;
}
else
{
$interval = Setting::get('nodes_check_interval', true);
}
$nodes = Node::where('last_check', '<=', Carbon::now()->subMinutes($interval))->get();
if (!$nodes)
{
exit();
}
foreach ($nodes as $node)
{
usleep(200000);
$fork_pid = pcntl_fork();
if ($fork_pid == -1)
{
die("Could not fork parent process");
}
if (!$fork_pid)
{
$node->check();
exit();
}
}
foreach ($nodes as $node)
{
pcntl_wait($status);
}
return true;
}
For anyone else that comes across this, setting the following attribute in "config/database.php" has caused this error to disappear. Warning: This is probably not the best way to do this.
'options' => array(
PDO::ATTR_EMULATE_PREPARES => true
),
Please or to participate in this conversation.