Run cron job tasks simultaneously

@c2373525 You can do that by running the cron jobs in the background. In *nix systems you do that by adding an & at the end of a command. Not sure if you can tell Laravel to add that.

@renedekat According to this answer, cron jobs are automatically run in the background:

http://stackoverflow.com/a/10503463/6347105

But it appears that Laravel runs tasks sequentially. I'm looking for a way to run the tasks in parallel rather than sequentially.

@c2373525 Like I said, try to add the & to the command:

$schedule->command('post_getter 0 5 &')->cron('*/15 * * * *'); That will force the command to return immediately so Laravel will continue with the next one.

@renedekat That didn't work. It still ran the tasks sequentially. Instead I added custom cron jobs for each task in crontab -e.

@c2373525 Just out of curiousity: Why is so important to run tasks simultaneously? That must be one piece of sophisticated code if you can sync processes.

I would also like to do this. Has anyone figured out how to do this? Is it impossible?

Here is how I did it, for this project we had multiple cron jobs and was easier for me to create a controller which contains every single cron job (method that will run from the crons controller), instead of modifying /app/Console/Kernel.php any time there was a new cron job I added a table which holds the crons, here is a sample of a cron job and the table structure:

CREATE TABLE `crons` (
  `id` int(11) NOT NULL,
  `pid` int(11) DEFAULT NULL,
  `server` int(11) DEFAULT NULL,
  `description` varchar(255) NOT NULL,
  `intervals` varchar(20) NOT NULL,
  `method` varchar(255) NOT NULL,
  `status` tinyint(4) NOT NULL DEFAULT '0',
  `running_status` tinyint(1) NOT NULL DEFAULT '0',
  `log` mediumtext NOT NULL,
  `last_start_run_time` int(11) NOT NULL DEFAULT '0',
  `last_end_run_time` int(11) NOT NULL DEFAULT '0',
  `updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  `last_runtime` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
  `created_at` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00'
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Then a sample record

INSERT INTO `crons` (`id`, `pid`, `server`, `description`, `intervals`, `method`, `status`, `running_status`, `log`, `last_start_run_time`, `last_end_run_time`, `updated_at`, `last_runtime`, `created_at`) VALUES
(1, NULL, 1, 'Do some fun in the background', '*/1 * * * * *', 'someAwesomeMethod', 1, 0, '', 1496276821, 1496276822, '2017-06-01 07:27:02', '2017-06-01 07:27:02', '2017-03-03 02:58:27');

This way I can manage through an interface the cron job and I can change the intervals of the cron easily;

Then all you do in /app/Console/Kernel.php@schedule is add ther following code

/**
     * Define the application's command schedule.
     *
     * @param  \Illuminate\Console\Scheduling\Schedule  $schedule
     * @return void
     */
    protected function schedule(Schedule $schedule)
    {
        if($_SERVER['argc'] === 3){
            /*This is a schedule cron job, let's validate it*/
            $skip = $_SERVER['argv'][2];
            if(strpos($skip, 'skip=') !== false){
                $skip = (int)str_replace('skip=', '', $skip);
                $cron = Crons
                    ::where('status', 1)/*status=1 in my table means enabled, this allows to easily disable crons through the interface*/
                    ->skip($skip)
                    ->first();
                if(is_object($cron)){
                    $schedule
                    ->call(
                        function() use ($cron){
                            app('App\Http\Controllers\MyVeryOwnCronController')->crontab($cron);
                        }
                    )
                    ->cron($cron->intervals);
                }
            }
            
        }
}

So that takes care of the crons in the DB now to run them simultaneously all you do is add multiple cron jobs all running every minute to the cron tab, so instead of adding this

* * * * * php /path/to/artisan schedule:run >> /dev/null 2>&1

you add this

* * * * * php /path/to/artisan schedule:run skip=0 >> /dev/null 2>&1
* * * * * php /path/to/artisan schedule:run skip=1 >> /dev/null 2>&1
* * * * * php /path/to/artisan schedule:run skip=2 >> /dev/null 2>&1
* * * * * php /path/to/artisan schedule:run skip=3 >> /dev/null 2>&1

And add a line for each cron job that is suppose to run in parralel increasing the count on the skip variable.

There is one last thing to do which not many will like but the artisan command will not run if the file /vendor/symfony/console/Input/ArgvInput.php is not changed, the following method needs to be changed so it excludes the parameter skip=0 from the exceptions;

    /**
     * Parses an argument.
     *
     * @param string $token The current token
     *
     * @throws RuntimeException When too many arguments are given
     */
    private function parseArgument($token)
    {
        if(preg_replace("/(skip=)(\d+)/", "", $token) === ''){
            return;
        }
        $c = count($this->arguments);

        // if input is expecting another argument, add it
        if ($this->definition->hasArgument($c)) {
            $arg = $this->definition->getArgument($c);
            $this->arguments[$arg->getName()] = $arg->isArray() ? array($token) : $token;

        // if last argument isArray(), append token to last argument
        } elseif ($this->definition->hasArgument($c - 1) && $this->definition->getArgument($c - 1)->isArray()) {
            $arg = $this->definition->getArgument($c - 1);
            $this->arguments[$arg->getName()][] = $token;

        // unexpected argument
        } else {
            $all = $this->definition->getArguments();
            if (count($all)) {
                throw new RuntimeException(sprintf('Too many arguments, expected arguments "%s".', implode('" "', array_keys($all))));
            }

            throw new RuntimeException(sprintf('No arguments expected, got "%s".', $token));
        }
    }

@tlacaelelrl how is the performance on this? I have a table similar where a cron syntax is one tab and was planning on running a loop on the table and throwing all the records into a queue if it's their time. My only is performance since right now there's 10-20 records but when it grows to thousands that could be an issue. Especially for the records where it all wants to be ran at the same time like '0 * * * *'

For me it is good enough although it is definitely faster to hardcode all the cron jobs but then they do not run symultaneously.

For me it was the best option since:

1. I can now track how long a job runs adding the last run time timestamp and also saving when it finished.
2. The jobs do not overlap (Although I think there is a way to do such thing directly with laravel) I can easily enable, disable and add more jobs through the interface without having to modify the kernel file directly.

The only extra thing that this does is that it needs to run an extra query to the DB to get the next task it needs to run so I don't think this will have any adverse perfomance and in my setup it has proven to run smoothly.

@renedekat Some jobs take longer than others, and I don't want to have to worry about moving faster jobs higher than slower ones in the schedule method. Also, I would like schedule:run to work exactly like the crontab.

Of course, I might be wrong in assuming that cron runs jobs sequentially. Am I?

Starting from version 5.7 you can use Background Tasks with ->runInBackground() (read more here).

But the runInBackground method may only be used when scheduling tasks via the command and exec methods. I hope this will help someone.