Level 102
Before I dig into the issue, I am curious why you start it from a controller if it needs to be run every 10 minutes? https://laravel.com/docs/9.x/scheduling#main-content
Jul 24, 2022
22
Level 1
Question about queues
Hello, I'm trying to create a queue that pings a list of ip addresses stored in my DB (I need to do it every 10 minutes, so i'll use a cron).
Here, I dispatch the job in my controller for each IP addresses :
public function ping()
{
$pings = Ping::all();
foreach($pings as $ping) {
dispatch(new PingJob($ping));
}
}
And here is the job (I do it using curl) :
class PingJob implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
protected $ping;
/**
* Create a new job instance.
*
* @return void
*/
public function __construct(Ping $ping)
{
$this->ping = $ping;
}
/**
* Execute the job.
*
* @return void
*/
public function handle()
{
$ping = $this->ping;
$url= $ping->ip_address;
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_TIMEOUT, 5);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 5);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$data = curl_exec($ch);
$health = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
if ($health) {
// responds ...
} else {
// doesn't respond ...
}
}
}
The issue is that I put a 5 seconds timeout to check if there is a response. That means that if an IP is not responding, the server will wait 5 seconds before pinging the next one and that's a big problem if I have like 1000 IP addresses that are not responding, the queue will take at least 5000 seconds to execute completely. So, is there any way to run all the pings simultaneously so that it doesn't wait 5 seconds to execute the next job ?
Thanks for your help
Level 1
@Sinnbeck Yeah sorry, I didn't explain, I'm just doing that to test the queue before actually creating the cron. I'm also using the database driver
Level 102
@KLassiux how many queues are running in your test?
Level 1
@Sinnbeck I'm running only 1 queue in my tests, and I have 100 IP records in the db (so 100 jobs)
Level 102
@KLassiux can you try starting a few. I noticed @oussamamater suggested this as well further down
Level 102
Are you perhaps using the sync driver instead of a proper queue?
Level 37
I think you're looking for this
- https://laravel.com/docs/9.x/queues#running-multiple-queue-workers
- https://laravel.com/docs/9.x/queues#supervisor-configuration
You can run multiple workers simultaneously (you may be interested in the "numprocs" ).
1 like
Level 1
@OussamaMater Hey thanks for your answer I'll take a look at that. What's the max value for numprocs ? If i have 1000 IP addresses, can I put numprocs at 1000 ? (I don't know anything about that)
Level 37
@KLassiux it will depend on the resources of your server, the numprocs indicates the processes number you want to run in the background, so giving a high number of processes will overload your server and take a lot of CPU, but it is doable since the processes won't really do much of a job, again it will depend on your resources.
Level 1
Here's what I finally did :
The controller :
public function ping(Request $request)
{
$ip = $request->input("ip");
$ping = new Ping;
$ping->ip_address = $ip;
$ping->save();
dispatch(new PingJob($ping));
}
The Job :
class PingJob implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
protected $ping;
/**
* Create a new job instance.
*
* @return void
*/
public function __construct(Ping $ping)
{
$this->ping = $ping;
}
/**
* Execute the job.
*
* @return void
*/
public function handle()
{
$ping = $this->ping;
$ip = $ping->ip_address;
$url = $ip;
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_TIMEOUT, 5);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 5);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$data = curl_exec($ch);
$health = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
if ($health) {
echo "\nRESPONSE FROM ".$ping->ip_address;
} else {
echo "\nNO RESPONSE FROM ".$ping->ip_address;
}
PingJob::dispatch($ping)->delay(now()->addMinutes(10));
}
}
I'm not sure if dispatching the same job at the end of another one is a good thing to do though...
The problem with using a cron is that it will execute all the jobs at the same time (every 10 minutes) and that's not what I really wanted to do.
I tested with a higher value of numprocs and it seems to be working well.
Level 37
@klassiux that's great, so if my solution helped you maybe mark this discussion as solved by setting a "best answer" for new comers who may encounter the same/similar problem.
1 like
Level 1
@OussamaMater Yes, I will, thanks for your help. I just have a last question, is it a good thing to dispatch a job at the end of another one ?
Level 37
@KLassiux I don't see a problem yes, it's very common I guess, I had done something similar in a marketing platform.
1 like
Level 37
@klassiux wht could be refactored though is using Guzzle instead of cURL, I think it more readable (even thought it uses curl under the hood).
Laravel have good docs about this have a look here:
Level 1
@OussamaMater K i'll take a look at that, thanks !
Level 102
@KLassiux I'm curious why you call it ping? It does not look like icmp? Just looks like a regular Http call
Level 1
@Sinnbeck yeah I don't really know to be honest... My boss asked me to develop a tool that would ping a specific ip address every 10 minutes and check if there is any response using Laravel. And all I found on google was people using curl.
Level 102
@KLassiux ok. Just be aware that it most likely will fail if there isn't a webserver in the other end
Example of icmp ping :)
exec("ping -c 1 " . $ping->ip_address, $output, $result);
dump($output);
if ($result == 0) {
echo "\nRESPONSE FROM ".$ping->ip_address;
} else {
echo "\nNO RESPONSE FROM ".$ping->ip_address;
}
Level 1
@Sinnbeck Okay, so I should use exec instead of curl then. Thanks for your help !
Level 102
@KLassiux you should at least compare :) and perhaps double check if he wants a web reply (curl) or icmp (ping)
Level 1
@Sinnbeck Okay, I'll check that with him but I just tried with the code you provided and it seems to be working perfectly, thanks a lot :)
Level 102
@KLassiux happy to help
Please or to participate in this conversation.