Running Laravel Queues at specific times / timeframe

You can try to do this:

$schedule->job(new YourJobHere)->daily()->between('4:00', '18:00');

This way your jobs will queue only between these hours.

Thanks @borisu I think that's pretty close.

Any thoughts how I would use that in conjunction with processing an existing queue? The documentation I read supported adding to a queue, but I didn't see anything about processing a queue.

Well you can run a queue daemon: https://laravel.com/docs/5.5/queues#running-the-queue-worker

It's always on and processes jobs. When there are none it will do nothing.

Thanks @Borisu

This might work. Related question..

If I have 2 queues now processing, is it possible to set a priority of the queues so when Queue1 is running, Queue2 pauses. Once Queue1 is done, Queue2 resumes processing. Almost like a dependency relationship.

i dont think that is possible.. queues run parallel to each other..

1. but in the new laravel (5.5 or 5.6?) you can have a task go through queues in order.. if that suits you..

2. or you can have a wait time (in each queue, but this isnt a real solution)

why do you need two queues to 'know' about each other? if you can tell me then we can let you know a better solution

Thanks @shez1983

I'm running 5.5

Right now I'm using a cronjob to make API calls every 20 seconds, because that's the typical length of time it takes to execute the call. It could take 2 seconds, or 4 seconds, or 20 seconds.

My thinking is that instead of wasting the time difference of completion to 20 seconds, I could use a queue so as soon as one API call is done, it starts the next one.

The reason I can't have these queues running in parallel is because I can only make 1 API call per second and I need to use the same API for something else, for a couple hours per day. So I can't have it running both at the same time.

then you make the first api call.. and after that finishes you send it to another queue... which will work on it if there is something in this second queue.. (and have it 'rest' for 20 seconds if no queue, forge allows you to do that not sure how you do that yourself)

secondly.. in both your queus you can have a rest period of say 10sec or whatever avoids the api limits per hour... (if you were using my api for something which you need a licence for i wouldnt be happy btw ;) )

My goal is to have it execute as many API calls as possible, which is why I want to avoid using a cronjob because it wastes valuable time in between calls.

Can you clarify your first point? I'm not following.

This particular API is for Best Buy, so they haven't shown any care that I call it as frequently as they allow for it to be.

 I call it as frequently as they allow for it to be.

contradicts your

 parallel is because I can only make 1 API call per second and 

either way my first point is that you do something like this

My thinking is that instead of wasting the time difference of completion to 20 seconds, I could use a queue so as soon as one API call is done, it starts the next one

so you have essentially two queues.. the first one does something.. taking 10/20/40 sec and sends it to second queue.. now the second queue then works on it once it gets it...

there is no other way.. unless you make them both a single queue.. ie do it all within one queue... but then you wont be able to do it as fast as possible... your 2nd q is ever going 2 be as fast as your 1st queue..

As I think about it, maybe something to do with "priority"

Might this work with a "high" priority single queue?

php artisan queue:work --queue=high,default

https://laravel.com/docs/5.5/queues

To further clarify my earlier statements, I want to call the API as frequently as possible. Most jobs will take under 1 second to complete, that's the queue1 that needs to run. I have a second "queue2" where the jobs may take 4 seconds or 20 seconds to complete.

I'm more than likley over complicating it because I'm still a beginner developer. I also likely don't yet know how to best explain what I'm trying to do, but giving it my best shot.

@jonathan1 Yes you can set priority on queues, it won't pause them but will prefer to do jobs with high priority (and you don't need two queues for this by the way).

If you are sending the jobs from a central location in your application you can throttle calls to the jobs by using a timeout.

Which API do you want to call and why all the limitations?

@Borisu that might work!

Is there a way for me to run a command from the job and have the command tell the job when the command is completed, so the job can wait for the command to be complete?

Yes of course, this is how code works :D So your command/job starts executing code, when you make the API call you can just wait one second and then continue with your code:

$this->someFunction();
sleep(1);
$this->someOtherFunction();

@borisu well that would defeat the purpose of using the queue, I can do that in a cronjob. But that's what I want to avoid, because the job could take a variable length of time, anywhere from 1 to 20 seconds.

@jonathan1 I think we're having some miscommunication. The queue will execute jobs/functions or whatever in a specific order, no need to communicate between jobs and queues to say "Hey I'm finished", that just happens automatically. So let's just image your queue ran and the statistic would look like this:

Job1 ran in 2s
Job2 ran in 1s
Job3 ran in 10s
Job4 ran in 5s
Job5 ran in 1s
....

You don't need the sleep function. You only need it if you want to throttle some API call in the job itself. For example in Job1 you make 3 API calls and you're allowed to make only one call per second. To make sure each call is at least one second apart you can use sleep. I hope this clears up the issue :)

@Borisu ahhh great thanks! I will give that a shot and see how it goes. Cheers!

Glad to help :)

So actually, the solution will be to create a second queue, "high" and then specify in my supervisord daemon, queue:work --queue=high,low and it will process high first.

Cheers!

@Borisu

so I have the jobs setup and it's processing, however, it's running every 1 second, set as the minimum by ->throttle()

What else do I need to do, to ensure that each job waits for the previous one to complete, where it could take 1 second or 20 seconds to complete.

my job processor:

    public function handle()
    {
        Redis::throttle('bestbuy_medium_matches')->allow(1)->every(1)->then(function () {
            Artisan::call('bestbuy:search', ['search' => $this->products->pluck('sku'), '--type' => 'match']);
        }, function () {
            return $this->release(1);
        });
    }

results:

[2018-02-02 14:03:02] Processing: App\Jobs\BestbuyFindMatch


Starting Search...
Title: Samsung 860 EVO 250GB 2.5 Inch SATA III Internal SSD (MZ-76E250B/AM)
Product ID: 6
SKU: B07864WMK8

[2018-02-02 14:03:02] Processing: App\Jobs\BestbuyFindMatch


Starting Search...
Title: Samsung 860 EVO 2TB M.2 SATA Internal SSD (MZ-N6E2T0BW)

Product ID: 7
SKU: B078218TWQ

[2018-02-02 14:03:03] Processing: App\Jobs\BestbuyFindMatch


Starting Search...
Title: Samsung 850 PRO - 512GB - 2.5-Inch SATA III Internal SSD (MZ-7KE512BW)
Product ID: 8
SKU: B00LF10KTO

[2018-02-02 14:03:04] Processing: App\Jobs\BestbuyFindMatch


Starting Search...
Title: Samsung 860 EVO 500GB mSATA Internal SSD (MZ-M6E500BW)
Product ID: 9
SKU: B07822P8JY

[2018-02-02 14:03:05] Processing: App\Jobs\BestbuyFindMatch