Help me understand queue/jobs with my project please!

It sounds like you are looking at automating those tasks. The easiest way would be to create commands: https://laravel.com/docs/5.6/scheduling#scheduling-artisan-commands, they run similar to cron or actually are cron, just much nicer ;-)

Commands have nothing to do with Jobs and Queues. Queue would be for instance when you have an email notification and you do not want to wait until you get confirmation that email was send. You put use Queueable; on top of the notification. To use them you need to start at least one Queue worker on your server.

I also wonder if you need to do those tasks in specific order? Is it one DB? With commands you can actually do a small trick and call another server endpoint and run an url, this way you would keep everything on one server in one command. It will look something like that:

namespace App\Console\Commands;

use Illuminate\Console\Command;

class Test extends Command
{
    /**
     * The name and signature of the console command.
     *
     * @var string
     */
    protected $signature = 'my:test';

    /**
     * The console command description.
     *
     * @var string
     */
    protected $description = 'Test another endpoint with curl';

    /**
     * Create a new command instance.
     *
     * @return void
     */
    public function __construct()
    {
        parent::__construct();
    }

    /**
     * Execute the console command.
     *
     * @return mixed
     */
    public function handle()
    {
        $curl = curl_init();

        curl_setopt_array($curl, array(
        CURLOPT_URL => "https://myotherapp/test",
        CURLOPT_RETURNTRANSFER => true,
        CURLOPT_ENCODING => "",
        CURLOPT_MAXREDIRS => 10,
        CURLOPT_TIMEOUT => 30,
        CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
        CURLOPT_CUSTOMREQUEST => "GET",
        CURLOPT_HTTPHEADER => array(
            "Cache-Control: no-cache"
        ),
        ));

        $response = curl_exec($curl);
        $err = curl_error($curl);

        curl_close($curl);

        if ($err) {
        echo "cURL Error #:" . $err;
        } else {
        echo $response;
        }
            }
}

Hope it helps!

Thank you so much. I should have been clearer with my question. Right now, I do have a few artisan commands that do all of this.

• First we go out and grab all of the listings and put them in a temp table.
• Next, the "migration" chore happens. That takes each temp listings, gets the geographical coordinates and also downloads photos.
• Finally, the individual listing (home for sale) is put into a "master" table of active listings.

The problem is that often their server goes down or something just happens to fail. I thought it would be best to put these things into a job/queue. It would be really cool to be able to see where we stand (as there can be over 1,000 listings a day getting added or updated).

Ok but the question is why does it go down? Do the limit you from accessing their servers?

The only difference you will achieve with jobs is that you might queue all of the houses as a separate job and then see them go in laravel horizon. But you can do the same with scheduling your command every minute I guess.

It all depends on what type of access you have to those machines. What I understand is that you do not controller them and do not have access to their databases?

Cheers!

That's just it. Their database is just for me to read from. They do throttle requests and on a busy day, where Realtors are putting a lot of new listings in, there may be 30 come in all at once.

Ok then you would ideally create a Job. You can define how many tries a job has on a server/worker lever or you can do this directly in th Job itself:

<?php

namespace App\Jobs;

class ProcessPodcast implements ShouldQueue
{
    /**
     * The number of times the job may be attempted.
     *
     * @var int
     */
    public $tries = 5;
}

I would get all the listings, put them to a temporary table. and tick each one of them once each of Jobs get completed.

This way you will be able to see what is happening and where you stand.