Queueable Job Works as "Sync" Driver but not "Database"

I have a weird issue with a Queueable Job. I started off creating it with the queue driver of "Sync" and it works fine, but when I switch it to "database", it appears that the calls involving "Spatie\Ssh\Ssh" stop functioning correctly, but works fine when I move back to "sync". Suggestions? I feel like I'm missing something simple. Spatie SSH works fine everywhere else in the project.

Another thread on Laracasts forums say type-hinting, but that post was 7 years old, and im not sure if relevant here.

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldBeUnique;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;

use Throwable;

use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Log;

use Spatie\Ssh\Ssh;

use App\Models\JobDetail;

class ProcessPackBuildJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    protected array $data;

    /**
     * Create a new job instance.
     */
    public function __construct(array $data)
    {
        $this->data = $data;
    }

    /**
     * Execute the job.
     */
    public function handle(): void
    {
        try {
        $data = $this->data;
        Log::info('Job ' . $data['jobId'] .' Ran!', $data);

        JobDetail::where('id', $data['jobId'])->update([
            'status' => 'building',
            'started_at' => now(), // Set the current timestamp.
            //'completed_at' => now(),
        ]);

        $packsToBuild = explode(",", $data['packs']);
        $packId = $packsToBuild[0];

        $parentPack = DB::connection('mysql_products')
            ->table('products')
            ->where('id', $packId)
            ->first();

        $packItems = DB::connection('mysql_products')
            ->table('set_products')
            ->join('products', 'set_products.product_id', '=', 'products.id')
            ->select('products.name', 'products.design_id', 'products.missing_formats')
            ->where('set_id', $packId)
            ->get();

        Log::info('Pack Items', $packItems->toArray());

        Log::info(env('SECURE_CDN_USER'));

        $secureCdnUser = env('SECURE_CDN_USER');
        $secureCdnHost = env('SECURE_CDN_HOST');
        $secureCdnKey = env('SECURE_CDN_PRIVATEKEY');
        $secureCdnPort = env('SECURE_CDN_PORT');

        Log::info($secureCdnUser . $secureCdnHost . $secureCdnKey . $secureCdnPort);

        $ssh = Ssh::create($secureCdnUser, $secureCdnHost)
                ->disablePasswordAuthentication()
                ->usePrivateKey($secureCdnKey)
                ->disableStrictHostKeyChecking()
                ->setTimeout(100)
                ->usePort($secureCdnPort);

        //Testing if ssh commands are even executing, which they arent.
        $derp = $ssh->execute('ls')->getOutput();
        log::debug($derp);

        //Other similar commands involving Spatie SSH, etc etc

        //etc etc
} catch (Throwable $e) {
            // Handle exceptions and errors
            Log::error('Error processing job: ' . $e->getMessage());

            // Update job status to indicate failure
            JobDetail::where('id', $data['jobId'])->update([
                'status' => 'failed',
                'stage' => 'error',
                'completed_at' => now(),
            ]);

            Log::error('Job failed.');
        }

    }
}

LaryAI

2 years ago

Level 58

When you switch from the "sync" queue driver to the "database" driver, Laravel serializes the job to store it in the database. This serialization process can cause issues with certain types of objects, especially those that represent resources or have open connections, such as an SSH session.

In your ProcessPackBuildJob class, you have a public $ssh property that is not initialized in the constructor. This property is likely causing the serialization issue because it's an instance of Ssh which cannot be serialized properly.

To resolve this issue, you should create the Ssh instance within the handle method rather than storing it as a property of the job class. This ensures that the Ssh instance is fresh and not subject to serialization when the job is dispatched to the queue.

Here's how you can modify your handle method:

public function handle(): void
{
    try {
        // ... existing code ...

        // Initialize the SSH connection within the handle method
        $ssh = Ssh::create($secureCdnUser, $secureCdnHost)
                ->disablePasswordAuthentication()
                ->usePrivateKey($secureCdnKey)
                ->disableStrictHostKeyChecking()
                ->setTimeout(100)
                ->usePort($secureCdnPort);

        // ... rest of your code ...

    } catch (Throwable $e) {
        // ... existing error handling code ...
    }
}

By making this change, you ensure that the Ssh instance is not serialized along with the job, which should resolve the issue you're experiencing when using the "database" queue driver. Remember to remove the public $ssh property from your class if it's not used elsewhere.

Additionally, make sure that your environment variables are correctly set for the queue worker process. Sometimes, environment variables can be different when running CLI commands (like queue workers) compared to web server processes. If necessary, use a configuration file or pass the SSH configuration directly to the job constructor to ensure the correct values are used.

DrRaccoon

2 years ago

Level 2

Oh whoops, the public $ssh; was leftover from an untested attempt.

DrRaccoon

2 years ago

Best Answer

Level 2

I solved the problem. Apparently when running the sync driver and database driver, its a user permissions issue to the privatekey. To trouble shoot I logged is_readable(env('SECURE_CDN_PRIVATEKEY')) which returned true on sync driver, but false on database driver. So I just did a chmod on the private key directory i made (sudo chmod -R a+rX /etc/internal-tools/keys/) and its working now.

My perspective on this is when its running as sync its using the permissions from www-data user, but when its database, its running as whatever permissions of the user that ran php artisan queue:work --queue=packBuild.

2 likes

Please or to participate in this conversation.