S3 upload failing on Job

Hi,

I have a csv export built on a job queue system which was working fine on laravel 7 but after upgrading to v8 the job throws 500 error. and does not output any error stack.

but if I run the same job with no changes by using dispatchSync it works just fine. Using Logging i have found that it gives the error when trying to upload the file to our s3 bucket. It cant be anything to do with credentials etc as it works fine on dispatchSync. Any idea what could be wrong?

$uploaded_file = new \Symfony\Component\HttpFoundation\File\File($filename);
        Log::debug("base: " . $basename);
        try{
            Log::debug("try catch");
            Log::debug(config('constants.s3_dir.task_exports'));
            Storage::disk('s3')->putFileAs(config('constants.s3_dir.task_exports'), $file, $basename . '.csv');
            Log::debug("s3: ". $s3);
        }
        catch (\Aws\S3\Exception\S3Exception $e) {
            Log::debug("s3 error: ".$se);
        }
      
        Storage::disk('public')->delete(public_path("storage/task_exports/" . $filename));

        return $s3;