Import CSV files and Excel using file streamed from S3

Because the queued jobs can not accept a file as a passed in parameter

You pass what you like to your job through the constructor

class ProcessCsv implements ShouldQueue
{
    protected $file;

    public function __construct($file)
    {
        $this->file = $file;
    }

    public function handle() 
    {
        // do your processing here
       $results = Excel::load($this->file);       
    }
}
...

// Dispatch your job in your controller, model, etc
ProcessCsv::dispatch($file);

that makes sense thanks

Actually I tried this and didn't have any luck I received the error

Serialization of 'Illuminate\Http\UploadedFile' is not allowed

I also tried to comment out the Serialization parts and received the same error

amespace App\Jobs;

use Illuminate\Bus\Queueable;
//use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;

class ProcessUploadedCDRFile implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable;//, SerializesModels;

    protected $file;

    /**
     * Create a new job instance.
     *
     * @return void
     */
    public function __construct($file)
    {
        $this->file = $file;
    }

@RossUK - I would pass in the filename rather than the file itself, then load it in the job. Example from your original code but you could probably just right to local storage.

    public function handle() 
    {
       $contents = Storage::disk('s3')->get($this->file);
       $results = Excel::load($contents);       
    }

That's what I was thinking. I just need to work out the best way of managing the files so they are deleted after they are processed :)

Thanks for your help

Just use use the storage facade

Storage::delete($this->file);

I just need to work out the best way of managing the files so they are deleted after they are processed

$contents = Storage::disk('s3')->get($this->file);
$results = Excel::load($contents); // file is now loaded in results, so don't need it
Storage::disk('s3')->delete($this->file);

?

Thanks almost there something isn't quite right when I try this I get the following

Could not open /Users/Ross/Projects/TestCDRProcess/Id , Customer, Telephone Number,

Those are the file headers ?

I think its because the PHPExcel_Reader is expecting a filename but I'm not sure how to sort it.

below extracted from error message

  public function canRead($pFilename)
    {
        // Check if file exists
        if (!file_exists($pFilename)) {
            throw new PHPExcel_Reader_Exception("Could not open " . $pFilename . " for reading! File does not exist.");
        }

Ah, that will be a problem. file_exists() will only work on a local filesystem, not something remote. You might have to copy the file from S3 to your localhost, then add it to excel, then delete it from both places.

Think I'm going to go with local storage as it's going to be deleted after use :)

Ill post back my solution once I have read how to store and delete off the local file system to a temp folder

Got it sorted using local storage thanks. Is it normal to have to prefix the path (in this case storage/app/) ?

here is what I ended up with

Controller Method:

public function store(Request $request)
    {

        if($request->hasFile('file')) 
        {
        $path = $request->file('file')->store('temp-files');
        ProcessUploadedCDRFile::dispatch($path);
        return back();
        } 
    }

and the job method

public function handle()
    {

       $results = Excel::load('storage/app/' . $this->path); 
       Storage::delete('storage/app/' . $this->path);

       // do something useful with results
    }

Thanks again