Lars-Janssen's avatar

Should I queue file uploads?

Hi,

Currently I'm trying to push files to a queue that are being uploaded. But is it normal to queue up files? Because I can only pass a model to the job.

Documentation: https://laravel.com/docs/5.3/queues

Because of the SerializesModels trait that the job is using, Eloquent models will be gracefully serialized and unserialized when the job is processing. If your queued job accepts an Eloquent model in its constructor, only the identifier for the model will be serialized onto the queue. 

So I can only pass models to a job. So how would it even be possible to queue uploaded files!?!

0 likes
10 replies
davorminchorov's avatar

Queue up the process of file uploads.

Example:

$s3 = Storage::disk('s3');

$s3->put('videos'. file_get_contents($videoFileFromInput), 'private');

this is a short example which should be in the Job's handle method.

Lars-Janssen's avatar

@Ruffles Great but how do I pass a file or files to a job. Because it looks like I can only pass a model?

davorminchorov's avatar

Edit: Why not just get the file from disk? When you upload a file, it is stored at a temporary path. There's a method $request->file('video')->getRealPath() or something similar, which returns the temporary path where the file is stored.

Lars-Janssen's avatar

@Ruffles cool, Already using S3 from Amazon. But when I pass files through the constructor job. I get the error:

Serialization of 'Illuminate\Http\UploadedFile' is not allowed
georaldc's avatar

I would imagine it being a needed step if you need to do further application-specific processing on a file upload before it gets sent to cloud storage (thumbnails with image uploads, or additional image filters maybe)

syropian's avatar

I just did something like this, a user uploads an image, I do some resizing on the server, save the image locally and use the local path when saving the model. I then fire off a job to the queue, and pass the model in. The job takes the path value from the model, and uploads the file at that path to the cloud and updates the model with the new cloud-based path. It's a pretty nice solution so far.

elena's avatar

@syropian that sounds very reasonable, but let's mention that it won't work for multi-server environment behind a load balancer, in case someone has that setup. In those cases might be simpler if we just upload the file as it is, save the path and pass that to the queue. Later, when the worker gets to resizing, it would have to read from the external disk(s3), save the resized files and update the model.

2 likes
syropian's avatar

@elena Good point! I was using my strategy on a pet project, where I know multiple servers won't be involved.

Please or to participate in this conversation.