joedawson's avatar

How should I handle time extensive file handling?

Yesterday, I posted another thread regarding my current approach to handling a sometimes time extensive (this varies depending on the size of the file or files - it can upload multiple files) file handling task - where an image is manipulated, send to S3 and is persisted to the database.

Currently, I'm using a command...

// App\Http\Controllers\AdminPhotosController.php

public function store(PhotoRequest $request)
{
    $this->dispatch(new AddPhotoCommand(
        $request->input('female_id'),
        $request->file('photos')
    ));

    return redirect()->route('admin.photos.index');
}

My AddPhotoCommand.php looks a little something like this;

// App\Commands\AddPhotoCommand.php

class AddPhotoCommand extends Command implements SelfHandling, ShouldBeQueued {

    use InteractsWithQueue, SerializesModels;

    protected $female_id, $photos;

    /**
     * Create a new command instance.
     *
     * @return void
     */
    public function __construct($female_id, $photos)
    {
        $this->female_id = $female_id;
        $this->photos = $photos;
    }

    /**
     * Execute the command.
     *
     * @return void
     */
    public function handle()
    {

        foreach($this->photos as $photo) {

            /* ------------------------------------
            #. Generate Thumb
            ------------------------------------ */
            $thumbnail = \Image::make($photo->getRealPath());
            $thumbnail->fit(300)->save(public_path('temp/').$photo->getClientOriginalName());

            /* ------------------------------------
            #. Random String for File
            ------------------------------------ */
            $str = str_random(10).'-';

            /* ------------------------------------
            #. Target Paths
            ------------------------------------ */
            $path       = s3_female_path($this->female_id).$str.$photo->getClientOriginalName().'';
            $thumb_path = s3_female_thumb_path($request->input('female_id', true)).$str.$photo->getClientOriginalName().'';

            /* ------------------------------------
            #. Put to S3
            ------------------------------------ */
            $disk = Storage::disk('s3'); // Mount Disk

            $disk->put($path, file_get_contents($photo)); // Full Image
            $disk->put($thumb_path, $thumbnail); // Thumbnail

            /* ------------------------------------
            #. Persist
            ------------------------------------ */
            $photo = Photo::create([
                'female_id' => $this->female_id,
                'path'      => s3_file_path($path),
                'thumb_path'=> s3_file_path($thumb_path),
                'size'      => $photo->getSize()
            ]);

        }
    }
}

When the command attempts to fire, it returns the following error...

Exception in Queue.php line 91:
Serialization of 'Symfony\Component\HttpFoundation\File\UploadedFile' is not allowed

As I mentioned, I posted another thread yesterday which @usman kindly assisted me with - but using the method suggested of:

$this->dispatch(new AddPhotoCommand(
    $request->input('female_id'),
    $_FILES['photos']
));

Instead of $request->files('photos') doesn't allow me to use methods such as getRealPath() etc. Which I most definitely need.

SO I have a few questions...

  • Is there another way for me to pass the uploaded files to my command?
  • Should I not be using commands?
  • If not, what else should I be using?

I hope someone can help me out here :)

0 likes
4 replies
usman's avatar

@JoeDawson The problems seems to be, that php cannot serialize the active file handles. So you will need to either save the file inside the store method and pass just the path to the command and read back inside the handle method or implement the __sleep and __wakeup methods inside your command and implement the logic for the serialization of the file. You can save the file there and on wakeup read that file back if you don't want to save inside the store method.

1 like
willvincent's avatar
Level 54

I would probably write the file to a temp directory and pass its path off to the queue for processing.. The processing then can resize, push to S3, and remove the temp file.

3 likes
jekinney's avatar

@willvincent great reply as usual. Great spot for event to fire too. Even if you use the default file queue it will pass the required info along and continue with the rest of your code so the user doesn't have to wait or error do to timing out.

Even a simple welcome email I queue it just to shave that few seconds off for the user to see success page.

2 likes
maxdeviper's avatar

Had the same problem when trying to transcode a video using laravel queue. What I discovered was that if you pass a file through the constructor of a command that "shouldBeQueued " , it gives a serialization exception also if you pass in a request object. So I solved mine by passing the path to the file and the value I needed from the request into the constructor and in the handle method created a new File instance using the path that was passed.

Please or to participate in this conversation.