ahmeda's avatar

Upload large files to S3 using Laravel 5

I have a file with a size of more than 4MB uploaded to S3 while the user registers in my app.

Here are my ways to upload:

  • Upload directly to S3 without any job queue as follow:
$data = $request->all();

if ($image = $request->file('image')) {
    Storage::disk('s3')->put('uploads/user/', $image, 'public');
    $data['image'] = $image->hashName();
}

$user = User::create($data);

With the above approach, it takes 3 to 4 seconds to upload!

  • I said let's try to create a job queue to make the request faster, so my controller is changed to
$data = $request->all();

$user = User::create($data);

if ($file = $request->file('image')) {
    $file->move(public_path() . '/temporary_images', $imageName = uniqid());
    $extension = $file->getClientOriginalExtension();
    dispatch(new RegisterJob($user, $imageName, $extension));
}

Then in my job:

class RegisterJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public $user, $imageName, $extension;

    public function __construct($user, $imageName, $extension)
    {
        $this->user = $user;
        $this->imageName = $imageName;
        $this->extension = $extension;
    }

    public function handle()
    {
        $imageName = $this->imageName . '.' . $this->extension;

        $imagePath = public_path() . '/temporary_images/' . $this->imageName;

        if (Storage::disk('s3')->put('uploads/user/' . $imageName, fopen($imagePath, 'r+'), 'public')) {

            $this->user->update(['image' => $imageName]);

            File::delete($imagePath);
        }
    }
}

With this approach, it takes 10 to 15 seconds to upload the file with 4MB!!!

What is the best approach to upload an image using job and s3!?

0 likes
6 replies
automica's avatar

is the 10-15 seconds the upload from your dev environment or from a public server?

ahmeda's avatar

My problem with the second way: I just stopped the queue running and then upload an image it takes 10-15 seconds ( that's mean the upload to my server is slow) !

ahmeda's avatar

I thought about it, but my app is just an API for mobile applications!

martinbean's avatar

@jean_ali That doesn’t really change anything. As an API, you’re still expecting your server to be able to handle a file that can be megabytes or even gigabytes in size in a single request, and it’s probably going to either time out or run out of memory before the upload is complete.

You can do something like Vapor where you generate a pre-signed URL server-side to upload files to. You then send this URL to the client (the movie app in your case) and then the mobile app can use an appropriate SDK to do the multi-part uploading from the mobile app to S3. AWS will have SDKs for both Android and iOS.

Please or to participate in this conversation.