I'm using Intervention Image and Flysystem with S3 (creating original and two thumbnails in the cloud) and I am running out of memory on 3mb images (Laravel 4.2). It should not be exhausting 128mb for this kind of operation, but I am never storing the images locally, just manipulating them in memory. Does anyone have insights or blog links to the most efficient way to manipulate images with cloud-only storage? Here is the code I'm using:
$file = Input::file('file');
$input = array('image' => $file);
$rules = array('image' => 'image');
$validator = Validator::make($input, $rules);
if(!$validator->fails()) {
$filename = $file->getClientOriginalName();
$userid = Sentry::getUser()->id;
$cloudPath = $userid.'/';
$cloudThumbnailPath = $cloudPath.'thumbnails/';
$filesystem = Flysystem::connection();
if ($filesystem->put($cloudPath.$filename, Image::make($file)->encode('png'))) {
try {
$large = Image::make($file)->fit(800)->encode('png');
$medium = Image::make($file)->fit(400)->encode('png');
$filesystem->put($cloudThumbnailPath . "large_" . $filename, $large);
$filesystem->put($cloudThumbnailPath . "medium_" . $filename, $medium);
} catch(Exception $e) {
print return Response::json(e->getMessage(), 400);
}
return Response::json('success', 200);
}
return Response::json('error uploading', 400);
} else {
return Response::json('not an image', 403);
}
Check out Cloudinary - they have a free tier which can get you started (and then some) and a Laravel 4 service, which will handle all the transformations for you IN THE CLOUD! They also facilitate storage and delivery using S3.
If you wanted to stick with what you're doing, I would consider offloading (as a first step) the image manipulation to a queue to make sure the user isn't waiting for the processing to happen.
That looks great but I'd need to use my own S3 and custom domains. I just lose too much control using Cloudinary. Great idea for smaller projects though.
I'm going to start by limiting the concurrent uploads and handling and releasing each image memory one thumbnail at a time. I also realized through benchmarking that fit() + changing encoding types consumes more memory. I was hoping for some more suggestions around if it makes sense to use image cache or a temp dir rather than in-memory. I'm sure this comes up with most people using a similar setup.
Yep, I can see how that would be an issue if you're wanting to use a custom domain. Not least of all because that feature only becomes available on the 'advanced' plan at $224/month.
If you were going to use queues to handle the image manipulation, you could save the file to a temporary directory on your server and pass the path to the queue worker. It will manipulate the image as necessary, store the manipulated content and purge the temporary file.
If you want to keep the original image, just move that to it's final location (in storage, or wherever you deem necessary) and work with that file when your worker does the manipulation. At some point, you're going to have to move the image to memory to work with it anyway, so best to do it in the background.
You can also scale the memory limits independently between your web server and command line configurations, so you can increase what's available to the queue worker whilst leaving your web application as it is.
Try using imagick instead of GD, that change may be all that's necessary to prevent memory issues.
// configure with favored image driver (gd by default)
Image::configure(array('driver' => 'imagick'));
This does seem like it would be a good instance for employing a queue though as @deringer explained.
Excellent. Great responses. By doing a combination of imagick and handling images one by one and clearing them from memory it helped out tremendously. A queue isn't needed just yet, but I will keep it in mind if I allow more than 5 images to be processed at once.
Please or to participate in this conversation.