MichaelDaly's avatar

Maximum number of images per upload?

I am in the middle of building a web app which is basically a social network for a niche market. One key feature is the ability for a user to be able to create a post (similar to a blog) and add photos.

I am struggling to figure out and make a decision on the maximum number of images I should allow a user to upload at once.

This is the first time I have built something like this so wondering if anyone has any advice or guidance into handling this sort of post data.

One thing to note is that I have the app functioning and I can upload images and fire a queue event which then handles resizing and file size reduction in the background. So my query today is mainly aimed towards best practices regarding total number of images uploaded in one post.

Maybe I am looking too much into this and there really isn't a problem - if a user wishes to upload 100 images then this is fine but just display a working gif so they are aware of how long the upload process will take.

Mike

0 likes
8 replies
nolros's avatar

I keep mine to 10 due to validation, etc. but manly from a support perspective, painful to really work out what went wrong in a 100 image upload i.e. is memory, size of files, bandwidth, validation, etc. If its is social then small batches is fine, but your call. Large number of images has lead to client memory and bandwidth issues. Keep it to something that is manageable from a support perspective. That said I've not run into any limitation issue on Laravel side and I know there are guys on this forum that do massive batch uploads of csv files.

SergioGregorutti's avatar

For this kind of functionalities I love how Facebook works.

Just take a look at the functionality when you create an album and upload multiple photos and maybe you can get an idea for a solution.

Hope this helps! :)

1 like
thepsion5's avatar

Keep in mind that when working with Apache that if you try and upload individual files larger than the max setting, the PHP will act as if it was never uploaded. If you want to offer multiple uploads, be sure to use client-side validation for the individual images so the server doesn't just ignore the ones that are too big.

2 likes
MichaelDaly's avatar

Thanks for the feedback.

Just an update to say I have client and server side validation in place and I upload the images straight to S3. I have a queue on SQS which receives an array of image names that have been uploaded to S3 for a given post. I have a separate app server to act as a queue worker for the purpose of retrieving, resizing and scaling the images asynchronously. (We will display original size image and use html/css resizing until the queue job has finished processing)

So I have that side of things working fine - or at least I hope it is fine.

I like the idea of allowing 10 uploads as this keeps things basic but it is very restrictive for the user. Do you think this will get in the way of the workflow when creating a post as I expect images and post data to be added in the one form and submitted together. This would mean the user will have to submit the form before adding another 10 images and so on.

hmmm. a bit of a tricky thing to decide on.

Perhaps it might be easier and a more straight forward workflow to upload the images instantly when the user drags and drops them into the post via ajax post (currently, I only show a preview thumbnail of the image and post all data and images when they submit). One downside to this is that the images may need to be stored locally and only sent to S3 during form submit. Maybe that isn't such a bad thing though?

I will have to do further research and perhaps more testing to see what the best solution is.

Thanks again

Mike

nolros's avatar

Mike, I would say start smallish and throttle up. Set some throttle variable you can increase. Make it "new feature" announcement i.e. "pleased to announce you can now upload 50" later x, etc.

MichaelDaly's avatar

Hi again, thanks for the feedback so far. I have a slightly different question now which I am hoping to get feedback on.

My original plan was to save user uploaded images straight to S3, however, I have been testing this and it takes a hell of a lot longer waiting for the images to save to S3 compared to local storage which is a fraction of the time.

So my question is to ask if anyone has advice or guidance with regards to this and whether you can recommend the best approach?

My concerns so far are both user experience and architecture of my app - I would like to keep the app as snappy as possible without compromising the architecture.

Thanks in advance.

Mike

sitesense's avatar

The only thing that I can add to the excellent advice already given, is that if you do allow a huge number of uploads - that may detract from some users experience if they need to wait a long time for their own uploads to be processed from the queue because the person beforehand decided to upload 100 at once.

1 like
MichaelDaly's avatar

Hi, thanks for the reply.

I think I have that one covered as I have the following workflow:

  • User uploads 50 images ( we do quick and basic resize to a maximum of 1100px wide or 720px height )
  • The images are immediately saved to S3 or Local storage (this is where I am struggling to make a decision and therefore require feedback)
  • Job is queued
  • Images will be output to browser in original format as described in step 1 ( we will use browser resizing to get thumbnail and other various sizes )
  • Job is processed which will resize, scale and save 3 alternative image size. 1. Original, 2. Medium for gallery overview, 3. Thumbnail for activity feed preview.
  • If we opt to store image locally, then the job will move the image to s3 in the background
  • Database updated and a flag placed on the image meta data to show the image is processed.
  • Job is deleted from queue
  • View then renders the processed image and respective sizes depending on the area of the site

So, by going through this process I think my workflow is good enough to get the app released in the first instance - however, I have never released an app like this before so it may fall flat on it's face when I get an increase in traffic!

I suppose my question is whether or not to save direct to S3 in the first instance or save locally then move to via a queued job S3. Maybe this is something that can only be answered when the app is released and under load unless someone has experience of this sort of thing and can give a heads up on potential pitfalls or bottlenecks.

Thanks again.

1 like

Please or to participate in this conversation.