Story on how I screwed up (and I need advice on how to fix)
Hi, I have a story to tell and also ask for opinions and suggestion.
I'm creating an advertise website and it has support for image upload, the main image and also the gallery. For the image upload portion I created a kinda of on demand image manipulation using http://glide.thephpleague.com, it works like http://imgix.com/ and https://cloudinary.com/ (kinda) where you can send the size and also the cropping and things like that BUT little did i know that this could give a really bad headache.
Glide has caching support and stuff and it's really nice, once it's loaded the glide will serve the cache. The problem is that for every image request is goes thought a PHP process, meaning that it's not calling the static file itself and it will always (even with cache) be slow, the first load (without the cache) is also kinda horrible. I couldn't notice the problem while I was working on the website cause I always added only one or two ads to test but today while I was trying to put online to show for a few investors and I tried to create 50 ads, the page couldn't load, the server couldn't handle and yeap, I removed the ads and added only one, even with one the page took around 20 seconds to load, which is REALLY bad. I went to my concurrents and noticed that them ALL use a CDN like img.example.com. It took me a long time to implement this image manipulation thing but my lack of experience specially with CDN and things like that made me thought that it was alright and now I have no idea on what to do.
The server is not in production yet, I don't have much time left as I need to put this online to show to a few possible investors and time is passing, I need advice.
The user enters on his panel and has a form so he can create an AD, this form is made using Vue meaning that my server has an API to handle image uploading. When the user uploads the image using this API (one image at a time), the API will get the image, upload to a folder, create the media on the database (https://github.com/plank/laravel-mediable) and the API will return for me the media id and also the image URL from Glide (http://glide.thephpleague.com/). Back on the front, when the API responds with the uploaded image id and the Glide url, I display the image to the user. Glide makes sure to cache this image so next time I request it will load the cache instead. That is pretty much the process.
I'm also using fortrabbit and elephantsql (sass) as a trial for testing purposes and I noticed that it's REALLY slow and perhaps it might be the reason for the bad performance and not only the image serving portion.
Sounds like you're processing images on the fly, always.. instead you ought to process images on upload and write out new files with the processing applied, then serve those static files.
Not sure about glide, but I'm quite sure that intervention image (for example) will handle that.
and yes, if your server is underpowered, it'll always be slow and awful. :)
Yes, I'm thinking about using Cloudinary but I have 0 experience with CDNs and I'm really afraid of going over the free plan having to pay A LOT even before reaching the amount of users that could cover it due to bad usage.
The stack I'm thinking:
DigitalOcean Droplet - $5
DigitalOcean Database (PostgreSQL) - $15
DigitalOcean Database (Redis) - $15
Laravel Forge? (I don't know how Forge would handle DO Databases)
Cloudinary - Free BUT $89 (it hurts - and don't allow custom domain)
It's a classified website, we estimate that we will have at least 1k ads when launched and the images has various sizes and also watermark. The logotype might change in a near future so having the possibility to keep the original image and change the sizing or watermark dynamically is a plus. Cropping images and storing them nowadays I don't see as a smart move, specially for a classified website. I see the competitors, they are all using a CDN with some kind of image manipulation like 500x500 and stuff.
@zefex Well given you have a time constraint, you need to choose a route to go down:
Generate thumbnails at pre-defined sizes that will serve their purpose.
Learn about CDNs, and use a CDN to serve your dynamically-generate thumbnails.
Stump up the cash and use a service like imgix or Cloudinary.
It sounds like you have a bit of knowledge gap, and unfortunately with the time constraint there isn’t really a shortcut that will magically solve your predicament.
You really should be using a CDN to each and serve your dynamically-generated thumbnails. I’d also imagine your potential investors would like your application to be able to handle a page with a small number of thumbnail images before parting with cash to grow your product and the number of images it’s serving.