I assume they are talking about "zipping" up groups of images for upload to reduce the number of uploads. You could then extract the zip files from the server to their destination folder.
Laravel + DigitalOcean Spaces Rate Limits
Hello,
I'm fiddling around with DigitalOcean Spaces and saw their Rate Limits. As per their documentation: https://www.digitalocean.com/docs/spaces/#limits
- You can have 750 requests per IP per second to all Spaces on an account. And what is troubling me...
- 150 PUTs per second to any individual Space
But total 750 requests, so I could do 150PUTs * 5 Spaces = 750/s
My only problem is, I require to upload 130.000.000 images.
They could lift the rates if requested, but they also mention in the documentation.
"If you plan to push more than 200 requests per second to Spaces, we recommend using the Spaces CDN or creating more Spaces."
How would creating more spaces help with the max 750 requests per second ?? Wouldn't it mean less requests per Space ? Eg. with 10 spaces doing 75PUTs/s ? Am I interpreting it wrong ?
Also what does it mean "we recommend using the Spaces CDN". How can you do PUTs with CDN? 2nd also... I got an email from their support team stating the following:
** if you are having issues with Spaces upload speed, we do offer a larger transfer speed by default for Spaces (without limits, up to 50MB/s if not more).
However, we also limit you to roughly 200 uploads per second. If your files are roughly 1Kb each, you would only be able to upload 200Kb/s.
If possible, we recommend bundling up your small files, and then moving them to Spaces. Otherwise, your upload speeds for various small files will be a lot lower. **
You might add now, "why you telling us? ask DO support..." I write this here just in case someone can explain in Laravel terms how to do this "bundling" they wrote me about. Becuase I didn't get an answer back from them. I do indeed have small files I need to upload. Per image there are 5 variations. 1 the original ~400-700KB, down to the smallest ~8-12KB
Any help would be much appreciated.
Please or to participate in this conversation.