Be part of JetBrains PHPverse 2026 on June 9 – a free online event bringing PHP devs worldwide together.
I am working a lot with the Shopify API - which imposes rate limits on usage per shop (user in my system).
I have been trying different ways to manage the rate limit to avoid hitting errors and wondering if anyone had any opinions?
So far the safest way is for me to execute api calls sequentially for each user and monitor the rate limit, pausing if I need to. However, this means a queue worker could be held up waiting to complete a large amount of calls for a single user.
I have been experimenting with the Redis throttle feature to rate limit my queue jobs based on the user - this works very well, but because it releases the job if it is unable to get a lock the job is basically released to the bottom of the queue and could be some time before it gets tried again at which point it might fail again anyway - I'd like the job which has been rate limited to have priority somehow when the lock is freed - has anyone managed to create this sort of architecture?
Any ideas would be very helpful, thanks in advance!
Would it be possible to rate limit access to your routes using the throttle middleware
Aligning your throttling to the shopify API would prevent you hitting their API limits? If you want to throttle each route independently (rather than by user) you could look at this with code to add your own bespoke middleware for this purpose
Thanks for taking the time to reply, that is a clever idea, but I am actually accessing the Shopify API via a queue job rather than a route.
@alexdover, I think that queue rate limiting might make the most sense in this situation. More details can be found at https://laravel.com/docs/5.7/queues#rate-limiting. It allows you to set a limit at the user, shop, or any other arbitrary level for queue jobs. If the threshold is already hit then it'll put the job back in the queue until a later time and then try to execute again, thereby avoiding the Shopify API limit.
@BRADC - Thanks for taking the time to reply. I have been experimenting with this solution and it is very close to what I need.
The main worry I have is that the job gets released to the back of the queue if the rate limit is being exceeded. That job will then work its way back through the queue and could potentially hit the same limit again and after a few tries will fail.
I'd like to get to a point where a job is prevented from running because of the rate limit, but it then has priority for the next 'slot'
You might want to look at this blog post for queue strategy ideas
https://ohdear.app/blog/how-to-size-scale-your-laravel-queues
One option might be to have a wait time variable based on how many events you have queued versus your rate limit e.g, if you can make 600 requests a minute to shopify the if you receive 601 requests you delay the 601st request by 60 seconds. Thereford you shoukdnt hit the rate limit (I think this I a leaky bucket algorithm)
You could also have a priority queue that if a job ever does fail you could handle the failed job event an move to this queue that yoj work first
Just thinking out loud.
And i just googled "leaky bucket algorithm laravel" and top result was a laracasts shopify answered question
Thank you, the ohdear app blog is very interesting. I think I might look at splitting the jobs across multiple task based queues and your idea about keeping a counter could certainly work, I will just have to try and make sure the jobs for each user/shop are on the same queue with a single worker so that multiple jobs are not attempted at the same time.
Thanks again, appreciate your help!
Please or to participate in this conversation.