Level 16
Using --stop-when-empty is a good use case for SQS.
In my case, I just stick with Supervisor on my ec2 instance.
1 like
Jan 16, 2021
2
Level 5
Using AWS SQS for queues and keeping the request count low
Hello,
A script to parse and load user data into the system leverages queue to process user's images. The queue for the application is hosted by AWS SQS. However, there is an unusually high number of requests on the queue. From the stats, came to know that it is because of 'Empty Receives'.
A couple of solutions to reduce the 'Empty Receives':
-
Enable Long Polling option on the SQS queue. However, not sure how this will impact queue processing. Sharing any experience in using this option will help.
-
Run the queue worker periodically with the
--stop-when-emptyoption. Should suit some use-cases but not this one. -
At the end of the script, trigger the queue processing in the background with the
--stop-when-emptyoption using theexecfunction.
Option 3 seems to be the best one for this use-case.
Is there any other better way to achieve this?
Level 5
Yes, @chaudigv. Given the use-case, that's the best option I can think of. Using option 3
Please or to participate in this conversation.