Level 50
Does the api have a rate-limit on sending? That might affect your choice and how you code it.
Oct 6, 2015
3
Level 2
Queue tactics for large iterative tasks
A part of an application we're working on allows a user to upload a file of users to send registration notifications to. So, for example, an admin can take a CSV file with, say, 500 users and then upload it. Each user will then be sent a registration email through an externally-hosts API, not the local mail system.
I want to handle this with a Queue and I thought of 3 options:
- Have one Job (queue) set to handle all 500 emails. When the Job/Queue is triggered, a method would iterate over a data set and send an email for each iteration.
- Have a Queue/Job for each user. When the admin uploads the CSV file, a Queue/Job would be created for each record in the file.
- Combine the 2 approaches. Have 1 queue that is created that then creates another queue for each user. So, if there are 500 users in this file, then 501 Jobs/Queues would be created.
Given the above, can anyone recommend a good approach, even if it lies outside of the 3 options I've mentioned?
Thank you
Level 2
@ohffs There's no limit on sending.
Level 2
Process the entire file in one job request. In the job request, send over a absolute url to the file. In in handler, load the file from the url and process all the records as necessary.
To create a process for each record would slow down the user experience, unless you created a job that would then create individual jobs for each record.
Please or to participate in this conversation.