I have a price-checker app that checks multiple 3rd party services and returns prices from them (think it like how Expedia checks a lot of 3rd party websites to return the prices).
Up until now, I've been using ajax and that's how I do the api-calls asynchronously. But as more services I add, I am thinking of if there is a better way to handle this.
With ajax I do:
let availableServices = [] // (comes from backend);
availableServiceIds.forEach(service => {
axios.post(`rates/${service.id}`, requestData) // and show results simultaneously as they arrive
});
But it started making me think that ajax might not be the best solution for this because 1) I'm letting front-end knows the available services, 2) simultaneous ajax calls have limit (of 5 as I read)
I'm thinking of being able to make calls to up to 25 services at one time.
If I want to handle this logic fully on backend, how should I set this up?
So instead of ajax code above, I can add:
axios.post(`rates`, requestData)
and in my backend, I can act accordingly:
public function getRates() {
$services = auth()->user()->availableServices;
foreach ($services as $service) {
// Here i need to somehow make calls to 3rd party services asynchronously
}
}
The best way I can think of is:
- I can create a queue worker called "rates" and do the api calls inside a job per service (so for loop above calls dispatches a job per service). And setup web-socket to deliver it back to the user.
getRates() might return a total number of services (to understand when all rates fetching finished
(I've also seen spatie's async (https://github.com/spatie/async) but I couldn't make it work because my service classes are not simple ones, they are formed of multiple classes with bunch of logic and service providers. I think spatie's async is for smaller tasks. (so couldn't make it work)
If you'd want to setup a logic like that, how would you set this up?
There are 5000 users in the system and this is the vital part of the application. So maybe up to 100 users can do the action at the same time (resulting 25 api calls to 3rd party services by each user). That's why I am doubting about the queue and web-socket approach.
And the reason queue system seems logical was because I could spawn new queue worker servers just for "rating" and scale it on demand.
I'm really curious about the creative ways you'd solve this.