How to run background api calls and 'sync' with database

Hi guys, I need help on something.

I want my app to fetch data on initial load from a 3rd party api and save it to the database. Then, a background process should run every 5mins (or so) and update the database.

So basically, the data that users will query should come from the database, and not the api call directly.

I don't know if that even makes sense, but I'm struggling to avoid my app reaching the API limit too soon, and also to have data in the database so as to cache frontend requests

Any suggestions??

tisuchi

5 years ago

Level 70

@ftina In that case, you can get help from Jobs.

You can define a job that will fetch records data from API and store in db in every 5 mints interval, in the background.

Once the user tries to fetch data, just fetch from the db, that will reduce your query time, surely.

Cons: In that approach, if your API data change for some reason, you may not get it directly unless your job searches for dissimilar data. Keep it in mind.

ftina

5 years ago

Level 7

thanks

ftina

5 years ago

Level 7

I also found this link: https://divinglaravel.com/dealing-with-api-rate-limits-in-queued-jobs

tisuchi

5 years ago

Level 70

@ftina Yeah, in this article he talks about a bit different about rate limit of the queue.

But you can find a lot of resources-

And a lot more...

Please or to participate in this conversation.