Be part of JetBrains PHPverse 2026 on June 9 – a free online event bringing PHP devs worldwide together.

TuffRivers's avatar

Update Database with API that doesnt have webhook

Hi All,

I have a laravel app that takes data from an API that doesnt have a webhook, so i need to basically fetch the data periodically when ever a new record is created (on a schedule).

This data is paginated in the response, so i have to loop through multiple pages, what is the most efficient way to update my table without downloading the entire data set every time i want to update?

Ideally if there was a webhook i would just download the data once and use a webhook to notify me when a new record has been created.

Thanks!!

0 likes
6 replies
Sinnbeck's avatar

Isnt it possible to fetch data in a specific interval? If you always only go 5 minutes back and query every 5 minutes that should do it (you might want a tiny overlap to ensure records created at 00 seconds aren't missed)

TuffRivers's avatar

One of the datasets i return doesnt have any date listed field or the ability to list a count how can i know what was added in the last five minutes? There is no query builder for this API.

Sinnbeck's avatar

Is there some documentation available? If not it is hard to give any real advise

Sinnbeck's avatar
Sinnbeck
Best Answer
Level 102

Yeah very. Your best bet is using pagination to get the least data possible, but it's not optimal. If they still work on it, you could request some more control (date or datetime)

TuffRivers's avatar

@sinnbeck thanks, i think ill just truncate and reupload every time, its only a few hundred records

Please or to participate in this conversation.