Be part of JetBrains PHPverse 2026 on June 9 – a free online event bringing PHP devs worldwide together.

FrancescoDiMuro's avatar

How to properly make a paginated HTTP call?

Good day everybody. I'm trying to understand what's the proper way to make a paginated call to an external endpoint without having PHP crashing everytime because of the lack of memory. I'm using PHP 8.3.0, with Laravel 11.31, and I'm making a "simple" HTTP GET call to an external endpoint. Here below, the code I'm using:

I'm using Redis as a cache store because I wanted to test it out, since every page has a size of 1+ MB and the data final size is over 800+ MB (on disk, with SQLite and serialization), while it's around 650 MB with Redis (with json_encode). Please note that I wrapped this logic in a command just for testing. Everytime I launch this command, I just see the memory increasing until it reaches the PHP default limit of 128 MB. I was taking a look at different ways how to implement it, and I'm trying to figure it out which one can be the best. The original logic is wrapped in a Job, and even there the PHP memory limit is increased to 512 MB, but I don't want to use it as a solution, since I believe there are better ways to achieve this. Thank you in advance for whoever is going to step in.

Kind regards.

0 likes
4 replies
tykus's avatar

How do you suppose this infinite loop is affecting your experience?

while(true) {

	// do HTTP request and other stuff

}
Snapey's avatar

@tykus there is a break statement, but this might not work if the api returns anything but an empty array when there are no more records

1 like
FrancescoDiMuro's avatar

@tykus I'm not quite sure I understood your statement, but I guess that the while loop is taking a lot of memory. If reading the comment from @snapey , as he rightly pointed out, there is a break statement that works in case the $result is completely empty, or there are no more records in the 'results' array, so that part should work just fine. Do you have any other useful approach to such feature? Thank you.

FrancescoDiMuro's avatar
FrancescoDiMuro
OP
Best Answer
Level 1

Anyway, I did my researches and I've found the Http::pool() method, and I used that to make concurrent paginated call.

Please or to participate in this conversation.