Be part of JetBrains PHPverse 2026 on June 9 – a free online event bringing PHP devs worldwide together.

meathanjay's avatar

Process Laravel/Redis job from multiple server

We are building a reporting app on Laravel that need to fetch users data from a third-party server that allow 1 request per seconds.

We need to fetch 100K to 1000K rows based on user and we can fetch max 250 rows per request.

So the restriction is:

#1. We can send 1 request per seconds

#2. 250 rows per request

So, it requires 400-4000 request/jobs to fetch a user data, So, loading data for multiple users is very time-consuming and the server gets slow.

So, now, we are planning to load the data using multiple servers, like 4-10 servers to fetch users data, so we can send 10 requests per second from 10 servers.

How can we design the system and process jobs from multiple servers?

Is it possible to use a dedicated server for hosting Redis and connect to that Redis server from multiple servers and execute jobs? Can any conflict/race-condition happen?

Any hint or prior experience related to this would be really helpful.

0 likes
0 replies

Please or to participate in this conversation.