Level 73
There is no way to do that, and why would you?
You need to paginate or chunk, but all 50M at less than 2 seconds, not do able.
1 like
Oct 12, 2023
6
Level 1
Efficient Loading of 50 Million Records in DataTable with Sub-2-Second Performance using Laravel 10
To load 5 crore (50 million) data entries into a DataTable using server-side rendering in Laravel 10 and achieve loading times within 1 or 2 seconds, if anyone knows how to do it, please assist me
Level 102
I assume you want to paginate that data? If not, I don't see any way. Even excel can only handle 1 million rows
Level 1
"I paginate the data. What I mean is, I have a dataset of 50 million transactions. When I list the transaction data on the admin panel, I display only 1000 entries at a time, along with the total data count. Now, when I click on a pagination number, I want to retrieve another set of 1000 records. Could you please assist me with this?"
In this version, the changes make it clear that clicking a pagination number triggers the retrieval of another 1000 records.
Level 73
@leoraj And you have the proper indexes on whatever you sort and filter on?
There is als an issue with pagination that you need to count all the rows everytime, unless you use simple pagination.
Level 102
@leoraj Seems there is a package to work with datatables
https://yajrabox.com/docs/laravel-datatables/10.0
It seems to handle pagination as well
But performance will probably not be awesome as @tray2 points out
Level 14
why would you phrase your question like that?
Please or to participate in this conversation.