Be part of JetBrains PHPverse 2026 on June 9 – a free online event bringing PHP devs worldwide together.
Hi,
My laravel application has a download functionality which gets JSON data from the server in user specified date range, converts and splits into 5 day interval CSVs and zip it and download. Right now I have tried Guzzle concurrent batch calls, livewire streamdownload. The files would only be less than 500MB size, but the processing time is so high like even an 8 MB file take 20 secs to download.
I'm very new to the laravel. Can somebody suggest me more efficient way to do this.
Thanks
My laravel application has a download functionality which gets JSON data from the server in user specified date range, converts and splits into 5 day interval CSVs and zip it and download. Right now I have tried Guzzle concurrent batch calls, livewire streamdownload. The files would only be less than 500MB size, but the processing time is so high like even an 8 MB file take 20 secs to download.
All of this takes 20 seconds or just the download?
@jlrdw the fetching, processing and zipping takes 20 secs, then the download happens instantly
There are many steps in your process. I suggest you set out to time which step takes the time.
@Snapey [2025-07-27 15:05:04] local.INFO: Starting optimized SSE download {"type":"json","sessionId":"V97rH2vtdUZpJ45d","url":"http://127.0.0.1:8000/download-stream/json"}
[2025-07-27 15:05:05] local.DEBUG: SSE Event sent {"event":"connected","data":{"message":"Download process started","type":"json"}}
[2025-07-27 15:05:05] local.DEBUG: SSE Event sent {"event":"progress","data":{"progress":0,"message":"Initializing export...","stage":"initializing","stats":[]}}
[2025-07-27 15:05:05] local.DEBUG: SSE Event sent {"event":"progress","data":{"progress":5,"message":"Preparing data batches...","stage":"preparing","stats":[]}}
[2025-07-27 15:05:05] local.INFO: Prepared 15 batches with 2 day intervals
[2025-07-27 15:05:05] local.DEBUG: SSE Event sent {"event":"progress","data":{"progress":10,"message":"Starting parallel data fetch...","stage":"fetching","stats":[]}}
[2025-07-27 15:05:09] local.INFO: Batch 5 completed in 3.72s with 1 trips
[2025-07-27 15:05:11] local.INFO: Batch 2 completed in 5.32s with 2 trips
[2025-07-27 15:05:11] local.INFO: Batch 9 completed in 5.55s with 2 trips
[2025-07-27 15:05:14] local.INFO: Batch 3 completed in 8.48s with 2 trips
[2025-07-27 15:05:14] local.INFO: Batch 4 completed in 9.1s with 4 trips
[2025-07-27 15:05:16] local.INFO: Batch 7 completed in 10.84s with 5 trips
[2025-07-27 15:05:20] local.INFO: Batch 6 completed in 14.8s with 7 trips
[2025-07-27 15:05:21] local.INFO: Batch 10 completed in 16.01s with 8 trips
[2025-07-27 15:05:22] local.INFO: Batch 8 completed in 16.54s with 7 trips
[2025-07-27 15:05:22] local.INFO: Batch 13 completed in 17.1s with 7 trips
[2025-07-27 15:05:24] local.INFO: Batch 12 completed in 18.74s with 6 trips
[2025-07-27 15:05:25] local.INFO: Batch 0 completed in 19.71s with 9 trips
[2025-07-27 15:05:27] local.INFO: Batch 14 completed in 21.74s with 12 trips
[2025-07-27 15:05:28] local.INFO: Batch 11 completed in 22.74s with 10 trips
[2025-07-27 15:05:30] local.DEBUG: SSE Event sent {"event":"progress","data":{"progress":15.333333333333332,"message":"Processed 1/15 batches (82 trips)","stage":"processing","stats":{"completed_batches":1,"total_batches":15,"total_trips":82}}}
[2025-07-27 15:05:30] local.DEBUG: SSE Event sent {"event":"progress","data":{"progress":20.666666666666664,"message":"Processed 2/15 batches (82 trips)","stage":"processing","stats":{"completed_batches":2,"total_batches":15,"total_trips":82}}}
[2025-07-27 15:05:30] local.DEBUG: SSE Event sent {"event":"progress","data":{"progress":26.0,"message":"Processed 3/15 batches (82 trips)","stage":"processing","stats":{"completed_batches":3,"total_batches":15,"total_trips":82}}}
[2025-07-27 15:05:30] local.DEBUG: SSE Event sent {"event":"progress","data":{"progress":31.333333333333332,"message":"Processed 4/15 batches (82 trips)","stage":"processing","stats":{"completed_batches":4,"total_batches":15,"total_trips":82}}}
[2025-07-27 15:05:30] local.DEBUG: SSE Event sent {"event":"progress","data":{"progress":36.666666666666664,"message":"Processed 5/15 batches (82 trips)","stage":"processing","stats":{"completed_batches":5,"total_batches":15,"total_trips":82}}}
[2025-07-27 15:05:30] local.DEBUG: SSE Event sent {"event":"progress","data":{"progress":42.0,"message":"Processed 6/15 batches (82 trips)","stage":"processing","stats":{"completed_batches":6,"total_batches":15,"total_trips":82}}}
[2025-07-27 15:05:30] local.DEBUG: SSE Event sent {"event":"progress","data":{"progress":47.333333333333336,"message":"Processed 7/15 batches (82 trips)","stage":"processing","stats":{"completed_batches":7,"total_batches":15,"total_trips":82}}}
[2025-07-27 15:05:30] local.DEBUG: SSE Event sent {"event":"progress","data":{"progress":52.666666666666664,"message":"Processed 8/15 batches (82 trips)","stage":"processing","stats":{"completed_batches":8,"total_batches":15,"total_trips":82}}}
[2025-07-27 15:05:30] local.DEBUG: SSE Event sent {"event":"progress","data":{"progress":58.0,"message":"Processed 9/15 batches (82 trips)","stage":"processing","stats":{"completed_batches":9,"total_batches":15,"total_trips":82}}}
[2025-07-27 15:05:30] local.DEBUG: SSE Event sent {"event":"progress","data":{"progress":63.33333333333333,"message":"Processed 10/15 batches (82 trips)","stage":"processing","stats":{"completed_batches":10,"total_batches":15,"total_trips":82}}}
[2025-07-27 15:05:30] local.DEBUG: SSE Event sent {"event":"progress","data":{"progress":68.66666666666666,"message":"Processed 11/15 batches (82 trips)","stage":"processing","stats":{"completed_batches":11,"total_batches":15,"total_trips":82}}}
[2025-07-27 15:05:30] local.DEBUG: SSE Event sent {"event":"progress","data":{"progress":74.0,"message":"Processed 12/15 batches (82 trips)","stage":"processing","stats":{"completed_batches":12,"total_batches":15,"total_trips":82}}}
[2025-07-27 15:05:30] local.DEBUG: SSE Event sent {"event":"progress","data":{"progress":79.33333333333334,"message":"Processed 13/15 batches (82 trips)","stage":"processing","stats":{"completed_batches":13,"total_batches":15,"total_trips":82}}}
[2025-07-27 15:05:30] local.DEBUG: SSE Event sent {"event":"progress","data":{"progress":84.66666666666667,"message":"Processed 14/15 batches (82 trips)","stage":"processing","stats":{"completed_batches":14,"total_batches":15,"total_trips":82}}}
[2025-07-27 15:05:30] local.DEBUG: SSE Event sent {"event":"progress","data":{"progress":90,"message":"Processed 15/15 batches (82 trips)","stage":"processing","stats":{"completed_batches":15,"total_batches":15,"total_trips":82}}}
[2025-07-27 15:05:30] local.DEBUG: SSE Event sent {"event":"progress","data":{"progress":90,"message":"Finalizing files...","stage":"finalizing","stats":[]}}
[2025-07-27 15:05:38] local.DEBUG: SSE Event sent {"event":"complete","data":{"progress":100,"message":"Download ready!","stage":"completed","downloadUrl":"http://127.0.0.1:8000/download/file?path=eyJpdiI6InZPOEUrUFI0Ni8xa1Rocm5MT0d2elE9PSIsInZhbHVlIjoiUlEvdGFhRGdNVVJIWi9hYlc5MEdEa20xSWNNM08ydVl3YmJFa25nRG9aY2hsYy9Pdk5aK3JmWnZPTkdaT3pTTXFCNW9RL1Z0d0g5L0VicjJHZGlkeXc9PSIsIm1hYyI6IjQ2MWU3YjYwMmFlN2Q3NGVmN2UyNWI2YTU5YmFlZDc1ZWM4NjVkNDliMDYwNjJjODU0ZTdmMmVlMWJmZTY1MWQiLCJ0YWciOiIifQ%3D%3D","stats":{"total_batches":15,"total_trips":82}}}
this is the log file for downloading data of 82 trips(9 MB file size approx). I'm currently using SSE and livewire . i have a progress indicator. but is not getting updated( i believe because i'm using php artisan serve SSE events are getting buffered and sent all at once , as you can see in the logs.
I'm currently using SSE to leverage this progress indication. I'm open to your suggestions as well for better progress indication and optimizing the processing speed.
I'm dont want to implement queues for this because max file size for now would be 500MB.
DownloadStreamCOntroller.php file
downloadManager.php file
Please or to participate in this conversation.