Level 74
Why are you splitting the csw in the first place?
You should just upload it and then register a job that imports it into the database.
Oct 12, 2021
3
Level 5
Laravel importing split csv files issue
Hello All, I am trying to upload large csv so i am looping through file and split every 1000 lines into seperate files. now i am trying to insert each file like below,
$path = base_path("resources/pendingcontacts/*.csv");
$g = glob($path);
//run 2 loops at a time
foreach (array_slice($g,0,2) as $file) {
//read the data into an array
$data = array_map('str_getcsv', file($file));
$insertlist = [];
//loop over the data
foreach($data as $row) {
$insertlist[] = ['email' => $row[0]];
}
// dump(count($insertlist));
ContactEmail::insert($insertlist);
//delete the file
unlink($file);
}
return true;
Above query is working fine but the problem is above query runs only 2 files and returns true
example i have 10 files in above path but after executing two files it is not running 3rd file instead it returns true. How to alter above code?
Level 5
@tray2 thank you for the reply. if i have more than 200000 emails then it is better to split and write a code in schedular to run all split files to import and now i have directly added all files in loop instead of splice like below,
$g = glob($path);
//run 2 loops at a time
foreach ($g as $file) {
//read the data into an array
$data = array_map('str_getcsv', file($file));
$insertlist = [];
//loop over the data
foreach($data as $row) {
$insertlist[] = ['email' => $row[0]];
}
// dump(count($insertlist));
ContactEmail::insert($insertlist);
// Storage::put('file_'.$key.'.txt',count($insertlist));
//delete the file
unlink($file);
}
Now it will work . but i want to run this files loop in chunks.
Level 11
Chunking is the answer.
https://docs.laravel-excel.com/3.1/imports/chunk-reading.html
1 like
Please or to participate in this conversation.