Be part of JetBrains PHPverse 2026 on June 9 – a free online event bringing PHP devs worldwide together.

AtomCoder's avatar

Bulk Insert Issues

Hi Guys,

I'm reading data from a text file and trying to insert data into a mysql database for each and every line within the text file.

Currently I'm just looping through each line in the file and executing the firstOrCreate method, to avoid inserting duplicate data.

However, some text files have 1000+ lines, so that's 1000+ insert queries (not good).

I'm looking to do something like this:

$data = [
   ['something' => value1, 'my_custom_timestamp' => date], // record 1
   ['something' => value2, 'my_custom_timestamp' => date], // record 2
];

Model::insert($data);

The problem I'm having with this insert is, some files may have the same my_custom_timestamp therefore I want to skip/exclude the duplicate row. This is why I'm currently using the firstOrCreate above.

Is there a way I can do the bulk insert using the firstOrCreate method?

0 likes
3 replies
AtomCoder's avatar

@Snapey The column I need to query for duplicates is "my_custom_timestamp". However, this should only be unique to the data within the file, the same timestamp shoule be allowed from within another file, potentially.

Snapey's avatar

@AtomCoder then copy the data to an array and de-duplicate it before going near the database. Then you can do a bulk update and forget about the exsisting records.

1 like

Please or to participate in this conversation.