Eloquent cursor() instead of get()

Hello,

I just discover that it's possible to use cursor() instead of get() to retrieve a huge amount of line without any risk to get out of memory.

I'd like to use this cursor to compare the same table from two databases.

$fields = collect(DB::
    connection($database_1)
    ->select('describe '.$table))
    ->map(function ($item) {
        return $item->Field;
    });

$datas_1 = DB::
    connection($database_1)
    ->table($table)
    ->orderByDesc('id')
    ->limit(100000)
    ->cursor();

$datas_2 = DB::
    connection($database_2)
    ->table($table)
    ->orderByDesc('id')
    ->limit(100000)
    ->cursor();

Then I loop over the datas_1 and I check the fields between both tables.

foreach ($datas_1 as $key => $datas_1_item) {
    $diff = false;

    foreach ($fields as $field) {
        if ($field != 'time_execution') {
            if ($datas_1_item->$field != $datas_2->slice($key, 1)->toArray()[0]->$field) {
                $diff = true;
				...
    }
}

The problem is that it's not possible to use this code : $datas_2->slice($key, 1)->toArray()[0]->$field because I don't loop over the datas_2.

Is there a way to load datas_2 progressively like it's the case with the datas_1 using the cursor() ? or without the cursor ?

Any idea is welcome ;).

LaryAI

3 years ago

Level 58

To compare two tables from different databases using cursor(), you can use the following code:

$fields = collect(DB::
    connection($database_1)
    ->select('describe '.$table))
    ->map(function ($item) {
        return $item->Field;
    });

$datas_1 = DB::
    connection($database_1)
    ->table($table)
    ->orderByDesc('id')
    ->limit(100000)
    ->cursor();

$datas_2 = DB::
    connection($database_2)
    ->table($table)
    ->orderByDesc('id')
    ->cursor();

foreach ($datas_1 as $key => $datas_1_item) {
    $diff = false;

    foreach ($fields as $field) {
        if ($field != 'time_execution') {
            $datas_2_item = $datas_2->next();
            if ($datas_1_item->$field != $datas_2_item->$field) {
                $diff = true;
                ...
            }
        }
    }
}

In this code, we are using cursor() to retrieve the data from both tables. We are looping over the data from the first table using foreach loop. Inside the loop, we are using next() method to get the next item from the second table. We are then comparing the fields of both items. If there is a difference, we are setting the $diff variable to true.

1 like

vincent15000

3 years ago

Level 63

@LaryAI That's unfortunately not a good solution, this code generates an error.

Method Illuminate\Support\LazyCollection::next does not exist.

Please or to participate in this conversation.