My model is pulling the same data three times instead of looping properly.

Published 5 months ago by ChrisF79

First, I'm a beginner and an idiot so please keep that in mind. However, I have been stuck on this all day now (trying to learn) but I need help. I know this is long and I'm really sorry. Now that the stage is set...

I have a console command that connects to 3 different servers and pulls records from each. The problem is that my code is pulling the exact same data 3 times.

Here is my command:

<?php
namespace App\Console\Commands;

use App\Models\GetListing;
use App\Models\RetsFeed;
use App\Models\RetsFeedApi;
use Illuminate\Console\Command;

class DownloadRetsData extends Command
{
    /**
     * The name and signature of the console command.
     *
     * @var string
     */
    protected $signature = 'chores:DownloadRetsData';

    /**
     * The console command description.
     *
     * @var string
     */
    protected $description = 'Downloads the RETS Data';

    public function __construct()
    {
        parent::__construct();
    }

    /**
     * Execute the console command.
     *
     * @return mixed
     */
    public function handle()
    {

        // Download each RetsFeed's Login Info
        $RetsFeeds = RetsFeed::where('active', '=', '1')->get();

        foreach ($RetsFeeds as $RetsFeed)
        {
            $getlistings = new GetListing($RetsFeed);
            $getlistings->downloadListings();
        }
    }

}

When the foreach runs, the $getlistings object does have all 3 server's info inside. So then, the downloadListings() method is:

<?php

namespace App\Models;

use App\Models\RetsFeedApi;
use Illuminate\Database\Eloquent\Model;

class GetListing
{
    public function __construct(RetsFeed $feed)
    {
        $this->feed = $feed;
    }


    public function downloadListings()
    {

        /* Initialize Rets API Object */
        $retsApi = new RetsFeedApi($this->feed);

        /* Run RETS Query for latest listings */
        $rets = $retsApi->findLatest();


        echo $this->feed->board . ": " . $rets->count() . "\n";
    }
}

I threw that echo command in there just for testing and it outputs:

Naples: 1836
Bonita: 1836
FtMyers: 1836

The RetsFeedApi class that is being used is:

<?php

namespace App\Models;

use App\Models\ProcessRetsListings;
use Carbon\Carbon;

class RetsFeedApi {

    public function __construct(RetsFeed $feed)
    {
        $this->feed = $feed;
    }

    public function connectToRetsServer()
    {
        $config = new \PHRETS\Configuration;
        $config->setLoginUrl($this->feed->feedurl);
        $config->setUsername($this->feed->username);
        $config->setPassword($this->feed->password);        

        $rets = new \PHRETS\Session($config);

        $connection = $rets->Login();
        return $rets;
    }



    public function findLatest()
    {
        // connect to server
        $rets = $this->connectToRetsServer();

        // define variables
        $last_query = $this->feed->last_query_at->format("Y-m-d\TH:i:s");
        $tablename = $this->feed->board;


        //query server for all listings after the last_query_at in the database for this feed
        $results = $rets->Search(
            "Property",
                "RES",
                "(
                    (MatrixModifiedDT={$last_query}+)
                )",
            array(
                'QueryType' => 'DMQL2',
                'Count' => 1, // count and records
                'Format' => 'COMPACT-DECODED',
                'Limit' => 1,
                'StandardNames' => 0, // give system names
            )
        );

        return new RetsFeedApiResult($rets, $results);

    }

}


class RetsFeedApiResult {
    public function __construct($rets, $results)
    {
        $this->rets = $rets;
        $this->results = $results;
    }

    public function count()
    {
        return $this->results->getTotalResultsCount();
    }
}
Best Answer (As Selected By ChrisF79)
Cronix

Have you tried logging out at the end of each loop before you login again to get the next batch?

Vilfago
foreach ($RetsFeeds as $RetsFeed)
        {
            $getlistings = new GetListing($RetsFeed);
            $getlistings->downloadListings();
        }

in words :

  • take the first Feed and allocate the listing to $getlistings
  • take the second record and allocate the listing to $getlistings (and delete everything before)
  • etc.

So I think you only have the last item.

Try :

$i = 0;
foreach ($RetsFeeds as $RetsFeed)
        {
            $getlistings[$i] = new GetListing($RetsFeed);
            $getlistings[$i]->downloadListings();
        $i++;
        }

You will have everything in your array.

ChrisF79

@Vilfago I really appreciate it but that produces the same result. If I do a dd(), I do see all of the feeds so it's pulling them out of the database ok. It just isn't working with them it seems.

jlrdw
jlrdw
5 months ago (252,680 XP)

First, I'm a beginner and an idiot

No you are not at least you are trying and have posted some code unlike some folks are just ask how do I do this and don't even try so Bravo for trying.

If three different servers I may be wrong or missed something but are you sure three different server connections are being passed correctly?

Vilfago

@ChrisF79 Yes, sorry, seems I don't read your code properly. Maybe need to sleep ;)

Just another try (but I'm not confident)

What is returned by $rets->Login(); ? You pass it to $connection, but don't use it ?

ChrisF79

@jlrdw Thanks so much. I appreciate it. My wife will tell you both are true however :)

As for the code part of your reply, I do think so. If I run:

dd($RetsFeeds->toArray());

right before my foreach loop, I get this:

array:3 [
  0 => array:9 [
    "id" => 1
    "board" => "Naples"
    "feedurl" => "http://matrix.swflamls.com/rets/login.ashx"
    "username" => "secret"
    "password" => "abcdefg"
    "active" => "1"
    "last_query_at" => "2018-04-22 15:29:07"
    "created_at" => "2018-04-22 15:29:07"
    "updated_at" => "2018-04-22 15:29:07"
  ]
  1 => array:9 [
    "id" => 2
    "board" => "Bonita"
    "feedurl" => "http://matrix.swflamls.com/rets/login.ashx"
    "username" => "secret"
    "password" => "abcdefg"
    "active" => "1"
    "last_query_at" => "2018-04-22 15:29:07"
    "created_at" => "2018-04-22 15:29:07"
    "updated_at" => "2018-04-22 15:29:07"
  ]
  2 => array:9 [
    "id" => 3
    "board" => "FtMyers"
    "feedurl" => "http://matrix.swflamls.com/rets/login.ashx"
    "username" => "secret"
    "password" => "abcdefg"
    "active" => "1"
    "last_query_at" => "2018-04-22 15:29:07"
    "created_at" => "2018-04-22 15:29:07"
    "updated_at" => "2018-04-22 15:29:07"
  ]
]
ChrisF79

@Vilfago That just connects to the server via the PHRets library.

jlrdw
jlrdw
5 months ago (252,680 XP)

Make sure you clear all cache and view cache between each code change. All cache.

ChrisF79

@jlrdw I was hoping it was that easy. Just cleared every bit of cache and getting the same issue.

ChrisF79

As a follow-up to prove to myself it isn't the server, I went in my RetsFeeds table and activated only one server at a time. Running it gave different numbers for each server's row count. Therefore, it's definitely something I'm doing and not something on the server that I'm pulling from.

jlrdw
jlrdw
5 months ago (252,680 XP)

print_r at different stages, somehow in loop it must be getting same server or rets. Make sure any Loop is setup correctly, temp Loop a count or something along with the data and see if the count increases.

Cronix
Cronix
5 months ago (798,940 XP)

One thing to note with rets feeds. All dates on the RETS server are stored as UTC. So when you run queries using dates, you need to convert your local date to UTC for the query to work correctly, or you could get duplicate results when retrieving the newest. This gets more complex though, because when the rets server returns timestamps, they are in the servers local timezone (not UTC). I don't know if you're running into this or not, but thought I'd throw it out there. It took awhile to figure it out.

So when querying, use UTC time. When retrieving the results, they are in the servers local timezone. Also, the rets server might not be physically located where you are. It might actually be in a different timezone than where the physical MLS is located.

And depending on what version of RETS the rets server is using (1.5/1.7.2/1.8 etc) this changes too. Rets 1.7 servers should report their timezone, but not all do, and not all advertise the correct timezone. 1.5 servers don't report the timezone they are in at all, so you have to figure it out by some tricky queries.

I noticed you doing this here and not taking the above into account because you use that date as the query date for last modified:

$last_query = $this->feed->last_query_at->format("Y-m-d\TH:i:s");

Here's a good read: https://github.com/troydavisson/PHRETS/wiki/Timezone-Handling

ChrisF79

@Cronix Yeah, we had to take that into account but we're matching time zones luckily.

Cronix
Cronix
5 months ago (798,940 XP)

Also, it's quite helpful to join the phrets slack channel. It's not super active, but there are usually very knowledgeable people there, including Troy himself (author of phrets).

http://phrets.troda.com/

ChrisF79

@Cronix That's a great idea, thanks. I am on the Google Group which used to be very active but I hardly ever see anything any more.

ChrisF79

@jlrdw I literally went through line by line and did exactly what you said var_dumping everything. It shows that it is using the login information from each server! I am so stumped!

Please sign in or create an account to participate in this conversation.