secondman's avatar

Converting SQL Queries Strategy

Hey everyone!

I've got a very challenging project on my plate that I need some advice on.

I've been tasked to create a full fledged API for OpenCart. They have the beginnings of one in the latest version but it doesn't do much, so my client needs a full version built.

Obviously my product of choice is Lumen, so I've done some initial set up and I've begun the building of my Repositories.

What I'd like to ask is for some advice on a manageable strategy on converting the OpenCart (so called) models, which are nothing more than plain PHP classes and methods of SQL queries, accessed via a DB object.

I'm using Repositories to try and make the process a bit easier so I can use the existing method signatures. In other words I can't change the Controller code that calls the models.

So, now for some code.

This is a small snippet from a very large method, I'm not including the whole thing so as to save space here. This is in catalog/model/catalog/product.php, the getProducts method.

public function getProducts($data = array()) {
    $sql = "
        SELECT
            p.product_id,
            (SELECT AVG(rating) AS total
                FROM " . DB_PREFIX . "review r1
                WHERE r1.product_id = p.product_id
                AND r1.status = '1'
                GROUP BY r1.product_id) AS rating,
            (SELECT price
                FROM " . DB_PREFIX . "product_discount pd2
                WHERE pd2.product_id = p.product_id
                AND pd2.customer_group_id = '" . (int)$this->config->get('config_customer_group_id') . "'
                AND pd2.quantity = '1'
                AND (
                    (pd2.date_start = '0000-00-00' OR pd2.date_start < NOW())
                    AND
                    (pd2.date_end = '0000-00-00' OR pd2.date_end > NOW())
                )
                ORDER BY pd2.priority ASC, pd2.price ASC LIMIT 1) AS discount,
            (SELECT price
                FROM " . DB_PREFIX . "product_special ps
                WHERE ps.product_id = p.product_id
                AND ps.customer_group_id = '" . (int)$this->config->get('config_customer_group_id') . "'
                AND (
                    (ps.date_start = '0000-00-00' OR ps.date_start < NOW())
                    AND
                    (ps.date_end = '0000-00-00' OR ps.date_end > NOW()))
                ORDER BY ps.priority ASC, ps.price ASC LIMIT 1) AS special";

    if (!empty($data['filter_category_id'])) {
        if (!empty($data['filter_sub_category'])) {
            $sql .= " FROM " . DB_PREFIX . "category_path cp
                          LEFT JOIN " . DB_PREFIX . "product_to_category p2c
                          ON (cp.category_id = p2c.category_id)";
        } else {
            $sql .= " FROM " . DB_PREFIX . "product_to_category p2c";
        }

        if (!empty($data['filter_filter'])) {
            $sql .= " LEFT JOIN " . DB_PREFIX . "product_filter pf
                          ON (p2c.product_id = pf.product_id)
                      LEFT JOIN " . DB_PREFIX . "product p
                          ON (pf.product_id = p.product_id)";
        } else {
            $sql .= " LEFT JOIN " . DB_PREFIX . "product p
                          ON (p2c.product_id = p.product_id)";
        }
    } else {
        $sql .= " FROM " . DB_PREFIX . "product p";
    }

    ....

    ....

    $product_data = array();

    $query = $this->db->query($sql);

    foreach ($query->rows as $result) {
        $product_data[$result['product_id']] = $this->getProduct($result['product_id']);
    }

    return $product_data;
}

This is the mess I'm dealing with.

This method simply creates a collection of product_ids based on a given filter array and then executes getProduct on each. Simple enough, though the code is spaghetti.

Anyone willing to help me develop a strategy for converting all these into Eloquent will having my undying admiration and appreciation.

Thanks.

V

0 likes
7 replies
jasrys's avatar

This looks exactly like something I would have written in my pre-laracasts days.

1 like
jasrys's avatar

Speaking generally, I'd first work on finding your models, then defining your relationships, and then building your constraints/queries. It seems at minimum, you have the following models/tables:

Product

Review

  • product_review pivot table

Discount

  • discount_product pivot table

Customer

Group

  • customer_group pivot table

Category

  • category_product pivot table
  • category_subcategory pivot table

SubCategory


Then you can start to define your Eloquent relationships. A Product has many Reviews. A Discount can belong to many Products. A Customer belongs to one (or many) Groups. A SubCategory belongs to one Category. etc.

Once you have working model structure and relationship definitions, you can build out your queries. Your queries may not be the most 'eloquent' given their complexity, but you can use the query builder to mix fluent 'where' or 'whereHas' with 'DB::raw' queries to get what you need.

1 like
ohffs's avatar

Heh - that looks like code I'm dealing with just now - only it's mixed with html and doesn't have any indentation (or where it does, it's pretty much random - woo!).

My process was pretty much to figure out what the query was doing and writing it out in as plain php - not really relationships or the like. Just building arrays/collections, foreach'ing over them etc - just to get it clear that I had the query right and it returned the same data. Then it was a lot clearer to me how to set up & refactor the relations with eloquent. If the queries you're dealing with a readable though you can probably skip some of that - though for me it was invaluable to build up a picture of how different bits of the system related.

Below is the first bit of the code I'm dealing with - this is actually one of the clearer bits too! No comments naturally and the method names very rarely give any clues (this one is in 'function check()' ) :-)

        $query = 'select system_id,jobname,holder,created,thickness, (select round(0.5*sum(runtime)) from ';
        $query .= 'bookedjobs  where userid=vj.userid and created > ';
        $query .= 'unix_timestamp(now())-24*7*3600)+ (select count(system_id) from bookedjobs ';
        $query .= 'where userid=vj.userid and created > unix_timestamp(now())-24*7*3600 and ';
        $query .= '(system_id<vj.system_id or status="ok"))+ (select count(registration=1) ';
        $query .= 'from bookedjobs  where userid=vj.userid and created > ';
        $query .= 'unix_timestamp(now())-24*7*3600 and (system_id<vj.system_id or ';
        $query .= 'status="ok")) + round((created-unix_timestamp(now()))/24/3600/2) ';
        $query .= 'as Priority,status,jobtime,registration,subs from bookedjobs vj ';
        $query .= 'where created > unix_timestamp(now())-24*7*3600 and (status="pending" ';
        $query .= 'or status="loaded") ';
1 like
secondman's avatar

@ohffs

Wow I thought I had it bad ... at least my rewrites are all factored into their own layer. Best of luck with yours :)

@jasrys

Unfortunately I don't have the ability to change any of the existing db structure either, this API needs to work with a native freshly installed OpenCart installation with a couple additional new files being added to direct the built in calls to models to the API vs a local db.

So for my eloquent models I'm already having to add quite a few properties to fit the existing tables.

<?php

namespace App\Models;

use Illuminate\Database\Eloquent\Model;

class Product extends Model {

    /**
     * Opencart table.
     *
     * @var string
     */
    protected $table = 'product';

    /**
     * Table primary key.
     *
     * @var string
     */
    protected $primaryKey = 'product_id';

    /**
     * Opencart doesn't have timestamps by default.
     *
     * @var bool
     */
    public $timestamps = false;

    /**
     * Create carbon instances for date fields.
     *
     * @var array \Carbon\Carbon
     */
    protected $dates = ['date_available', 'date_added', 'date_modified'];

    /**
     * Table columns that are fillable.
     *
     * @var array
     */
    protected $fillable = [
        'model', 'sku', 'upc', 'ean', 'jan', 'isbn', 'mpn', 'location', 'quantity',
        'stock_status_id', 'image', 'manufacturer_id', 'shipping', 'price', 'points',
        'tax_class_id', 'date_available', 'weight', 'weight_class_id', 'length',
        'width', 'height', 'length_class_id', 'subtract', 'minimum', 'sort_order',
        'status', 'viewed', 'date_added', 'date_modified',
    ];

    /*
    |--------------------------------------------------------------------------
    |  Place all your relationships below this comment.
    |--------------------------------------------------------------------------
    */

}

And of course the existing pivot tables aren't named correctly so there's that to deal with. Is there something in a model that I need to do specifically to set a given model as a pivot table? Or do I not make models for those tables?

I appreciate everyone's help a lot.

secondman's avatar

@jasrys

Disregard that last question, I didn't see the docs on defining a pivot table in the relationships, duh.

I do have another question though. In looking through the schema for the OpenCart DB I find product_id in many of the tables.

How do I decide whether a given table is meant to be a pivot table, or if I should create a model for a given table. Take this example for instance.

The below schema has a column for product_id:

DROP TABLE IF EXISTS `oc_coupon_product`;
CREATE TABLE `oc_coupon_product` (
  `coupon_product_id` int(11) NOT NULL AUTO_INCREMENT,
  `coupon_id` int(11) NOT NULL,
  `product_id` int(11) NOT NULL,
  PRIMARY KEY (`coupon_product_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;

But I wouldn't think this would be a pivot table since it has it's own primary key, but it's primary responsibility in the system is to hold data for a coupon specified for a specific product, so perhaps I'm wrong.

I guess what I meant by strategy in my topic title is how do I scan through these classes and determine what relationships to build? Which tables should be used as pivot and which need their own models?

I think once I have all these relationships correct, building the queries should be fairly straight forward using all the tools in Eloquent.

Thanks again for the help.

Hebilicious's avatar

I'm so glad I don't have to do that ... Anyways when I have to do that kind of stuff, I try to "draw" what needs to happen during a request: I want to get some kind of Products, so I need a Product model. I want to get the name of the current user so I need a User model. I want this User to purchase thoses product, so I need an Order model. I need to apply a coupon to a specific product, so I need a Coupon model. However, I don't need a model to apply a coupon to a product, so a model is not necessary and a pivot table useful. (Ideally if I have to create a new coupon/product association, I'd like to do : Product::where('name', 'laravel')->applyCoupon($coupon) or smt)

If the name of the table is stuff_thing, id's say it's intended to be a pivot table.

Anyways good luck !

jasrys's avatar

How do I decide whether a given table is meant to be a pivot table, or if I should create a model for a given table.

But I wouldn't think this would be a pivot table since it has it's own primary key, but it's primary responsibility in the system is to hold data for a coupon specified for a specific product, so perhaps I'm wrong.

A pivot table is needed when you have a one-to-many or many-to-many relationship. In Laravel, your pivot tables are still "models" in the sense you'll use php artisan make:model to create them and they'll have their own extends Model class and file, but they really often serve merely as a way to define relationships between other models. If, for example, a User can have many Orders (a one-to-many relationship), you'll have a User model, an Order model and an OrderUser model (which is your pivot table). The schema might look something like this:

User

  • id (primary key)
  • first_name
  • last_name
  • email

Order

  • id (primary key)
  • description
  • amount

OrderUser

  • id (primary key)
  • order_id (foreign key)
  • user_id (foreign key)

Note that while the OrderUser table/model has a primary key of its own, it's serving a bridge (or "pivot") between the User and Order models.

In your example, the oc_coupon_product table is akin to the OrderUser table/model in the Users/Orders example. It exists to serve as a bridge between (guessing names here) an oc_coupon table/model and an oc_product table/model. Whenever you see a table with just two columns referencing ids on other tables/models, it's a giveaway that you're looking a pivot table.

Please or to participate in this conversation.