Mfrancik

Mfrancik

Member Since 3 Years Ago

Experience Points 10,200
Experience Level 3

4,800 experience to go until the next level!

In case you were wondering, you earn Laracasts experience when you:

  • Complete a lesson — 100pts
  • Create a forum thread — 50pts
  • Reply to a thread — 10pts
  • Leave a reply that is liked — 50pts
  • Receive a "Best Reply" award — 500pts
Lessons Completed 60
Lessons
Completed
Best Reply Awards 0
Best Reply
Awards
  • start-engines Created with Sketch.

    Start Your Engines

    Earned once you have completed your first Laracasts lesson.

  • first-thousand Created with Sketch.

    First Thousand

    Earned once you have earned your first 1000 experience points.

  • 1-year Created with Sketch.

    One Year Member

    Earned when you have been with Laracasts for 1 year.

  • 2-years Created with Sketch.

    Two Year Member

    Earned when you have been with Laracasts for 2 years.

  • 3-years Created with Sketch.

    Three Year Member

    Earned when you have been with Laracasts for 3 years.

  • 4-years Created with Sketch.

    Four Year Member

    Earned when you have been with Laracasts for 4 years.

  • 5-years Created with Sketch.

    Five Year Member

    Earned when you have been with Laracasts for 5 years.

  • school-session Created with Sketch.

    School In Session

    Earned when at least one Laracasts series has been fully completed.

  • welcome-newcomer Created with Sketch.

    Welcome To The Community

    Earned after your first post on the Laracasts forum.

  • full-time-student Created with Sketch.

    Full Time Learner

    Earned once 100 Laracasts lessons have been completed.

  • pay-it-forward Created with Sketch.

    Pay It Forward

    Earned once you receive your first "Best Reply" award on the Laracasts forum.

  • subscriber-token Created with Sketch.

    Subscriber

    Earned if you are a paying Laracasts subscriber.

  • lifer-token Created with Sketch.

    Lifer

    Earned if you have a lifetime subscription to Laracasts.

  • lara-evanghelist Created with Sketch.

    Laracasts Evangelist

    Earned if you share a link to Laracasts on social media. Please email [email protected] with your username and post URL to be awarded this badge.

  • chatty-cathy Created with Sketch.

    Chatty Cathy

    Earned once you have achieved 500 forum replies.

  • lara-veteran Created with Sketch.

    Laracasts Veteran

    Earned once your experience points passes 100,000.

  • 10k-strong Created with Sketch.

    Ten Thousand Strong

    Earned once your experience points hits 10,000.

  • lara-master Created with Sketch.

    Laracasts Master

    Earned once 1000 Laracasts lessons have been completed.

  • laracasts-tutor Created with Sketch.

    Laracasts Tutor

    Earned once your "Best Reply" award count is 100 or more.

  • laracasts-sensei Created with Sketch.

    Laracasts Sensei

    Earned once your experience points passes 1 million.

  • top-50 Created with Sketch.

    Top 50

    Earned once your experience points ranks in the top 50 of all Laracasts users.

30 Dec
7 months ago

Mfrancik left a reply on Too Many Concurrent HTTP Requests

since im using the api route, there is a thottle on the API middleware.

 protected $middlewareGroups = [
        'web' => [
            \App\Http\Middleware\EncryptCookies::class,
            \Illuminate\Cookie\Middleware\AddQueuedCookiesToResponse::class,
            \Illuminate\Session\Middleware\StartSession::class,
            \Illuminate\View\Middleware\ShareErrorsFromSession::class,
            \App\Http\Middleware\VerifyCsrfToken::class,
            \Illuminate\Routing\Middleware\SubstituteBindings::class,
            \Laravel\Spark\Http\Middleware\CreateFreshApiToken::class,
        ],

        'api' => [
            'throttle:60,1',
            'bindings',
        ],
    ];

This throttle value was the issue.

28 Dec
7 months ago

Mfrancik left a reply on Too Many Concurrent HTTP Requests

from a few more packet captures, it looks as if these jobs are being handled and hitting my server, but a large majority of them are not being added to queue when dispatched. Anybody else had issues with amazon SQS?

I am not exceeding the 300 per second limit.

Mfrancik left a reply on Too Many Concurrent HTTP Requests

yeah, i have 5 workers working off that queue.

im using amazon SQS and it looks like the jobs are not all getting pushed to the queue. the handler im using for these requests hits one of my urls and throws the content straight into the queue.

27 Dec
7 months ago

Mfrancik left a reply on Too Many Concurrent HTTP Requests

When the callback comes in, the request is genuinely put into a queue.

Right now, the request comes in, i convert the body of the request to json, then throw that content into a queue.

Unfortunately i do need send the messages this quickly.

I am quite unfamiliar with nginx. I was curious if this had settings for allowing concurrent http requests, and if too many come in, its blocking the ones coming in behind it.

Mfrancik started a new conversation Too Many Concurrent HTTP Requests

I have a platform that sends a large amount of SMS. I may an API request and pass a callback URL which then notifies me of the outcome of the message. If i send 10k text messages over the span of 3 mins, i get bombarded with delivery responses.

Right now when a delivery response comes in, i throw the response in a queue and it gets handled. I believe what is happening is TOO MANY requests are coming in, and I cannot handle all of them.

any suggestions on handling this?

05 Oct
10 months ago

Mfrancik started a new conversation Queue Logic

I have an application thats purpose is to send many text messages in a short period of time. I run into limitations because I can only send 1 text message per second per phone number. There are times the user may want to send 1000 text messages with 5 different phone numbers (but sometimes 500 with one number, 200 with another, etc..). Now I want to concurrently send text messages if the phone numbers are unique.

I could setup seperate queues per phone number, but that will be a nightmare, and expensive.

Using mysql as a queue is an easy solution, and run a cron that groups by phone number and sends every second, but i KNOW this is bad practice and every mysql performance site says dont do this.

maybe a silly question, but if i setup a second database on my server with just one table acting as a queue, will this slow mysql down as a whole on the server?

18 Sep
11 months ago

Mfrancik left a reply on .csv Upload

I dont do any duplicate checking as of the moment, but if i do create a unique field i will be sure to index it. Currently no fields in this table are index aside from the foreign keys by default.

Mfrancik left a reply on .csv Upload

so the code overall is very quick, but when i save() the model, this is what takes so long. its taking about .09 seconds per record, which on 50k records is VERRRYYY slow. Is there a way of I can speed this up?

31 Aug
11 months ago

Mfrancik started a new conversation Php-fpm7.1 Max CPU Usage In Top Command

when making http requests my server is very very slow. The page takes about 8 seconds to load. when using monitor laravel-debugbar the querys are not taking an excessive amount of time compared to actual load time.

below when i reload the brower i see the php-fpm7.1 goes to 100% cpu usage, is this normal?

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

23370 forge 20 0 906288 351988 40328 R 98.4 2.1 2:53.68 php-fpm7.1

Mfrancik started a new conversation Odd Requests In Nginx Logs

on my forge server in /var/log/nginx logs im seeing requests from an ip address in UK (i have no clients there) showing:

/601.7.7 (KHTML, like Gecko) Version/9.1.2 Safari/601.7.7"
5.189.226.102 - - [31/Aug/2018:05:23:03 -0400] "GET /admin/assets/js/views/login.js HTTP/1.1" 301 194 "-" "python-requests/2.19.1"
5.189.226.102 - - [31/Aug/2018:05:29:39 -0400] "GET /admin/assets/js/views/login.js HTTP/1.1" 400 280 "-" "python-requests/2.19.1"

What are the best ways of preventing some of these types of repeat requests other than explicitly whitelisting a firewall? can i block an IP that makes a bunch of repeat requests in a short duration?

29 Aug
11 months ago

Mfrancik left a reply on .csv Upload

its a compliance tool that tells you which phone numbers have requested NOT to be contacted, keeping the marketing firms compliant.

I will look at the three options you have listed, Thank you!

Mfrancik left a reply on .csv Upload

The purpose of the app it to upload phone numbers, then I make api requests to update relevant info on those phone numbers. Typically the users are going to upload files of 25k+ and I'd like to speed the process up if possible. Right now its creating ~300 leads every 7 seconds (im updating user every 7 seconds, this is why i know this), which is incredibly slow.

Mfrancik left a reply on EXTREMELY Slow Server

The primary issue was an excessive number of blade conditionals and querys on the view. As of last week the file upload would work very well, now seems to have drastically slowed to. CPU usage is not maxed out so im not sure what caused it.

https://laracasts.com/discuss/channels/code-review/csv-upload

Mfrancik started a new conversation .csv Upload

I allow the user to map the fields in the csv file to my database fields, i then loop through the file and create a record for each record with eloquent. It takes quite a bit of time but want to use eloquent over directly importing via sql.

thoughts? suggestions?

while(! feof($file))
                {
                    $lead_value = fgetcsv($file);
                    $lead = new Lead();
                    $nullCount = 0;
                    $failBool = false;
                    foreach ($fields as $key => $field_value) {
                        //lead validation
                        if($request["$field_value"] != "null" ){
                            if(is_array($lead_value) && array_key_exists((int)$request["$field_value"], $lead_value)){
                                if($field_value == "billing_state" || $field_value == "phone"){
                                    if($field_value == "billing_state" && strlen($lead_value[(int)$request["$field_value"]]) >= 2) {
                                        $lead["$field_value"] = $this->setStateAbbr($lead_value[(int)$request["$field_value"]]);
                                    }
                                    else{
                                        if($field_value == "phone" && (int)($lead_value[(int)$request["$field_value"]]) >= 10 && is_numeric($lead_value[(int)$request["$field_value"]]+0)){
                                            $lead["$field_value"] = $lead_value[(int)$request["$field_value"]];
                                        }
                                        else{
                                            $lead["$field_value"] = null;
                                            $failBool = true;
                                        }
                                    }
                                }
                                else $lead["$field_value"] = $lead_value[(int)$request["$field_value"]];
                            }
                            else{
                                $nullCount++;
                            }
                        }
                        else{
                            $nullCount++;
                        }
                    }
                    if(!$failBool){
                        $success++;
                        $lead->upload_id = $upload->id;
                        $lead->user_id = $user_id;
                        $lead->waiting_to_check = false;
                        if(isset($lead->phone) && is_numeric($lead->phone) && strlen($lead->phone) == 10) {
                            try {
                                $lead->save();
                            } catch (\Exception $e) {
                                //   echo($e);
                                $failure++;
                            }

                        }else{
                            $lead->delete();
                        }
                    }
                    else{
                        $failure++;
                    }
                }

Mfrancik left a reply on EXTREMELY Slow Server

Locally is also quite slow. On Debug bar I see the queries that were ran (.75 seconds), but still took the page 20 seconds to load using 275mb of memeory

where do i see what is happing on the application side?

Booting (953.86ms) Application (18.63s)

Mfrancik left a reply on EXTREMELY Slow Server

for others:

fixed my using password to access github . ^^

Mfrancik left a reply on EXTREMELY Slow Server

currently i have two sites on the server, i tried moving the issue site off the current server and move to a completely separate one. i pulled the clockwork package and now whenever trying to pull to this same server i get the following:

Fetching origin
error: insufficient permission for adding an object to repository database .git/objects
fatal: failed to write object
fatal: unpack-objects failed
error: Could not fetch origin

is this referring to a permission issue from forge accessing github?

Mfrancik left a reply on EXTREMELY Slow Server

ill add an index on created at. Locally the site performs well, it was sort of a sudden thing where the website started slowing down. I looked at the processes running on the server, and it doesnt seem anything out of the ordinary. On digital forge, I can see the even CPU usage isnt maxed out and response time is still extremely slow.

ill add break statement.

Locally site works just fine.

Mfrancik left a reply on EXTREMELY Slow Server

In this case the only indexed field would be user_id, which since is a foreign key gets indexed, correct? I have one user where the upload returns 1 record, even to load that it takes 20 seconds.

I will use simplepaginate instead and see if it helps any.

In my .blade.php file i have a lot of blade conditionals, could that drastically decrease performance?

Mfrancik left a reply on EXTREMELY Slow Server

Thanks Bobby, I will split the program into different servers for my workers if needed. But even when no workers are running, and no files are being uploaded, its taking forever. I have one page for example:

$plan = Auth::user()->subscription()->stripe_plan;

        $uploads = Upload::where('user_id', Auth::user()->id)
            ->orderBy('created_at', 'DESC')
            ->where('created_at', '>=', Carbon::now()->subDays(14))
            ->paginate(25);

        $fileUploading = false;
        foreach($uploads as $upload){
            if($upload->uploading == true){
                $fileUploading = true;
            }
        }
        
        return view('uploads.index')
            ->with('plan', $plan)
            ->with('fileUploading', $fileUploading)
            ->with('uploads', $uploads);

This currently returns 30 uploads, one of which gets queried in blade ({{$uploads->leads()->count()} returning querying about 6k records. this page is taking over 30 seconds to load with no workers or uploads occurring right now.

Mfrancik started a new conversation EXTREMELY Slow Server

I am using digital ocean and laravel forge. My server has 6vCPU and 16 gb of ram. I have two sites on forge tied to two separate droplets. One of my databases has aprox 8m records in one of the tables (i've deleted a few million thinking it would help). There are some processes that are heavy on the server, for example i allow the user to upload a csv of contact information (looping the CSV file and creating 'leads' with eloquent now creates about 100 records every 3 seconds). I have one page that returns the csv files the user has uploaded and prints them in a table. if the file is processing, blade querys the total current uploaded, but when pulling from just the uploads tables (returning 100 or so results), the page can take up to 30 seconds to load. I 6 workers in case the users of the platform all perform an action at once, very rare do i have more than 3 concurrent workers performing tasks.

Not only do any pages where a query is run very slow, but even my home pages do not load quickly, which pull nothing from the database.

When i look on digital ocean i notice there are times where my CPU gets maxed out to 100% (not sure if this is a normal process).

Im currently working on separating the two sites on different forge servers so i can atleast pinpoint which one is causing me issues, but its become quite a task moving the database (this should be complete tomorrow).

Although this is a bit vague, if somebody can point me in the right direction so i can begin troubleshooting, that would be excellent.

13 Aug
1 year ago

Mfrancik started a new conversation Wix Landing Page With Laravel Project

I am using forge and digital ocean to host my laravel project. Id like to build a landing page with something like Wix or Wordpress but the landing page must be hosted on their servers. How do i have the welcome page show the wix site, but any other pages of my site use my laravel project?

05 Aug
1 year ago

Mfrancik left a reply on Working With Large Tables

My thoughts were to push the data to the archived table based on the created date, so knowing which table to query would be easy.

The archived table would be queried very infrequently, while the 'primary' table is going to be queried almost daily, many times per day.

I will certainly index, but have not done so yet.

Mfrancik started a new conversation Working With Large Tables

I have an application where the user uploads a CSV of records, my app then makes an api request and updates a couple values tied to each of these numbers. this particular table can potentially grow to 100m records.

Would it be wise to push the records from the table that gets queried for the API check to an achieved table which gets queried much less often?

Mfrancik left a reply on Php Artisan Migrate Stuck At The Beginning

Does typing "php artisan" in terminal respond?

Mfrancik left a reply on Running Concurrent Jobs On Prioritized Queues

I created:

2x workers watching high prirority queue 1 worker watching mid priority queue 1x worker watching high and mid priority queue 2x workers watching mid and low priority queue

Mfrancik left a reply on Run Multiple Jobs In Queue At Once

I created three tiers of priority queues which a job is pushed based on pending jobs by that user and using multiple workers to watch each queue.

03 Aug
1 year ago

Mfrancik left a reply on Loop To Update Many Records Is Very Slow

the API is a bit of a bottleneck considering this loop happens a few times.

When you say database transactions, are you referring to using raw mySQL queries, rather than eloquent?

The Lead table will grow very large (currently 5mil records), would it be wise to push these to an 'achieved table' so this process would be quicker pulling the records that need to be checked so the leads table stays smaller?

Mfrancik left a reply on Loop To Update Many Records Is Very Slow

This is the sole purpose of the app, so if many users upload a file, some of the other users will have to wait so i want to optimize this code as much as possible (https://laracasts.com/discuss/channels/forge/running-concurrent-jobs-on-prioritized-queues).

It takes aprox 4min to update 20k records currently.

The Lead table will grow very large (currently 5mil records), would it be wise to push these to an 'achieved table' so this process would be quicker pulling the records that need to be checked so the leads table stays smaller?

I will test both of these options now @Snapey .

Mfrancik started a new conversation Loop To Update Many Records Is Very Slow

My app allows user to upload a large list of phone numbers (potentially 100k records). When the phone numbers are uploaded, my app makes an api request to a service which returns a boolean value tied to each phone number. The api allows for batch delivery of 1,000 phone numbers max.

Using the loop takes a lot of time, would it be quicker to update all values for bool1 that are true all at once?

Suggestions on speeding this process up?

public function handle()
    {
        $numberToBatch = 850;

        $upload = Upload::find($this->upload->id);
        $upload->being_processed = true;
        $upload->update();

        $leadsToScrub = Lead::where('upload_id', $this->upload->id)
            ->where('hasBeenChecked', false)
            ->take($numberToBatch)
            ->get();

        do {
            //prepares batch to scrub
            $scrubBatch = $leadsToScrub->take($numberToBatch);


            $csvNumbers = implode(', ', $scrubBatch->pluck('phone')->toArray());

            $client = new Client();
            $res = $client->request('POST', 'http://*******.com/api/checkNumbers', [
                'form_params' => [
                    'api_key' => $_ENV['api_key'],
                    'numbers' => $csvNumbers,
                ]
            ]);
            $checkedNumbers = json_decode($res->getBody());


            foreach ($checkedNumbers->results as $checkedNumber) {
                $lead = $leadsToScrub->where('phone', $checkedNumber->number)->first();

                if (isset($lead)) {
                    if ($checkedNumber->bool1 == 'Y') {
                        $lead->val1 = true;
                    } else {
                        $lead->val1 = false;
                    }
                    if ($checkedNumber->bool2 == 'Y') {
                        $lead->val2 = true;
                    } else {
                        $lead->val2 = false;
                    }
                    $lead->hasBeenChecked = true;
                    $lead->update();
                }
            }


            $upload = Upload::find($this->upload->id);
            if($upload->being_processed == false){
                return;
            }

            $leadsToScrub = Lead::where('upload_id', $this->upload->id)
                ->where('hasBeenChecked', false)
                ->take($numberToBatch)
                ->get();

        } while (count($leadsToScrub) > 0);

        $upload->being_processed = true;
        $upload->update();

        return;
    }

Mfrancik started a new conversation Running Concurrent Jobs On Prioritized Queues

I have an app which user uploads a list of phone numbers, and api request is made, then updates a value tied to those phone numbers. Its a spark app with many users. If user 1 uploads a list, and process begins to check phone numbers (potentially 100k phone numbers so it takes some time).

I have setup 3 queues for high, mid, and low priority, and 3 workers to run 3 jobs concurrently MAX. Each job and the upload_id, not individual phone numbers.

If I have a job on the high priority queue, and a job in the medium priority queue, job 1 (high priority) begins to process as expected, but the other workers do not pick up jobs from the mid priority queue until the high priority queue job is completed.

If I have 3 workers, and a job in high, medium, and low priority queue, how can i make the workers process each, rather than wait for the high priority job to complete?

13 Jun
1 year ago

Mfrancik left a reply on Run Multiple Jobs In Queue At Once

Im using Twillio for the action of SMS, but still need to have some control on my end for user settings.

even making an asynchronous request to my controller method would achieve the task, but not sure how to do that in laravel

Mfrancik started a new conversation Run Multiple Jobs In Queue At Once

I am building an application which mass texts many phone numbers ( potentially 5k - 10k records). Its a multi tenant app which is the reason id like to run multiple jobs. when the user submits a text blast (many numbers to text), i push one job to the queue, not each individual text. the process of texting this many numbers is time consuming so if user1 submits a blast, then user2 submits a blast a second later, user2 will have to wait until user1 is complete.

If running multiple workers at once to handle the different jobs is not an option, whats another solution to achieve the goal?

26 Mar
1 year ago

Mfrancik left a reply on How To Handle A Long Running Request

Im a bit confused on how to do this.

1.) can I create a table with all needed information to send text? (to number, from number, message).

2.) have a job run to pull each record off the table, send message, delete record from table.

3.) next job runs and repeats....

Mfrancik started a new conversation How To Handle A Long Running Request

I am using twilio to send 1000s of texts to a defined group of phone numbers. What is the best logic to follow when trying to achieve this?

is this a job which should be used with queues?

Can I send an ajax request to the method and allow the user to navigate elsewhere, will this process continue to run if page is reloaded?

10 May
2 years ago

Mfrancik left a reply on Token Mismatch On Forge But Not Locally

My storage/framework/sessions folder was missing

09 May
2 years ago

Mfrancik started a new conversation Token Mismatch On Forge But Not Locally

I am having issues on the login page and getting the token mismatch error. I have verified the CSRF token is being posted with my form. In dev tools I can see a token and laravel_session being created.

I have taken my project completely off my github repo and pushed back up (not sure if this could cause any issues).

Mfrancik left a reply on Database Migration Issues

I actually moved the database to mysql. I was having issues with foreign keys in mysql so I created a new migration files and placed all foreign keys in that file. Thank you above for the efforts.

28 Apr
2 years ago

Mfrancik left a reply on Database Migration Issues

The database.sqlite file is completely empty

Mfrancik left a reply on Database Migration Issues

I attempted this but still receive the same error. I want to reiterate, just to help come to a conclusion, even running just php artisan throws this same error.

Mfrancik left a reply on Database Migration Issues

I have about 45 tables so it'll take me some time. I already banged my head on the wall for a couple hours on this one so I will do the above tomorrow morning (very late here). Thank you.

Mfrancik left a reply on Database Migration Issues

I do not have anything querying accounts in any of my models classes

Mfrancik left a reply on Database Migration Issues

I was unaware of this and makes sense why I experienced some weird activity. I do not have an existing row. I made sure to delete the mysql database on my vagrant machine. I also deleted the database.sqlite file in my database directory and used touch database/database.sqlite to recreate one

Mfrancik left a reply on Database Migration Issues

Accounts migration does come before users table and I do have foreign key set. Everything was working fine before I attempted to switch to mysql

DB::table('users')->insert( [ 'name' => 'Mike Francik', 'email' => '[email protected]', 'userType' => 'Super_Admin', 'account_id' => '0', 'password' => bcrypt('sk84life'), ] );

Mfrancik left a reply on Database Migration Issues

i am not using seeders to do so. I am inserting data in my users table which adds account_id. I am not inserting any data in the accounts table

Mfrancik started a new conversation Database Migration Issues

I have been using sqlite for development since I started my project and attempted to change everything to mysql with no avail. I am trying to build database in sqlite to continue dev but when running the php artisan migrate command I get:

[Illuminate\Database\QueryException] SQLSTATE[HY000]: General error: 1 no such table: accounts (SQL: select * fr om "accounts" limit 1)

[Doctrine\DBAL\Driver\PDOException] SQLSTATE[HY000]: General error: 1 no such table: accounts

[PDOException] SQLSTATE[HY000]: General error: 1 no such table: accounts

Ive changed my database config file to default = sqlite and did not change anything in the sqlite array. Ive tried deleteing the sqlite database and recreating and migrating I get the same issue.

I get the same above two errors when running just php artisan

25 Apr
2 years ago

Mfrancik left a reply on Domain Prefixes

How do I go about creating each db instance? And how do I tell (from my middleware I assume) which database to look at in my controllers?

Mfrancik left a reply on Multiple Account Databases

I am trying to achieve a similar design. were you able to achieve this?

Mfrancik started a new conversation Domain Prefixes

I would like to use prefixes, or subdomains, to identify which database should be accessed within my domain. for example:

site1.domain.com => database1,

site2.domain.com => database2

Can somebody point me in the right direction on what I need to research in order to achieve this?