Level 5
Anyone?
Jun 8, 2018
19
Level 5
What can you tell from these test results? Api Pressure test.
What can you tell from these test results?
Help me understand, please. Can this system handle 10000 posts per minute without losing the posted data?
3000 posts per minute
10000 posts per minute:
Level 67
Click on the details tab. The errors in orange aren't getting processed as they're timing out. So no, it can't handle 3,000 requests/min or 10,000 requests/min. It starts timing out earlier when 10,000 requests are sent. Do you really anticipate the site getting 3,000 hits in a minute?
Level 5
It's a super simple hello world page done with Codeigniter.
In your experience, will I get different results if I do this with Laravel?
Level 67
It looks like it handles about ~1,500 requests/minute.
Level 67
in your experience, will I get different results if I do this with Laravel?
Highly doubtful. Most likely the server doesn't have enough processing power/memory to handle more than 1,500 requests/minute.
If you need that much, most likely you'll want to start working with a load balancer and multiple servers.
Level 5
So this means if 3000 people opt-in in 1 minute, some of them are not gonna make it to the database, right?
Level 67
Right. If they are all hitting the database at the same time, which is very unlikely unless you have one hell of a popular app.
Level 5
Well, this is an endpoint that receives emails with webhooks. I might get 100k emails in 15 min from Ontraport https://support.ontraport.com/hc/en-us/articles/217882438-Send-a-Webhook-a-k-a-Ping-URL-
Here are the server specs. Thoughts? What would you upgrade?
Processor: Intel Xeon E3-1230 v5 Quad-Core
OS: Linux OS
WebServer: Apache
centos: CentOs 7 - 64Bit
ControlPanel: cPanel/ Web Host Manager - Fully Managed
ServerSecure: Server Secure
Fantastico: Softaculous
RAM: 16GB DDR4 SDRAM
HD1: 2 x SSD
PrimaryDriveSize: 250 GB SSD
PrimaryRAID: Software RAID 1
HD2: Single SATA HDD (7,200 RPM)
BackupDriveSize: 1 TB SATA HDD (7,200 RPM)
HD3: No Additional Storage Array
NVMe: No NVMe Storage
GUARD: No Remote Backup Needed
SERVERCHASSIS: Standard - Single PSU - No Hot Swap Bays
DDOSProtection: Standard DDoS Attack Protection (up to 2gbps)
Level 67
You'd want to test using real world data, not just a "hello world". Hello world wouldn't even really be hitting the database so it's not a good benchmark. Simulate what you'll actually be doing to get real-world data.
Level 5
I did. Almost exact results. ~5% different.
As a consultant what do you recommend? What to look at next?
Here is the script. It's not perfect.
I wouldn't get picky on the script itself since hello world did almost the same results.
<?php
defined('BASEPATH') OR exit('No direct script access allowed');
class Data_receiver extends CI_Controller {
public function __construct()
{
parent::__construct();
}
public function index()
{
//$this->output->enable_profiler(TRUE);
$this->db->trans_start();
$sql = "SELECT * FROM launch_owners INNER JOIN launch_launches
ON launch_launches.user_id = launch_owners.id
AND launch_launches.id = ?
WHERE launch_owners.id = ?";
$query = $this->db->query($sql, array($_POST['launch_id'], $_POST['user_id']));
if($query->num_rows()!=1)
{
echo "Invalid Request";exit();
}
else
{
$results = $query->result_array()[0];
if($results['status']!='active' || $results['secret_key']!=$_POST['secret_key'])
{
echo "Inactive Account Or Invalid Secret";exit();
}
else
{
//Get launch timezone:
$timezone = $this->custom_functions->get_timezone_by_launch_id($_POST['launch_id']);
$this->custom_functions->set_timezone($timezone);
//Validate Launch
$sql = "SELECT * FROM launch_launches WHERE id = ? AND launch_type = ?";
$query = $this->db->query($sql, array($_POST['launch_id'], 'evergreen'));
if($query->num_rows()==1)
{
//Does Prospect exist under this user?
$sql = "SELECT * FROM launch_prospects WHERE email = ? AND owner_id = ?";
$query = $this->db->query($sql, array($_POST['prospect_email'], $_POST['user_id']));
if($query->num_rows()==1)
{
$prospect_id = $query->result_array()[0]['id'];
}
else
{
$data = array(
'email' => $_POST['prospect_email'],
'owner_id' => $_POST['user_id'],
);
$this->db->insert('launch_prospects', $data);
$prospect_id = $this->db->insert_id();
}
//Delete prospect from this launch if already exist in the seqeunce
$this->db->delete('launch_launch_prospect', array('launch_id' => $_POST['launch_id'], 'prospect_id'=>$prospect_id));
//Add Prospect To Launch
$data = array(
'launch_id' => $_POST['launch_id'],
'prospect_id' => $prospect_id,
'time_added' => date('H:i'),
'date_added' => date('Y-m-d'),
'source' => (isset($_POST['source']) ? $_POST['source'] : '')
);
$this->db->insert('launch_launch_prospect', $data);
$last = $this->db->insert_id();
//Insert the same into stats table
$data = array(
'launch_id' => $_POST['launch_id'],
'prospect_id' => $prospect_id,
'time_added' => date('H:i'),
'date_added' => date('Y-m-d'),
'source' => (isset($_POST['source']) ? $_POST['source'] : '')
);
$this->db->insert('launch_launch_prospect_history', $data);
echo "Added Successfully";
}
else
{
echo "Invalid launch";exit();
}
}
}
$this->output->enable_profiler(TRUE);
$this->db->trans_complete();
}
}
Level 67
Do these need to be processed in real time, or could you just throw them in a queue to be processed by multiple queue workers?
Level 5
It can.
I know Laravel would be GOLD for this.
Question is, can I set it up with Codeigniter fast? I am very beginner with Laravel and don't really have time to learn and set it up for thias unless CI is not capable.
Level 67
I really don't think there would be very much of a difference in speed whether it was Laravel or CI processing this. It's a very simple thing. It has more to do with the server it's running on.
Level 5
I was asking about queue workers. Can I do it with CI?
Level 67
I don't know, I haven't used CI in years and don't know if they have a queue system.
With laravel queue workers, they respond very fast because an entire laravel instance gets loaded in memory (one per worker), and it stays there all loaded, and all queued tasks get sent to that. So it's very fast. Without queues, a request comes in, laravel/codeigniter has to load all dependencies and bootstrap itself from scratch, and then process the request, and then completely terminate. Another request comes it, it has to do it all over again. With queues, it only loads and bootstraps once.
1 like
Level 5
Goddamn Cronix I might have to learn it then.
That still doesnt solve the live traffic issue.
Thanks for helping dude.
Level 50
@behnampmdg3 Is there any need to do select * for every query?
Level 5
Hi;
Not the right time to optimize the code just yet. i tell you why:
The server was updated, now it works flawlessly with 10000 requests per minute. https://ldr.io/2HzL5GB
Updates and what they added:
oPcache
php-fpm
-
update apache configuration in the Pre VirtualHost Include
-
moved onto PHP7.2
-
switched PHP handler to PHP-FPM
-
Included cache directives in the php.ini so that it was being utilized However, 20000 R/PM still messes up.
I am not sure about the cloud, I am using https://www.liquidweb.com/products/dedicated/
They said they can optimize more.
Second, I am gonna break the system into 3 parts:
If I am gonna move the API, members area and timers to separate locations. By locations, I mean somewhere they won't interfere with each other. It maybe sub-domain? Kicken mentioned different database server? Is sub-domain gonna do that?
Once I get CI hello world to work 100% with 500 requests per second, then I get into optimizing the app's code.
For this specific project I picked Codeigniter, the main reason I picked Codeigniter is the speed and simplicity. It does outperform all other frameworks. It's simple. In comparison, Laravel is a beast no doubt but I want simple for this for now.
Once they say "this is as good as the server gets, then I tell them about separating database server from the server. I'll also suggest:
"You can get a strong server to host your database, then multiple smaller VPS's to handle the web traffic. With some providers, you can even set things up so that during peak load times you can spin up extra VPS's to handle the extra traffic, then get rid of them after traffic dies down."
Level 1
I think you need to go line by line in your script and to think for a second on each line and what is it doing. For example, you override the constructor for no reason, the only call there is to the parent constructor, the code in general is one big if-else maze which makes it error-prone. I think you can squeeze performance by optimising it.
You are right about laravel being heavy and for the task you are after probably you dont need any framework at all, plain php with pdo would be more than enough. However if you look for something easy and not heavy, there is a subset framework of laravel, its called lumen that will probably be just the right thing. Also there is one framework called phalcon that I think is the only php framework written as C modules which makes it probably the fastest out there.
Please or to participate in this conversation.