Are you sure that is not the total processing time of the request?
SELECT count(*) is very slow
- I have a small project, database tables contain max. 30 records ...
- Running a SELECT count (*) query directly in the database is a query of less than 1 millisecond.
- However, in the console (Clockwork) I see completely different times that come crazy to me (31.66ms for querying the number of records for a table that has 4 records ... ???).
How is it possible that the query "select * from" is several times faster than "select count (*) from"? Again, this only happens when querying via Eloquent, directly in the database it is a matter of microseconds.
ExampleProduct
SELECT * FROM `exampleproducts`
6.99 ms
ExampleCategory
SELECT * FROM `examplecategories` WHERE `examplecategories`.`id` in (4)
1.33 ms
ExampleProduct
SELECT count(*) as aggregate FROM `exampleproducts`
31.66 ms
ExampleCategory
SELECT count(*) as aggregate FROM `examplecategories`
17.43 ms
Did you repeat these measures? (I don't see any difference)
do $yorElequentQuery->toSql(); to see what is running in the background if it is the same as you are expecting, check for any events...
@inyansuta can i guess you are also on windows?
This issue comes up depending on how you've got your local development environment set up.
Often switching to homestead speeds up the requests.
I uploaded the demo to the server.
Only the code that is displayed on the page is executed on the link. The duration is always calculated just before and after the end of the code block (always three queries). In addition, when you open the console, Clockwork is enabled, where you can see what queries actually took place.
I do not understand that. The models are absolutely clean, no "with, appends or casts".
@automica I have PHP8 on localhost and server (digital ocean). Same code, same database. At the same time, I have significantly faster times on localhost (windows), which is exactly the opposite for all other projects.
The strange thing is that it is still true that the above behavior applies to queries related to count (SELECE count (*) FROM ...), queries to select records are faster.
The only thing that comes to mind is PHP8, but I also have it on localhost and there is no problem. So the server or database is busy with Digital oceans. Is something like this possible? Or what could the behavior involve?
So you are comparing apples and pears.
You can't compare how long time something in the database database takes against how long a php script that connects to the database and then performs the same query. The php script needs to be parsed and executed on the server which takes some time and if that script connects to the database and then does the same query as the script in the database and returns the result to the server so that it can continue to parse the page.
I suggest you install debugbar instead of trying to compare the whole php execution including the database fetch to running an SQL query directly in the database.
I'm sorry, but I really don't compare apples to pears. See my new post below.
There is no reason to install another debugbar, I wrote that I use Clockwork. In addition, I have the actual duration of queries measured directly in the code.
You can't compare
SELECT * FROM users;
SELECT COUNT(*) FROM users;
To
Route::get('/users', function () {
$milliseconds = round(microtime(true) * 1000);
$users = \App\Models\User::all();
$timeSelect = round(microtime(true) * 1000) - $milliseconds;
$milliseconds = round(microtime(true) * 1000);
$count = \App\Models\User::count();
$timeCount = round(microtime(true) * 1000) - $milliseconds;
return([
'count' => $count,
'timeSelect' => $timeSelect,
'timeCount' => $timeCount,
'users' => $users->toArray(),
]);
});
The top one will always be faster since it doesn't need to connect to the database and run those queries since you are already in the database when you execute the SQL.
Like I said apples and pears.
I don't understand what you mean. The upper example concerns the selection of everything, the lower the selection of the number. And just the second query (for the number of records) is slower than the first ...
Like I said install the debugbar then you will see how long the query it self took and how long the application took.
Take this query for example
select `authors`.*, (select count(*) from `books` inner join `author_books` on `books`.`id` = `author_books`.`book_id` where `authors`.`id` = `author_books`.`author_id`) as `books_count` from `authors` order by `last_name` asc, `first_name` asc
Run in laravel It took 1.81ms in the database according to the debugbar. The whole request on the other hand took 31.52 ms.
Running it from the MySQL prompt
MariaDB [mediabase]> select `authors`.*, (select count(*) from `books` inner join `author_books` on `books`.`id` = `author_books`.`book_id` where `authors`.`id` = `author_books`.`author_id`) as `books_count` from `authors` order by `last_name` asc, `first_name` asc;
+----+------------+-----------+-------------------+---------------------+---------------------+-------------+
| id | first_name | last_name | slug | created_at | updated_at | books_count |
+----+------------+-----------+-------------------+---------------------+---------------------+-------------+
| 5 | Sarah | Ash | ash-sarah | 2020-12-05 10:56:20 | 2020-12-05 10:56:20 | 0 |
| 4 | Anne | Bishop | bishop-anne | 2020-12-05 10:56:20 | 2020-12-05 10:56:20 | 0 |
| 3 | Patricia | Briggs | briggs-patricia | 2020-12-05 10:56:20 | 2020-12-05 10:56:20 | 0 |
| 8 | David | Eddings | eddings-david | 2020-12-05 10:56:20 | 2020-12-05 10:56:20 | 1 |
| 2 | Terry | Goodkind | goodkind-terry | 2020-12-05 10:56:20 | 2020-12-05 10:56:20 | 0 |
| 6 | Faith | Hunter | hunter-faith | 2020-12-05 10:56:20 | 2020-12-05 10:56:20 | 0 |
| 1 | Robert | Jordan | jordan-robert | 2020-12-05 10:56:20 | 2020-12-05 10:56:20 | 3 |
| 7 | Brandon | Sanderson | sanderson-brandon | 2020-12-05 10:56:20 | 2020-12-05 10:56:20 | 1 |
+----+------------+-----------+-------------------+---------------------+---------------------+-------------+
8 rows in set (0.001 sec)
You know https://github.com/itsgoingd/clockwork? ... :)
I use a debugbar and I know how long the overall request takes and how long specific queries to the database take.
Once again. Queries run at one time, in the order in which they are listed.
$mysqli->query("SELECT * from users"); // why is fast?
$mysqli->query("SELECT count(*) from users"); // why is slow?
$mysqli->query("SELECT * from users"); // why is fast?
$mysqli->query("SELECT count(*) from users"); // why is slow?
Inconsistencies in a nutshell
SELECT ALL:
Localhost: Eloquent total: 14ms | Raw total: 13ms
Digital ocean: Eloquent total: 8ms | Raw total: 8ms
SELECT COUNT:
Localhost: Eloquent total: 3ms | Raw total: 2ms
!!! Digital ocean: Eloquent total: 60ms | Raw total: 58ms !!!
Live example: http://138.197.185.91
i'd get onto DO about this. Sounds like an issue with the DB server.
Strange.
Can you do the same test on another server ? (your laptop?)
Which db? Try a vacuum?
I use mysql. But problem is with server. I think...
So from the beginning:
-
I assumed that the problem would not be in my models or code, but I wanted to make sure. So I created a brand new, clean project of the latest version of Laravel.
-
I ran seeder to create 30 users. No other models or data.
-
I added the following code:
Route::get('/users', function () {
$milliseconds = round(microtime(true) * 1000);
$users = \App\Models\User::all();
$timeSelect = round(microtime(true) * 1000) - $milliseconds;
$milliseconds = round(microtime(true) * 1000);
$count = \App\Models\User::count();
$timeCount = round(microtime(true) * 1000) - $milliseconds;
return([
'count' => $count,
'timeSelect' => $timeSelect,
'timeCount' => $timeCount,
'users' => $users->toArray(),
]);
});
And the result? Here:
The count query is slower. What is striking, however, is that it is in the order of tens of milliseconds. The same project on localhost takes me less than 1 millisecond to query for a count.
So where is the buried dog?
I uploaded the same (new, clean Laravel project) to another server, PHP 7.4. Identical behavior.
http://speed.petrzavicak.cz/users
I'm going now to try clean mysql queries without using Eloquent PDO.
You may feel like I'm working a few milliseconds. However, this is a huge problem - I need to read the number of records in the database in the administration, and for example in the first post when requesting the number of records in three tables takes 55ms. I optimize a thousand times more complex queries to get savings, and at the same time a simple query on the number takes so incomprehensibly long ...
For your information on my local server with postgresl
array:3 [▼
"count" => 64
"timeSelect" => 45.0
"timeCount" => 3.0
]
Can you test also in postgresql or do you want I test on mysql?
Do you have the opportunity to try on a sharp server? I'll be grateful ... I've tried two already.
Not necessary, you 'll have different values in absolute but not in relative (ie count should be always shorter than select all)
I added a clean mysql connection. The problem is the same, only with an even bigger difference. On localhost, both queries (for both count and result selection) take less than 1 millisecond. On a sharp server, query for a number in the tens of milliseconds.
In conclusion: this is not a problem of Laravel, Eloquent or PDO, but of the server. Why? Is there anything that can be done?
$mysqli = new mysqli(env('DB_HOST'),env('DB_USERNAME'),env('DB_PASSWORD'),env('DB_DATABASE'));
$sql = "SELECT * from users";
$milliseconds = round(microtime(true) * 1000);
$result = $mysqli->query($sql);
$timeSelectPureMysql = round(microtime(true) * 1000) - $milliseconds;
$sql = "SELECT count(*) from users";
$milliseconds = round(microtime(true) * 1000);
$result = $mysqli->query($sql);
$timeCountPureMysql = round(microtime(true) * 1000) - $milliseconds;
See again http://speed.petrzavicak.cz/users.
.> In conclusion: this is not a problem of Laravel, Eloquent or PDO, but of the server.
As I though and @automica
Optimizing sql is really on the way you write your sql and on the db side, you can read this
https://www.optimizdba.com/optimizdba-blog
In term of performance (large db) postgresql is better than mysql, but for this 'small' test it should have no difference, and it 'll no sense to conclude with this test (except of course for the duration of count and select all, count should be always shorter)
Please or to participate in this conversation.