Level 37
You are almost certainly running up against some kind of imposed limit from RDS. They created RDS Proxy specifically to deal with situations like this.
Oct 12, 2023
11
Level 6
Laravel Performance
I have a tournament app that is anticipating high traffic this weekend, as the tournament schedule will be announced, potentially bringing in 100,000 users within minutes.
The app is hosted on AWS. I have set up a load balancer with three instances (8 GB RAM, 2 vCPUs, 160 GB SSD). The RDS MySQL database is a single t3.medium instance with 2 vCPUs and 4 GB RAM. All queries are utilizing Laravel Cache; essentially, I cache the results for 30 seconds.
The problem arises when I am testing using:
# sudo ab -n 50000 -c 1000 -t 10 https://my.domain.com
The site basically becomes unresponsive. It appears the issue lies with the database, as I observe more than 300 open connections.
I am seeking comments or suggestions on performance improvements or managing the open database connections. Am I being too aggressive with this testing?
# sudo ab -n 50000 -c 1000 -t 10 https://my.domain.com
Level 80
@extjac Well, this is the thing with scaling. The web server/PHP application is almost never the bottleneck, and you’ll most likely find bottlenecks elsewhere—which you have—such as the number of concurrent database connections allowed.
I’d look at caching things that don’t change frequently. For example, if your tournament schedule is defined and then doesn’t change, it doesn’t make sense to keep fetching it from the database on every request when you can cache it and pull it from the cache instead.
Level 122
If you have caching enabled, what are all these long queries that block your app?
Level 6
@Snapey i am not sure. to be honest. I will enable that on AWS RDS.
But do think this a good way to test/emulate the possible traffic of 100k users?
sudo ab -n 50000 -c 1000 -t 10 https://my.domain.com
Level 122
@extjac probably not from the same server
Level 6
@Snapey i checked slow query and there is not log. It is enable but no log.
Level 6
also tried to run the ab from another server, but the mysql connections jump to 335 Connections. is this how laravel supposed to do it?
Level 80
@extjac Why would moving the database to another server have any effect on the number of connections…?
Level 6
@martinbean i tried running this sudo ab -n 50000 -c 1000 -t 10 https://my.domain.com from another server. I was running from one of the insntaces. The DB is RDS MYSQL,i didnt move it.
Level 74
My guess is that you have som really unperformant queries, if the database is the bottle neck, it's usually an issue with the queries, I suggest taking a look with a tool like clockwork or laravel debugbar, to see what queries are performed, and see if you can speed them up with an index or two.
Level 6
@Tray2 yes , i did that too. I use Laravel debugbar a lot.
Please or to participate in this conversation.