High load API - CPU intensive

Hi there,

I need some help with Laravel 8 and PHP8 on project with performance issues.

Here some context : -MySQL database with 2vCPU, 10GB of memory, 85GB for storage, 250 IO/ps -Laravel and PHP hosted on a Docker container with 2vCPU, 8GB of memory, on SSD Database seems pretty quiet without spikes on CPU or memory, and IO/ps have been improved to fight some spikes every 5 minutes.

Laravel is providing an API that users can reach after authentication with a regular /app/login POST request (authentication process is handle by Passport). After users got the token, the process is to update a timestamp and 2 others columns on the same table every 10 seconds. Here what we have done to optimize response time : -multi-stage Docker build -composer install --no-cache --prefer-dist --no-interaction --no-progress --optimize-autoloader --no-ansi (at Docker build) -PHP FPM pool tuning with static mode (max.children around 110) -Nginx tuning with UNIX socket to forward PHP process to FastCGI, we upgrade number of Nginx workers -php artisan route:cache -php artisan view:cache -php artisan config:cache -php artisan artisan optimize -PHP OPCache enabled -JIT compiler enabled

Usually, response time is around 250-300ms, but we are struggling with a perf issue, the API can only handle 100 users per vCPU core. Load tests are performed with Locust (written with Python) : -1st call is authentication with POST API call -other calls are performed every 10 seconds to update timestamps and 2 others columns on the same table

The main bottleneck seems to be the CPU, we are around 90-95% when Locust reach 200 users (20-22 requests/s) We already tried to upgrade the Docker host to use 4vCPU but the bottleneck is still here (100 users per vCPU)

Do you have some tips or tweaks to improve our performances ? Some help will be appreciated

Joseph