How to figure out what is causing high CPU usage spikes
I have built a relatively complicated system (CMS) which consists of two projects - admin site and user site. On admin site we create content and add it to pages. User site reflects those pages. The problem I'm facing is that something is causing 100% cpu usage spikes and I have no idea what that is. I've implemented data caching on crucial parts of the system which in my opinion are most resource-hungry but that only helped a little bit, It's still hitting that 100% cpu usage and the whole site goes down as it is unable to handle requests. We have around 300 users daily, which isn't that many IMO that Laravel app couldn't handle. I'm using blade templates so it's server-side rendering (thinking that a rework to a full Vue frontend might solve it but not sure). Perhaps someone could guide me to a right direction of what steps to take to figure out which function(s) in my code are causing these problems. I've tried using Laravel debugbar but I don't know what parameters are considered "satisfy-able". I've tried enabling long query logging in mysql, that provided no results. We're hosting on a dedicated server
CPU Intel(R) Xeon(R) Gold 6150 CPU @ 2.70GHz (4 core(s)).
OS AlmaLinux 8.4.
8GB RAM
I have a full control over the server but I'm not a qualified system administrator so I'm not sure if there's anything I can do to figure this out by running some terminal commands. I've tried running the "top" command and it just shows mysql and php-fpm as the processes that are consuming all of the CPU resources but that doesn't help me.
I'm suspecting there might be a memory leak somewhere but how may I find it?
@Snapey Thanks, I now have a more clear view of what the parameters should be. However, is 10 really the optimal number for queries no matter the nature of the project? In my case, there are objects with recursive polymorphic relationships. And from what I've read there's really no way to load everything eagerly at once other than load by parts, then go through those parts and figure out what else needs to get loaded. I'll provide an example to clarify, perhaps you might have some insight.
There's an abstract model which can morph into several different object types. Among these object types is a group object, which might have a relationships of many to many with other abstract models. This abstract model also has tags relationship, gallery relationship and other ~5 relationships. Let's say I retrieve 3 objects for a page to display. One of those objects is a group, there was no way to eagerly load this group's objects so here's another query for loading this group's objects. Then within those loaded objects there might be a group object again, which then again results in another query. I've simplified this a lot but just wanted to give a bit more clear view of this specific project.
Here are my current debugbar results for home page.