AWS EC2 server cpu spike on PHP-FPM process

Have you got any code that could possibly be looping and taking memory?

You obviously can't keep raising the limits since it's just going to fill it.

I haven't deployed any new code in the last few days. No one had any access to server. It was fine until 11.40am this morning and then a sudden jump.

Also the DB is hosted on RDS and the CPU there is high as well. It was around 10% and then jumped at the same time to around 50-60%.

Unfortunately this can have many reasons.

• Is there a table which has suddenly explosively many rows?

• How many request does your server currently handle?

• Is it possible that the server has been hacked?

Maybe this post can help you: https://laracasts.com/discuss/channels/servers/php-fpm-is-using-100-cpu

Might look at the network traffic to your box. Sounds like someone could be flooding you with requests.

It looks like it. I had someone look into it and he said it could be DDOS on EC2 as well as RDS.

I am not sure about the security groups. Can someone help me on that please.

Basically this is the server forge created and I left the security rules as they were.

This is the security group sg-bd34ffda (default) for EC2. Inbound rules:

All TCP TCP 0 - 65535 0.0.0.0/0 All traffic All All sg-bd34ffda (default) All UDP UDP 0 - 65535 0.0.0.0/0

The inbound rules for RDS:

MYSQL/Aurora TCP 3306 sg-bd34ffda (default)

The sites hosted all using https. So I don't even need to listen to http.

Is there any way to mitigate this attack? I am not using Cloudfare or Route53. Just a elastic IP address and A record on specific sites.

Any help will be appreciated.

Here are some security tips from Amazon: https://aws.amazon.com/articles/1233/

Beside these tips I would recommend you to add two factor authentication on your account as well, this will take less than 5 mins: https://aws.amazon.com/blogs/security/improve-the-security-of-your-aws-account-in-less-than-5-minutes/

Be sure you make a backup on regular basis and keep your box up-to-date.

Hi @joydasolive,

From what you describe above it sounds like the instance forge created is full open to the internet which invites mischievous activity. These networks are routinely scanned by botnets for open, exploitable ports so I would definitely work towards making this "least privileged". Depending on the age of the instance, you definitely want your servers in a minimal VPC. This is really an exercise of security groups and network ACLs. They also introduced https://aws.amazon.com/shield/ late last year which I believe you only get if you're in a VPC (not EC2 classic).

Are you using an ELB/ALB load balancer? If so do you see an increase in requests in the ELB logs or on the server (apache/nginx)?

Which instance type are you on? Some of the lower instance types operate on a CPU credit system that I've seen act up when the allowance is exhausted. http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/t2-instances.html

Yes, it was open to the internet to all ports and I think it was most likely UDP flood that caused it. On top of that, lots of rows were deleted from the DB few days earlier than the attack and slowly the queries were taking longer. Both contributed to the high usage of CPU.

I am on VPC and now locked down the access to all ports apart from http(s) and only allow SSH to me and Forge. I also optimized the DB so the query time is quicker.

Will probably look into Shield if it comes back through different medium.

I am on m4.large which should be good enough for my case as I have the DB hosted elsewhere and the number of users are not big enough.

Thanks everyone for your input, really appreciated.